CN109408589A - Method of data synchronization and device - Google Patents
Method of data synchronization and device Download PDFInfo
- Publication number
- CN109408589A CN109408589A CN201811074438.9A CN201811074438A CN109408589A CN 109408589 A CN109408589 A CN 109408589A CN 201811074438 A CN201811074438 A CN 201811074438A CN 109408589 A CN109408589 A CN 109408589A
- Authority
- CN
- China
- Prior art keywords
- sentence
- statement
- action statement
- cancel
- action
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The disclosure provides a kind of method of data synchronization and device, for the data in source database to be synchronized to MPP database, method includes: to obtain preset quantity action statement from the journal file of source database, and every action statement includes the version information of the action statement；Search the cancel statement in preset quantity action statement, and when finding cancel statement, the version information object run sentence older than the cancel statement is determined in preset quantity action statement, and the insertion sentence for the deletion condition for meeting the cancel statement is searched in object run sentence as sentence to be deleted；The sentence to be deleted is deleted from preset quantity action statement, the insertion sentence in remaining action statement is merged, and the data of MPP database are updated according to the action statement after merging.So, it is possible to reduce the number of operations executed when more new data, so that the time delay of more new data is reduced, to improve the consistency of source database and the data in MPP database.
Description
Technical field
This disclosure relates to database technical field, in particular to a kind of method of data synchronization and device.
Background technique
In practical applications, it is sometimes desirable to be synchronized to the data in a database (commonly referred to as " source database ") separately
In one database (commonly referred to as " target database ").Currently, mainly by OracleCDC, (ChangeDataCapture changes
Becoming data capture) technology realizes.
OracleCDC technology mainly includes following two implementation: first, the method for synchronization is directed in source database
Trigger is respectively set in the operations such as insertion, modification, deletion, when above-mentioned variation occurs for the data in source database, can touch
Sending out trigger corresponding is written to changed data in interim table, then passes through ETL (Extract-Transform-
Load, extraction-interaction conversion-load) tool by interim table data update into target database.Second, asynchronous system,
Action statement is parsed from the journal file in source database, then the action statement is repeated in target database, from
And the data in source database are synchronized in target database.
It has been investigated that above two mode for source database and target database be traditional database (such as,
Oracle, SQLserver, MySQL etc.) situation be applicable in, but when target database be distributed parallel (Massively
Parallel Processor, MPP) database when, using above two mode carry out data it is synchronous during, exist tight
The time delay of weight, causes the data in source database and target database inconsistent.
Summary of the invention
In view of this, the disclosure is designed to provide a kind of method of data synchronization and device, at least partly to improve
The above problem.
In order to achieve the above object, the disclosure adopts the following technical scheme that
In a first aspect, the disclosure provides a kind of method of data synchronization, which comprises
Preset quantity action statement is obtained from the journal file of source database, wherein every action statement includes should
The version information of action statement；
The cancel statement in the preset quantity action statement is inquired, and when finding cancel statement, described pre-
If determining the version information object run sentence older than the cancel statement in quantity action statement, and in the object run
The insertion sentence for the deletion condition for meeting the cancel statement is searched in sentence as sentence to be deleted；
The sentence to be deleted is deleted from the preset quantity action statement, and remaining action statement is closed
And and according to the data of the action statement update MPP database after merging.
Second aspect, the disclosure provide a kind of data synchronization unit, and described device includes:
Action statement obtains module, for obtaining preset quantity action statement from the journal file of source database,
In, every action statement includes the version information of the action statement；
Searching module, the cancel statement for inquiring in the preset quantity action statement, and finding deletion language
When sentence, the version information object run sentence older than the cancel statement is determined in the preset quantity action statement,
The insertion sentence for the deletion condition for meeting the cancel statement is searched in the object run sentence as sentence to be deleted；
Removing module, for deleting the sentence to be deleted from the preset quantity action statement；
Merging module, for being merged to remaining action statement；And
Update module, for updating the data of MPP database according to the action statement after merging.
In terms of existing technologies, the disclosure has the advantages that
A kind of method of data synchronization and device that the disclosure provides, obtain preset quantity from the journal file of source database
Action statement, every action statement includes the version information of the action statement.Search deleting in preset quantity action statement
Except sentence, and when finding cancel statement, determine that version information is older than the cancel statement in preset quantity action statement
Object run sentence, and in object run sentence search meet the cancel statement deletions condition insertion sentence as to
Cancel statement.The sentence to be deleted is deleted from preset quantity action statement, remaining action statement is merged, and root
The data of MPP database are updated according to the action statement after merging.So, it is possible to reduce the number of operations executed when more new data,
The time delay for reducing more new data, to improve the consistency of source database and the data in MPP database.
Detailed description of the invention
In order to illustrate more clearly of the technical solution of the disclosure, letter will be made to attached drawing needed in the embodiment below
It singly introduces, it should be understood that the following drawings illustrates only some embodiments of the disclosure, therefore is not construed as to range
It limits, it for those of ordinary skill in the art, without creative efforts, can also be according to these attached drawings
Obtain other relevant attached drawings.
Fig. 1 is a kind of application scenarios schematic diagram that the disclosure provides；
Fig. 2 is a kind of flow diagram for method of data synchronization that the disclosure provides；
Fig. 3 is the sub-step schematic diagram of step S21 shown in Fig. 2；
Fig. 4 is the sub-step schematic diagram of step S22 shown in Fig. 2；
Fig. 5 is the another flow diagram for the method for data synchronization that the disclosure provides；
Fig. 6 is the block diagram for a kind of electronic equipment that the disclosure provides；
Fig. 7 is a kind of functional block diagram for data synchronization unit that the disclosure provides.
Icon: 10- electronic equipment；11- machine readable storage medium；110- data synchronization unit；111- action statement obtains
Obtain module；112- searching module；1121- traverses submodule；1122- first searches submodule；1123- second searches submodule；
113- removing module；114- update module；115- merging module；116- format analysis processing module；12- processor；13- communication unit
Member；20- source database；30- target database.
Specific embodiment
To keep the purposes, technical schemes and advantages of the disclosure clearer, below in conjunction with the attached drawing in the disclosure, to this
Technical solution in open is clearly and completely described, it is clear that and described embodiment is disclosure a part of the embodiment,
Instead of all the embodiments.The component for the disclosure being usually described and illustrated herein in the accompanying drawings can be matched with a variety of different
It sets to arrange and design.
Therefore, the detailed description of the embodiment of the disclosure provided in the accompanying drawings is not intended to limit below claimed
The scope of the present disclosure, but be merely representative of the selected embodiment of the disclosure.Based on the embodiment in the disclosure, this field is common
Technical staff's every other embodiment obtained without creative efforts belongs to the model of disclosure protection
It encloses.
It should also be noted that similar label and letter indicate similar terms in following attached drawing, therefore, once a certain Xiang Yi
It is defined in a attached drawing, does not then need that it is further defined and explained in subsequent attached drawing.
As shown in Figure 1, being a kind of application scenarios schematic diagram that the disclosure provides, wherein electronic equipment 10 can pass through net
Network accesses source database 20 and target database 30, wherein target database 30 is MPP database, specifically be can be
Greenplum, GBase etc., source database 20 can be the traditional databases such as Oracle, SQLserver, MySQL, the present embodiment
It is without limitation.
In the disclosure, electronic equipment 10 can be the equipment for being arbitrarily able to access that simultaneously operating database, such as can be
Server.
As shown in Fig. 2, being a kind of flow diagram for method of data synchronization that the disclosure provides, which is used
In the data in source database 20 are synchronized to target database 30, this method can the electronic equipment 10 as shown in Fig. 1 hold
Row will be below for Oracle, target database are MPP databases, to include the steps that doing to this method by source database 20
It elaborates.
Step S21 obtains preset quantity action statement from the journal file of source database 20, wherein every operation
Sentence includes the version information of the action statement.
Wherein, the journal file can be source database 20 redo log or archive log, including user couple
The action statement of source database 20, the action statement specifically can be insertion (INSERT) sentence, cancel statement (DELETE),
Update (UPDATE) sentence etc..
In the disclosure, every action statement includes the version information for uniquely indicating the action statement, the version
This information can be the sequence of operation number (SystemChangeNumber, SCN), the behaviour of each action statement in the journal file
It is usually incremented by successively according to sequencing to make sequence number, in other words, the sequence of operation number of the preceding action statement of timing is less than
The sequence of operation number of the posterior action statement of timing.
In the disclosure, step S21 may include sub-step as shown in Figure 3.
Step S31 parses the journal file by log analysis tool, obtains the operation language of the source database 20
Sentence.
In the disclosure, the journal file is scanned by log analysis tool, can therefrom extracted once to the source
The action statement that 20 library of data executes.Wherein, the log analysis tool can be OracleLogminer.
Every action statement that OracleLogminer is parsed from the journal file is usually a SQL statement, such as:
Insertion sentence may is that insert into " logminer " " emp " (" empno ", " job ") values
("3333","clerk")；
Cancel statement may is where " empno "=" 3333 " deletefrom " emp "；
Updating sentence may is that update " emp " set job=" manager " where " empno "=3333.
Step S32, in the action statement of the source database 20, from the newest operation of version information of last time interception
Sentence starts to intercept the preset quantity action statement.
In one embodiment, the preset quantity action statement can refer to: update source database from the last time
After 20, newly-increased all action statement in the journal file.
In another embodiment, it is contemplated that after the last time updates source database 20, newly-increased operation in journal file
The quantity of sentence is more, can be updated in batches to newly-increased action statement.In this case, it is possible to flexibly set institute
Preset quantity is stated, such as can be set to 50,100,200 or 500 etc..
In the disclosure, after intercepting the preset quantity action statement every time, the preset quantity item behaviour can be determined
Make the action statement for carrying latest version information in sentence, and the value of the mark version information prestored is updated to the latest edition
Information.When next time carries out intercept operation, it can be intercepted since the action statement for carrying the mark version information described default
Quantity action statement.
Optionally, the mark version information can be pre-stored in electronic equipment 10, and initial value can be empty (NULL).
In implementation process, if electronic equipment 10 detects that the initial value of mark version information for sky, can be determined and currently be traversed
Cancel statement be first cancel statement, so as to directly by the operation language before the cancel statement currently traversed
Sentence is used as object run sentence.
In the disclosure, it is usually wrapped from the preset quantity action statement that the journal file of source database 20 obtains
Include insertion sentence, cancel statement and other some sentences.Wherein, cancel statement is to delete the data for meeting specific deletion condition
Action statement.
In some cases, there may be cancel statements from the preset quantity action statement that journal file obtains, and
There may be the insertion sentence for the deletion condition for meeting the cancel statement before the cancel statement, in other words, in the cancel statement
The data being inserted into insertion sentence before may can be deleted in the cancel statement.
For example, it is assumed that obtaining following action statement from journal file:
SCN1 is inserted into data to tables of data 1；
SCN2 is inserted into data to tables of data 1；
SCN3 is inserted into data to tables of data 2；
SCN4 is inserted into data to tables of data 1；
SCN5 deletes tables of data 1.
In aforesaid operations sentence, the data being inserted into action statement SCN1, SCN2 and SCN4 are in action statement SCN5
In be deleted, thus, update target database 30 in data when, it may not be necessary to the sentence SCN1 that redos,
The corresponding action statement of SCN2 and SCN4, also can be such that the data in source database 20 and target database 30 are consistent.?
In the disclosure, which can be reached by subsequent steps S22- step S24.
For another example, it is assumed that obtain following action statement from journal file:
SCN1 is inserted into the data record of field a=1 to tables of data 1；
SCN2 is inserted into the data record of field a=1, field b=2 to tables of data 1；
SCN3 is inserted into the data record of field c=1, field d=3 to tables of data 2；
SCN4 is inserted into the data record of field c=1 to tables of data 1；
SCN5 deletes tables of data 2；
SCN6 deletes the data statement of a=1 from tables of data 1.
In aforesaid operations sentence, the data of action statement SCN3 insertion have been deleted in action statement SCN5, therefore,
When updating target database 30, it may not be necessary to which redo sentence SCN3.Accordingly, it is inserted in action statement SCN1 and SCN2
The data entered have been deleted in action statement SCN6, then when updating target database 30, it may not be necessary to which redo language
Sentence SCN1 and SCN2.
Step S22 inquires the cancel statement in the preset quantity action statement, and when finding cancel statement,
The version information object run sentence older than the cancel statement is determined in the preset quantity action statement, and described
The insertion sentence for the deletion condition for meeting the cancel statement is searched in object run sentence as sentence to be deleted.
In practical applications, one, two be may include from the preset quantity action statement that journal file obtains
Item or a plurality of cancel statement.Wherein, when there are a plurality of cancel statement, all there may be meet this to delete before every cancel statement
Except the insertion sentence of the deletion condition of sentence.
In the case, in a kind of specific embodiment of the disclosure, step S22 may include step shown in Fig. 4.
Step S41, according to a plurality of action statement described in the order traversal of version information from the old to the new.
Step S42, when the action statement currently traversed is cancel statement, by version in a plurality of action statement
The information action statement older than the cancel statement currently traversed is determined as the object run sentence.
For example, it is assumed that there are the first cancel statement and the second cancel statement totally two deletion languages in a plurality of action statement
Sentence, is likely to occur following situation in practical applications: in the action statement before the first cancel statement, existing and meet described second
The insertion sentence of the deletion condition of cancel statement.In this case, it is possible to by before the cancel statement currently traversed
Action statement is used as object run sentence.Be specifically as follows: when traversing the first cancel statement, by the first cancel statement it
Preceding all action statement are determined as object run sentence；When traversing the second cancel statement, before the second cancel statement
All action statement be determined as object run sentence.Herein it is worth noting that, a certain operation referred in the disclosure
Sentence before sentence each means that the version information sentence older than the action statement, the sentence after a certain action statement are each meant
The version information sentence newer than the action statement.
Step S43 searches the deletion item for meeting the cancel statement currently traversed in the object run sentence
The insertion sentence of part is as the sentence to be deleted.
In another specific embodiment of the disclosure, step S22 can be realized by following sub-step:
According to a plurality of action statement described in the order traversal of version information from the old to the new；In the action statement currently traversed
When for cancel statement, version information is in the cancel statement currently traversed and upper one in a plurality of action statement
Action statement between the secondary cancel statement traversed is determined as the object run sentence；
The insertion for meeting the deletion condition of the cancel statement currently traversed is searched in the object run sentence
Sentence is as the sentence to be deleted.
Still by taking above-mentioned the first cancel statement and the second cancel statement as an example, in this embodiment, when traversing
When two cancel statements, the action statement that version information is between the first cancel statement and the second cancel statement is determined as target
Action statement.For example, it is assumed that the version information of the cancel statement (the second cancel statement) currently traversed is V6, last time traversal
The version information of the cancel statement (the first cancel statement) arrived is V2, then version information ratio V2 new and older than V6 action statement is equal
For object run sentence.
Specifically, by taking version information is sequence of operation SCN as an example, if (second deletes the cancel statement currently traversed
Except sentence) the sequence of operation number be S1, the sequence of operation number of cancel statement (the first cancel statement) that the last time traverses is S5,
It is object run sentence that then sequence of operation S, which meets S5 < S < S1 action statement,.
In this way, the sentence to be deleted can be determined from the action statement between adjacent cancel statement every time.
Step S23 deletes the sentence to be deleted from the preset quantity action statement.
In this way, the number of the insertion operation of required execution when the data of update MPP database can be reduced.
Step S24 merges the insertion sentence in remaining action statement.
Wherein, the remaining action statement refers to: deleting from the preset quantity action statement described to be deleted
Remaining all action statement after sentence.
For example, it is assumed that there is insertion sentence 1 and insertion sentence 2, wherein insertion sentence 1 is to be inserted into field a to tables of data 1
=1, the data record of field b=1, insertion sentence 2 are the data record that field a=1, field c=3 are inserted into tables of data 1.Then
It can determine that insertion sentence 1 and insertion sentence 2 insert data to same tables of data (tables of data 1), therefore, can will be inserted into
Sentence 1 and insertion sentence 2 merge into an insertion sentence.
It has been investigated that MPP database is usually column storage, such as a, b, c, d in above-mentioned example indicate data
A column (list) in table 1, they adhere to different memory spaces separately.To be inserted into sentence insertintotest (a, b, c, d)
For values (1,1, NULL, NULL), when executing the insertion sentence, the corresponding list of field a and field b are required to look up
Corresponding list just can be carried out data insertion, this needs I/O operation twice.In this way, all needing elder generation when executing every insertion sentence
Search corresponding list.
It is related to carrying out data insertion, above-mentioned insertion sentence to the corresponding list of field a in view of above-mentioned insertion sentence 1
2, which also relate to list progress data insertion corresponding to field a, need to can only search a field a by above-mentioned merging process
Corresponding list reduces the number of I/O operation.When combined insertion sentence is more, it can largely reduce and update MPP data
The number of the I/O operation executed when the data in library accelerates renewal speed.In implementation process, what is found inserts to same tables of data
Enter data different insertion sentences may have it is a plurality of.It in one embodiment, can be complete by the different insertion sentences found
Portion merges into an insertion sentence.
Optionally, in an embodiment of the present disclosure, step S24 can be realized by following sub-step:
The identical insertion sentence of the target matrix of operation is merged into an insertion in the remaining action statement
Sentence.
Wherein, target matrix refers to tables of data operated by action statement.It accordingly, will be described in above-mentioned steps
The sentence that is entirely insertable that same tables of data is operated in remaining action statement merges into an insertion sentence.
In another embodiment, step S24 can be realized by following sub-step:
In the remaining action statement, by version information be in the cancel statement currently traversed and it is described on
Between the cancel statement once traversed and operation the identical insertion sentence of target matrix merges into a parenthesis
Sentence.
Particularly, in this embodiment, for the cancel statement traversed for the first time, version information can be deleted in this
Except before sentence, operation the identical insertion sentence of target matrix merges into a poor sentence.
In practical applications, the remaining action statement may be divided into multiple action statement by multiple cancel statements
Group.For example, it is assumed that remaining action statement has SCN1, SCN 2, SCN5, SCN6, SCN7, SCN8 and SCN9, wherein SCN5 and
SCN9 is cancel statement, then it is 2 action statement groups that the remaining action statement, which is deleted sentence SCN5 and SCN9 points, wherein the
1 action statement group includes SCN1 and SCN2, and the 2nd action statement group includes SCN6, SCN7 and SCN8.
When implementing, it can be directed to each action statement group, the insertion of same tables of data will be operated in the action statement group
Sentence merges into an insertion sentence.
Step S24 is described below by a specific example.
Assuming that remaining action statement is such as after deleting the sentence to be deleted in the preset quantity action statement
Under:
SCN2 is inserted into the data record of field b=1, field c=1 to tables of data 1；
SCN3 is inserted into the data record of field b=1, field d=2 to tables of data 1；
SCN4 deletes the data record of field a=1 from tables of data 1；
SCN5 is inserted into the data record of field b=1, field d=1 to tables of data 1；
SCN7 is inserted into the data record of field a=2, field b=1 to tables of data 1；
SCN8 deletes the data record of field d=2 from tables of data 1.
It in implementation process, in one embodiment, can determine in above-mentioned remaining action statement, be inserted into sentence
SCN2, SCN3, SCN5 and SCN7 operation target matrix be tables of data 1, therefore, can will insertion sentence SCN2,
SCN3, SCN5 and SCN7 merge into an insertion sentence.
In another embodiment, it can determine in above-mentioned remaining action statement, before cancel statement SCN4
The target matrix for being inserted into sentence SCN2 and SCN3 operation is tables of data 1, therefore insertion sentence SCN2 and SCN3 can be closed
It and is an insertion sentence；The target data of insertion sentence SCN5 and SCN7 operation between cancel statement SCN4 and SCN8
Table is tables of data 1, thus insertion sentence SCN5 and SCN7 can be merged into an insertion sentence.
Optionally, in the disclosure, for convenient for insertion sentence merge, can be described before executing step S22
Method of data synchronization can also include the steps that Fig. 5 is shown.
Step S51 determines that the insertion sentence operates for every insertion sentence in the preset quantity action statement
Target matrix, and the field that the target matrix includes is determined as aiming field.
Step S52 judges whether there is the insertion sentence aiming field not to be covered.
Step S53, if so, increase the aiming field not to be covered in the insertion sentence, and by increased target
The inserted value of field is set as empty.
Wherein, the inserted value is directed to the value of the increased aiming field insertion.
For example, it is assumed that there are above-mentioned insertion sentences 1 and insertion sentence 2, then in the preset quantity action statement
For insertion sentence 1, it can will be inserted into the tables of data 1 that operates of sentence 1 and be determined as target matrix, and then by 1 (target of tables of data
Tables of data) the field a, field b, field c and the field d that include be determined as aiming field.It is then possible to determine insertion sentence
It does not include aiming field c and aiming field d in 1, to increase aiming field c and aiming field d in insertion sentence 1, and will
The inserted value of increased aiming field c and aiming field d is set as NULL.
Accordingly, it is referred to be handled in the above process for insertion sentence 2, to increase into insertion sentence 2
Aiming field b and aiming field d, and the inserted value of aiming field b and aiming field d are disposed as NULL.
In this case it is assumed that insertion sentence 1 are as follows: insertintotest (a, b) values (1,1)；It is inserted into sentence 2
Are as follows: insertintotest (a, c) values (1,3).Then, after handling according to step S61- step S63, insertion sentence 1 changes
Are as follows: insertintotest (a, b, c, d) values (1,1, NULL, NULL)；Sentence 2 is inserted into change are as follows:
Insertintotest (a, b, c, d) values (1, NULL, 3, NULL).
Then, treated is inserted into sentence 1 and insertion sentence 2 it may be combined in that: insertintotest (a, b, c, d)
Values (1,1, NULL, NULL) (1, NULL, 3, NULL).
Optionally, in the disclosure, above-mentioned steps S61- step S63 can also be before step S51, to described remaining
Insertion sentence in action statement executes.
Step S25 updates the data of MPP database according to the action statement after merging.
Wherein, the insertion sentence after the action statement after merging includes at least the cancel statement and merges.
Accordingly, described surplus if further including updating sentence and query statement in the preset quantity action statement
It also include updating sentence and query statement in remaining action statement, the action statement after the merging also includes updating sentence and looking into
Ask sentence.
It should be appreciated that after the action statement after the merging refers to and merges operation to the remaining action statement
Obtained all action statement.For example, in some embodiments, only being closed to the division statement in remaining action statement
And the action statement after then merging includes: that action statement that the division statement merges and another part are not merged
Action statement.
When implementing, the MPP data may be updated in action statement after the MPP database executes the merging
Data in library (target database 30), so that the data in source database 20 are synchronized in target database 30.
It has been investigated that needing to set in the traffic table of source database 20 in the method for synchronization of Oracle CDC technology
Trigger is set, and trigger needs any operation of source database 20 will be repeated once, by the operation corresponding data
It is recorded in interim table.This can occupy the vast resources of source database 20, impact to the performance of source database 20.?
In the asynchronous system of OracleCDC, need to execute a large amount of insertion operation to MPP database, and MPP database is adapted for
OLAP (On-Line Analysis Processing) supports complicated analysis operation, is unsuitable for doing large batch of insertion operation.
For example, traditional database such as Oracle is per second can to execute secondary insertion operations up to ten thousand, and MPP database is usually per second to hold
The several hundred secondary insertion operations of row.If all operationss sentence of acquisition all executed MPP database one time, meeting when actual motion
There are a large amount of action statement not executed in time, the data so as to cause source database and MPP database (target database) are same
Time delay occurs for step, causes data inconsistent.
And the method for data synchronization that the disclosure provides, it is synchronous to be used for data to obtain action statement from journal file, this
Mode does not have aggressiveness to the resource of source database 20, i.e., will not occupy the resource of source database 20, accordingly, not will lead to source
The reduced performance of database 20.In addition, by deleting the insertion sentence for meeting the condition of the cancel statement before cancel statement,
The number of the insertion operation executed in renewal process can be reduced, to alleviate source database 20 and target database 30 in data
Time delay in synchronizing process, and then alleviate the data inconsistence problems as caused by time delay.
In addition, can reduce in renewal process and execute by the merging to the different insertion sentences for operating same tables of data
I/O operation number, further promote the synchronous speed of data.
A specific example is given below, above-mentioned steps to be described in detail.
Assuming that the preset quantity action statement obtained from journal file is as shown in table 1 below, wherein every a line indicates one
Action statement, serial number indicate the version information of the action statement.
Table 1
In table 1, insert into test (a, b) values (1,2) indicate to tables of data test be inserted into field a=1,
The data record of field b=2.Accordingly, insertintotest (a, c) values (1,3) indicates to be inserted into tables of data test
The data record of field a=1, field c=3；Insertintotest (c, d) values (1,5) indicates to insert to tables of data test
Enter the data record of field c=1, field d=5.The meaning of other similar sentence is referred to above-mentioned example and determines.
Delete from testwherea=1or b=6 indicates the data that field a=1 is deleted from tables of data test
The data record of record and field b=6.
It, can be according to above-mentioned step when electronic equipment 10 obtains above-mentioned preset quantity action statement in implementation process
Rapid S61- step S63 handles the format of the preset quantity action statement, to obtain operation as shown in table 2 below
Sentence.
Table 2
| Serial number | SQL |
| 1 | Insert into test (a, b, c, d) values (1,2, NULL, NULL) |
| 2 | Insert into test (a, b, c, d) values (1,2, NULL, NULL) |
| 3 | Insert into test (a, b, c, d) values (1, NULL, 3, NULL) |
| 4 | Insert into test (a, b, c, d) values (1, NULL, NULL, 4) |
| 5 | Insert into test (a, b, c, d) values (NULL, NULL, 1,5) |
| 6 | Insert into test (a, b, c, d) values (NULL, NULL, 1,6) |
| 7 | Insert into test (a, b, c, d) values (1,7, NULL, NULL) |
| 8 | Insert into test (a, b, c, d) values (1,8, NULL, NULL) |
| 9 | Insert into test (a, b, c, d) values (1,9, NULL, NULL) |
| 10 | Delete from test where a=1or b=6 |
| 11 | Insert into test (a, b, c, d) values (1,10, NULL, NULL) |
| 12 | Insert into test (a, b, c, d) values (1,11, NULL, NULL) |
| 13 | Insert into test (a, b, c, d) values (1,12, NULL, NULL) |
| 14 | Insert into test (a, b, c, d) values (1,13, NULL, NULL) |
| 15 | Insert into test (a, b, c, d) values (1,14, NULL, NULL) |
| 16 | Insert into test (a, b, c, d) values (1,15, NULL, NULL) |
| 17 | Insert into test (a, b, c, d) values (1,16, NULL, NULL) |
| 18 | Insert into test (a, b, c, d) values (1,17, NULL, NULL) |
| 19 | Insert into test (a, b, c, d) values (1,18, NULL, NULL) |
| 20 | Insert into test (a, b, c, d) values (1,19, NULL, NULL) |
| 21 | Delete from test where b=10 |
According to the order traversal of serial number from small to large treated each action statement (action statement i.e. in table 2), when time
When going through to the action statement of serial number 10, it can determine that the action statement is cancel statement, at this point, not having before the cancel statement
There are other cancel statements, can be searched directly in all action statement before the cancel statement and meet deleting for the cancel statement
Except the insertion sentence of condition (that is, a=1 or b=6), it may thereby determine that the insertion sentence of serial number 1-4,7-9 meets serial number
For the deletion condition of 10 cancel statement, so as to which the insertion sentence of serial number 1-4,7-9 is deleted.Wherein, serial number 1-
4, the insertion sentence of 7-9 can serve as the sentence to be deleted in the disclosure.
When traversing the action statement of serial number 21, it can determine that the action statement is cancel statement.In the first side
In formula, it can determine that the last cancel statement traversed is the action statement (rear to claim cancel statement 10) of serial number 10, thus
It is searched in the action statement (i.e. the action statement of serial number 11-20) that can be between 10-21 in serial number and meets cancel statement
The insertion sentence of 21 deletion condition (a=1 or c=1) is as sentence to be deleted, in this way, electronic equipment 10 is by searching for by sequence
It number as sentence to be deleted and is deleted for 11 insertion sentence.
In the second way, when the action statement that traverses serial number 21 and when determining that the action statement is cancel statement,
The action statement of serial number 1-20 can be all determined as to object run sentence and deleted.Below in the first manner in determine
For object run sentence, subsequent process is described in detail.
By above-mentioned first way, remaining action statement is as shown in table 3 below in the preset quantity action statement:
Table 3
It is inserted by searching for that can determine to exist in above-mentioned remaining action statement to the difference of same tables of data insertion data
Enter sentence, specifically 5, the 6 and 11-20 of insertion sentence to tables of data test insertion data.
In one embodiment, insertion 5,6 and 11-20 of sentence can be merged into an insertion sentence, obtained:
Insertintotest (a, b, c, d) values (NULL, NULL, 1,5) (NULL, NULL, 1,6) (1,11, NULL, NULL)
(1,12, NULL, NULL) (1,13, NULL, NULL) (1,14, NULL, NULL) (1,15, NULL, NULL) (1,16, NULL,
NULL) (1,17, NULL, NULL) (1,18, NULL, NULL) (1,19, NULL, NULL).
By the above process, the action statement after finally obtained merging is as shown in table 4 below:
Table 4
In another embodiment, insertion sentence 5 and insertion sentence 6 can be merged, is obtained
Insertintotest (a, b, c, d) values (NULL, NULL, 1,5) (NULL, NULL, 1,6).
Insertion sentence 12- insertion sentence 20 is merged, insert into test (a, b, c, d) values is obtained
(1,11, NULL, NULL) (1,12, NULL, NULL) (1,13, NULL, NULL) (1,14, NULL, NULL) (1,15, NULL,
NULL) (1,16, NULL, NULL) (1,17, NULL, NULL) (1,18, NULL, NULL) (1,19, NULL, NULL).
By the above process, the action statement after finally obtained merging is as shown in table 5 below:
Table 5
Electronic equipment 10 executes the operation after the merging in table 4 or table 5 to target database 30 (i.e. MPP database)
Corresponding data in source database 20 can be synchronized in MPP database by sentence.
Fig. 6 is please referred to, Fig. 6 is the block diagram for a kind of electronic equipment 10 that the disclosure provides, the electronic equipment 10 packet
Include machine readable storage medium 11, processor 12 and communication unit 13.
Directly or indirectly electrically connect between the machine readable storage medium 11, processor 12 and communication unit 13
It connects.It is electrically connected for example, these elements can be realized between each other by one or more communication bus or signal wire.Machine can
Read the machine-executable instruction that the method for data synchronization provided for realizing the disclosure is provided in storage medium 11, processor 12
By reading and executing the machine-executable instruction on machine readable storage medium 11, the method for data synchronization may be implemented.
Communication unit 13 takes for establishing with where the communication connection of external equipment, such as foundation and source database 20 or target data 30
Communication connection between business device.
The machine readable storage medium 11 being mentioned herein can be any electronics, magnetism, optics or other physical stores dress
It sets, may include or store information, such as executable instruction, data, etc..For example, machine readable storage medium 11 may is that
RAM (RandomAccessMemory, random access memory), volatile memory, any kind of storage dish (such as CD, DVD
Deng) perhaps similar storage medium or their combination.
Fig. 7 is please referred to, Fig. 7 is a kind of functional block diagram for data synchronization unit 110 that the disclosure provides, and the data are same
Step device 110 may include that at least one can be stored in machine readable storage Jie in the form of software or firmware (Firmware)
In matter 11 or the software function module that is solidificated in the operating system (OperatingSystem, OS) of electronic equipment 10.Processor
12 can execute the executable module on the machine readable storage medium 11, such as included by the data synchronization unit 110
Software function module and computer program etc..The data synchronization unit 110 may include action statement obtain module 111,
Searching module 112, removing module 113, merging module 114 and update module 115.
Wherein, action statement obtains module 111 for obtaining preset quantity item behaviour from the journal file of source database 20
Make sentence, wherein every action statement includes the version information of the action statement.
In the disclosure, the description as described in action statement obtains module 111 is specifically referred to step S21 shown in Fig. 2
Detailed description executes that is, step S21 can obtain module 111 by action statement.
Searching module 112 is used for the cancel statement inquired in the preset quantity action statement, and deletes finding
When sentence, the version information object run sentence older than the cancel statement is determined in the preset quantity action statement,
The insertion sentence for the deletion condition for meeting the cancel statement is searched in the object run sentence as sentence to be deleted.
In the disclosure, description specifically can retouching in detail with step S22 shown in reference pair Fig. 2 as described in searching module 112
It states, i.e. step S22 can be executed by searching module 112.
Optionally, in the disclosure, searching module 112 may include that traversal submodule 1121, first searches submodule
1122 and second search submodule 1123.
Wherein, traversal submodule 1121 is used to grasp according to preset quantity item described in version information order traversal from the old to the new
Make sentence.
First, which searches submodule 1122, is used for when the action statement currently traversed is cancel statement, in the present count
The version information action statement older than the cancel statement currently traversed is determined as the object run in amount action statement
Sentence, or version information is in the cancel statement currently traversed and upper one in the preset quantity action statement
Action statement between the secondary cancel statement traversed is determined as the object run sentence.
Second lookup submodule 1123 in the object run sentence search meet it is described currently traverse delete
Except sentence deletion condition insertion sentence as the sentence to be deleted.
Removing module 113 from the preset quantity action statement for deleting the sentence to be deleted.
In the disclosure, the description as described in removing module 113 specifically refers to the detailed description to step S23 shown in Fig. 2,
That is step S23 can be executed by removing module 113.
Merging module 114 is for merging the insertion sentence in remaining action statement.
In the disclosure, merging module 114 can be used for executing step S24 shown in Figure 2, about merging module 114
Description specifically can be with the detailed description of reference pair step S24.
In the disclosure, merging module 114 specifically can be used for the target of operation in the remaining action statement
The identical insertion sentence of tables of data merges into an insertion sentence；Alternatively, in the remaining action statement, by version information
Between the cancel statement that the cancel statement currently traversed and the last time traverse and operation number of targets
An insertion sentence is merged into according to the identical insertion sentence of table.
Update module 115 is used to update the data of MPP database according to the action statement after merging, wherein the merging
Action statement afterwards includes at least the insertion sentence after the cancel statement and merging.
In the disclosure, description specifically can retouching in detail with step S25 shown in reference pair Fig. 2 as described in update module 115
It states, i.e. step S25 can be executed by update module 115.
Optionally, in the disclosure, the data synchronization unit 110 can also include format analysis processing module 116.
Format analysis processing module 116 is used for for every insertion sentence in the preset quantity action statement, and determining should
It is inserted into the target matrix of sentence operation, and the field that the target matrix includes is determined as aiming field；Judge whether to deposit
In the insertion sentence aiming field not to be covered, if so, increase the aiming field not to be covered in the insertion sentence, and
Set empty for the inserted value of increased aiming field.
In the disclosure, format analysis processing module 116 can be used for executing step S51, step S52 shown in Fig. 5 and step
Rapid S53, the description as described in format analysis processing module 116 specifically refer to retouching in detail to step S51, step S52 and step S53
It states.
Optionally, in the disclosure, action statement obtains module 111 and specifically can be used for parsing by log analysis tool
The journal file obtains the action statement of the source database, and in the action statement of the source database, from the last time
The newest action statement of the version information of interception starts to intercept the preset quantity action statement.
In conclusion the disclosure provides a kind of method of data synchronization and device, obtained from the journal file of source database
Preset quantity action statement, every action statement include the version information of the action statement.It searches in a plurality of action statement
Cancel statement, and when finding cancel statement, determine version information than the cancel statement in preset quantity action statement
Old object run sentence, and the insertion sentence conduct for meeting the deletion condition of the cancel statement is searched in object run sentence
Sentence to be deleted.The sentence to be deleted is deleted from preset quantity action statement, to the parenthesis in remaining action statement
Sentence merges, and the data of MPP database are updated according to the action statement after merging.So, it is possible to reduce when more new data
The insertion operation of execution and the number of I/O operation reduce the time delay of more new data, to improve in source database and MPP database
Data consistency.
In the embodiment provided by the disclosure, it should be understood that disclosed device and method, it can also be by other
Mode realize.The apparatus embodiments described above are merely exemplary, for example, the flow chart and block diagram in attached drawing are shown
According to device, the architectural framework in the cards of method and computer program product, function of multiple embodiments of the disclosure
And operation.In this regard, each box in flowchart or block diagram can represent one of a module, section or code
Point, a part of the module, section or code includes one or more for implementing the specified logical function executable
Instruction.It should also be noted that function marked in the box can also be attached to be different from some implementations as replacement
The sequence marked in figure occurs.For example, two continuous boxes can actually be basically executed in parallel, they sometimes may be used
To execute in the opposite order, this depends on the function involved.It is also noted that each of block diagram and or flow chart
The combination of box in box and block diagram and or flow chart can be based on the defined function of execution or the dedicated of movement
The system of hardware is realized, or can be realized using a combination of dedicated hardware and computer instructions.
In addition, each functional module in each embodiment of the disclosure can integrate one independent portion of formation together
Point, it is also possible to modules individualism, an independent part can also be integrated to form with two or more modules.
It, can be with if the function is realized and when sold or used as an independent product in the form of software function module
It is stored in a computer readable storage medium.Based on this understanding, the technical solution of the disclosure is substantially in other words
The part of the part that contributes to existing technology or the technical solution can be embodied in the form of software products, the meter
Calculation machine software product is stored in a storage medium, including some instructions are used so that a computer equipment (can be a
People's computer, server or network equipment etc.) execute each embodiment the method for the disclosure all or part of the steps.
And storage medium above-mentioned includes: that USB flash disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), arbitrary access are deposited
The various media that can store program code such as reservoir (RAM, Random Access Memory), magnetic or disk.
It should be noted that, in this document, relational terms such as first and second and the like are used merely to a reality
Body or operation are distinguished with another entity or operation, are deposited without necessarily requiring or implying between these entities or operation
In any actual relationship or order or sequence.Moreover, the terms "include", "comprise" or its any other variant are intended to
Non-exclusive inclusion, so that the process, method, article or equipment including a series of elements is not only wanted including those
Element, but also including other elements that are not explicitly listed, or further include for this process, method, article or equipment
Intrinsic element.In the absence of more restrictions, the element limited by sentence "including a ...", it is not excluded that
There is also other identical elements in process, method, article or equipment including the element.
The above, the only specific embodiment of the disclosure, but the protection scope of the disclosure is not limited thereto, it is any
Those familiar with the art can easily think of the change or the replacement in the technical scope that the disclosure discloses, and should all contain
It covers within the protection scope of the disclosure.Therefore, the protection scope of the disclosure should be subject to the protection scope in claims.
Claims (10)
1. a kind of method of data synchronization, which is characterized in that the described method includes:
Preset quantity action statement is obtained from the journal file of source database, wherein every action statement includes the operation
The version information of sentence；
The cancel statement in the preset quantity action statement is inquired, and when finding cancel statement, in the present count
It measures and determines the version information object run sentence older than the cancel statement in an action statement, and in the object run sentence
The middle insertion sentence for searching the deletion condition for meeting the cancel statement is as sentence to be deleted；
The sentence to be deleted is deleted from the preset quantity action statement, to the insertion sentence in remaining action statement
It merges, and updates the data of MPP database according to the action statement after merging, wherein the action statement after the merging
Insertion sentence including at least the cancel statement and after merging.
2. the method according to claim 1, wherein inquiring the deletion language in the preset quantity action statement
Sentence, and when finding cancel statement, determine version information than the cancel statement in the preset quantity action statement
Old object run sentence, and the parenthesis for meeting the deletion condition of the cancel statement is searched in the object run sentence
Sentence is used as sentence to be deleted, comprising:
According to preset quantity action statement described in the order traversal of version information from the old to the new；
When the action statement currently traversed is cancel statement, by version information ratio in the preset quantity action statement
The old action statement of the cancel statement currently traversed is determined as the object run sentence, or grasps in the preset quantity item
Make that version information is between the cancel statement currently traversed and the last cancel statement traversed in sentence
Action statement is determined as the object run sentence；
The insertion sentence for meeting the deletion condition of the cancel statement currently traversed is searched in the object run sentence
As the sentence to be deleted.
3. according to the method described in claim 2, it is characterized in that, in the deletion inquired in the preset quantity action statement
Before sentence, the method also includes:
For every insertion sentence in the preset quantity action statement, the target data of insertion sentence operation is determined
Table, and the field that the target matrix includes is determined as aiming field；
Judge whether there is the insertion sentence aiming field not to be covered；
If so, increase the aiming field not to be covered in the insertion sentence, and by the inserted value of increased aiming field
It is set as empty.
4. according to the method described in claim 3, it is characterized in that, being closed to the insertion sentence in remaining action statement
And, comprising: the identical insertion sentence of the target matrix of operation is merged into an insertion in the remaining action statement
Sentence；Alternatively,
In the remaining action statement, version information is in the cancel statement currently traversed and the last time
Between the cancel statement traversed and operation the identical insertion sentence of target matrix merges into an insertion sentence.
5. method according to any of claims 1-4, which is characterized in that from the journal file of the source database
Obtain preset quantity action statement, comprising:
The journal file is parsed by log analysis tool, obtains the action statement of the source database；
In the action statement of the source database, institute is intercepted since the newest action statement of version information of last time interception
State preset quantity action statement.
6. a kind of data synchronization unit, which is characterized in that described device includes:
Action statement obtains module, for obtaining preset quantity action statement from the journal file of source database, wherein every
Action statement includes the version information of the action statement；
Searching module, the cancel statement for inquiring in the preset quantity action statement, and when finding cancel statement,
The version information object run sentence older than the cancel statement is determined in the preset quantity action statement, in the mesh
The insertion sentence for the deletion condition for meeting the cancel statement is searched in mark action statement as sentence to be deleted；
Removing module, for deleting the sentence to be deleted from the preset quantity action statement；
Merging module, for being merged to the insertion sentence in remaining action statement；And
Update module, for updating the data of MPP database according to the action statement after merging, wherein the behaviour after the merging
Make the insertion sentence after sentence merges including at least the cancel statement and the merging module.
7. device according to claim 6, which is characterized in that the searching module includes:
Submodule is traversed, for according to preset quantity action statement described in version information order traversal from the old to the new；
First searches submodule, for being grasped in the preset quantity item when the action statement currently traversed is cancel statement
Make that the version information action statement older than the cancel statement currently traversed is determined as the object run sentence in sentence, or
Version information is in the cancel statement currently traversed and primary time upper in the preset quantity action statement by person
The action statement between cancel statement gone through is determined as the object run sentence；
Second searches submodule, meets the cancel statement currently traversed for searching in the object run sentence
The insertion sentence of deletion condition is as the sentence to be deleted.
8. device according to claim 7, which is characterized in that described device further include:
Format analysis processing module, for determining the parenthesis for every insertion sentence in the preset quantity action statement
The target matrix of sentence operation, and the field that the target matrix includes is determined as aiming field；This is judged whether there is to insert
Enter sentence aiming field not to be covered, if so, increasing the aiming field not to be covered in the insertion sentence, and will increase
The inserted value of aiming field be set as empty.
9. device according to claim 8, which is characterized in that the merging module is specifically used for: in the remaining behaviour
Make that the identical insertion sentence of the target matrix of operation is merged into an insertion sentence in sentence；Alternatively, described remaining
In action statement, by the cancel statement that version information is in the cancel statement currently traversed and the last time traverses
Between and operation the identical insertion sentence of target matrix merge into an insertion sentence.
10. the device according to any one of claim 6-9, which is characterized in that it is specific that the action statement obtains module
For parsing the journal file by log analysis tool, the action statement of the source database is obtained, and in the source number
It is grasped according to the preset quantity item in the action statement in library, is intercepted since the newest action statement of version information of last time interception
Make sentence.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811074438.9A CN109408589B (en) | 2018-09-14 | 2018-09-14 | Data synchronization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201811074438.9A CN109408589B (en) | 2018-09-14 | 2018-09-14 | Data synchronization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN109408589A true CN109408589A (en) | 2019-03-01 |
| CN109408589B CN109408589B (en) | 2020-08-14 |
Family
ID=65464262
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201811074438.9A Active CN109408589B (en) | 2018-09-14 | 2018-09-14 | Data synchronization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN109408589B (en) |
Cited By (7)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059096A (en) * | 2019-03-16 | 2019-07-26 | 平安城市建设科技(深圳)有限公司 | Data version management method, apparatus, equipment and storage medium |
| CN111597257A (en) * | 2020-05-09 | 2020-08-28 | 远光软件股份有限公司 | Database synchronization method and device, storage medium and terminal |
| CN112632114A (en) * | 2019-10-08 | 2021-04-09 | 中国移动通信集团辽宁有限公司 | Method and device for MPP database to quickly read data and computing equipment |
| CN112988916A (en) * | 2021-03-05 | 2021-06-18 | 杭州天阙科技有限公司 | Full and incremental synchronization method, device and storage medium for Clickhouse |
| CN114647659A (en) * | 2020-12-17 | 2022-06-21 | 金篆信科有限责任公司 | Data processing method and device, electronic equipment and storage medium |
| CN116723204A (en) * | 2023-08-07 | 2023-09-08 | 南京国电南自轨道交通工程有限公司 | Method for synchronizing large-data-volume real-time data among layered systems |
| CN116795831A (en) * | 2022-12-26 | 2023-09-22 | 慧之安信息技术股份有限公司 | Method and system for maintaining number of time sequence data items |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102110121A (en) * | 2009-12-24 | 2011-06-29 | 阿里巴巴集团控股有限公司 | Method and system for processing data |
| CN102346775A (en) * | 2011-09-26 | 2012-02-08 | 苏州博远容天信息科技有限公司 | Method for synchronizing multiple heterogeneous source databases based on log |
| CN105955970A (en) * | 2015-11-12 | 2016-09-21 | 中国银联股份有限公司 | Log analysis-based database copying method and device |
| CN107038231A (en) * | 2017-04-11 | 2017-08-11 | 南京南瑞集团公司 | A kind of database high concurrent affairs merging method |
-
2018
- 2018-09-14 CN CN201811074438.9A patent/CN109408589B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102110121A (en) * | 2009-12-24 | 2011-06-29 | 阿里巴巴集团控股有限公司 | Method and system for processing data |
| CN102346775A (en) * | 2011-09-26 | 2012-02-08 | 苏州博远容天信息科技有限公司 | Method for synchronizing multiple heterogeneous source databases based on log |
| CN105955970A (en) * | 2015-11-12 | 2016-09-21 | 中国银联股份有限公司 | Log analysis-based database copying method and device |
| CN107038231A (en) * | 2017-04-11 | 2017-08-11 | 南京南瑞集团公司 | A kind of database high concurrent affairs merging method |
Cited By (9)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110059096A (en) * | 2019-03-16 | 2019-07-26 | 平安城市建设科技(深圳)有限公司 | Data version management method, apparatus, equipment and storage medium |
| CN112632114A (en) * | 2019-10-08 | 2021-04-09 | 中国移动通信集团辽宁有限公司 | Method and device for MPP database to quickly read data and computing equipment |
| CN112632114B (en) * | 2019-10-08 | 2024-03-19 | 中国移动通信集团辽宁有限公司 | Method, device and computing equipment for fast reading data by MPP database |
| CN111597257A (en) * | 2020-05-09 | 2020-08-28 | 远光软件股份有限公司 | Database synchronization method and device, storage medium and terminal |
| CN114647659A (en) * | 2020-12-17 | 2022-06-21 | 金篆信科有限责任公司 | Data processing method and device, electronic equipment and storage medium |
| CN112988916A (en) * | 2021-03-05 | 2021-06-18 | 杭州天阙科技有限公司 | Full and incremental synchronization method, device and storage medium for Clickhouse |
| CN116795831A (en) * | 2022-12-26 | 2023-09-22 | 慧之安信息技术股份有限公司 | Method and system for maintaining number of time sequence data items |
| CN116723204A (en) * | 2023-08-07 | 2023-09-08 | 南京国电南自轨道交通工程有限公司 | Method for synchronizing large-data-volume real-time data among layered systems |
| CN116723204B (en) * | 2023-08-07 | 2024-02-09 | 南京国电南自轨道交通工程有限公司 | Method for synchronizing large-data-volume real-time data among layered systems |
Also Published As
| Publication number | Publication date |
|---|---|
| CN109408589B (en) | 2020-08-14 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109408589A (en) | Method of data synchronization and device | |
| US10706093B2 (en) | System for organizing and fast searching of massive amounts of data | |
| US11392604B2 (en) | Designating fields in machine data using templates | |
| US20200334295A1 (en) | Merge tree garbage metrics | |
| US9953051B2 (en) | Multi-version concurrency control method in database and database system | |
| CN103902623B (en) | Method and system for the accessing file in storage system | |
| CN109416694A (en) | The key assignments storage system effectively indexed including resource | |
| CN103678556A (en) | Method for processing column-oriented database and processing equipment | |
| US7636731B2 (en) | Approximating a database statistic | |
| US8560500B2 (en) | Method and system for removing rows from directory tables | |
| US20080120275A1 (en) | Merging synopses to determine number of distinct values in large databases | |
| CN106294772A (en) | The buffer memory management method of distributed memory columnar database | |
| CN106682042B (en) | A kind of relation data caching and querying method and device | |
| CN109522271B (en) | Batch insertion and deletion method and device for B + tree nodes | |
| CN105589894B (en) | Document index establishing method and device and document retrieval method and device | |
| CN113420026B (en) | Database table structure changing method, device, equipment and storage medium | |
| US20210243221A1 (en) | Systems and methods for rapidly generating security ratings | |
| CN114911830A (en) | Index caching method, device, equipment and storage medium based on time sequence database | |
| CN108363791A (en) | A kind of method of data synchronization and device of database | |
| CN112699187A (en) | Associated data processing method, device, equipment, medium and product | |
| CN105025013A (en) | A dynamic IP coupling model based on a priority Trie tree | |
| CN112835923A (en) | Correlation retrieval method, device and equipment | |
| CN107291938A (en) | Order Query System and method | |
| CN110389939A (en) | A kind of Internet of Things storage system based on NoSQL and distributed file system | |
| CN116340318A (en) | Method, device, equipment and storage medium for processing secondary index record |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |