CN108121827A - A kind of synchronous method and device of full dose data - Google Patents
A kind of synchronous method and device of full dose data Download PDFInfo
- Publication number
- CN108121827A CN108121827A CN201810037013.4A CN201810037013A CN108121827A CN 108121827 A CN108121827 A CN 108121827A CN 201810037013 A CN201810037013 A CN 201810037013A CN 108121827 A CN108121827 A CN 108121827A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- file
- transaction information
- transaction
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
- G06F16/275—Synchronous replication
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Computing Systems (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The present invention provides a kind of synchronous method and device of full dose data.The described method includes：The data file of source database is read, and parses the data dictionary, database object and Transaction Information of the data file；Data receiver of the data dictionary with the data framework that the database object is formed to target database is sent by the form of transaction file；The Transaction Information is sent by the form of transaction file and is sent to data receiver；The data framework and Transaction Information that receive are loaded into target database by the data receiver in order, realize that source database is synchronous with target database.The synchronous efficiency of database can be promoted using the present invention, the load of database is reduced, reduces the resources occupation rate of database.
Description
Technical field
The present invention relates to database technical fields, and in particular to a kind of synchronous method and device of full dose data.
Background technology
Database is component most crucial in application business system, and the stabilization of database and the continuity of business are
The problem of most being paid close attention in application business system.
In many database application scenes (such as dual-active, long-distance disaster), two or more databases are usually needed
In data keep synchronous.
The full dose data in original database are usually transferred to target database by third party's data capsule now.Example
Such as, can be by the data copy in original database to disk, then by diskcopy into target database, calling can also be used
SQL modes carry out data transmission, but these method data synchronizing speeds are slow, and the load of database is heavier, can not quickly realize
The real-time synchronization of original database and targeted database data, and can also influence in data synchronization process on line database just
Often operation.
Therefore, how to propose a kind of method, the efficiency of source database and targeted database data real-time synchronization can be improved,
As urgent problem to be solved.
The content of the invention
For in the prior art the defects of, an embodiment of the present invention provides a kind of synchronous method and device of full dose data.
In a first aspect, the embodiment of the present invention provides a kind of synchronous method of full dose data, including：
The data file of source database is read, and parses the data dictionary, database object and thing of the data file
Business data；
The data dictionary and the data framework that the database object is formed to mesh are sent by the form of transaction file
Mark the data receiver of database；
The Transaction Information is sent by the form of transaction file and is sent to data receiver；
The data framework and Transaction Information that receive are loaded into target database by the data receiver in order
In, realize that the source database is synchronous with the target database.
Second aspect, the embodiment of the present invention provide a kind of sychronisation of full dose data, including：
File module is read, for reading the data file of source database, and parses the data word of the data file
Allusion quotation, database object and Transaction Information；
Data framework module is sent, the data dictionary and the database pair are sent for passing through the form of transaction file
As composition data framework to target database data receiver；
Transaction Information module is sent, data receiver is sent to for passing through the form of the transaction file transmission Transaction Information
End；
Load document module, for the data receiver by the data framework and Transaction Information that receive in order
It is loaded into target database, realizes that the source database is synchronous with the target database.
The third aspect, the embodiment of the present invention additionally provide a kind of computer equipment, including memory and processor, the place
Reason device and the memory complete mutual communication by bus；The memory storage has what can be performed by the processor
Program instruction, the processor call described program instruction to be able to carry out following method：
The data file of source database is read, and parses the data dictionary, database object and thing of the data file
Business data；
The data dictionary and the data framework that the database object is formed to mesh are sent by the form of transaction file
Mark the data receiver of database；
The Transaction Information is sent by the form of transaction file and is sent to data receiver；
The data framework and Transaction Information that receive are loaded into target database by the data receiver in order
In, realize that the source database is synchronous with the target database.
Fourth aspect, the embodiment of the present invention provide a kind of computer readable storage medium, are stored thereon with computer program,
For storing the method for foregoing computer program when the computer program is executed by processor.
Full dose method of data synchronization and device provided in an embodiment of the present invention, pass through the data that will be parsed in source database
Frame is first sent to target database, then the Transaction Information being loaded in data framework is sent to target database, realizes source
The real-time synchronization of database and target database need not directly invoke SQL statement to realize the synchronization of database, utilize this hair
It is bright to improve database synchronization efficiency, database loads are reduced, reduce the resources occupation rate of database.
Description of the drawings
It in order to illustrate more clearly about the embodiment of the present invention or technical scheme of the prior art, below will be to embodiment or existing
There is attached drawing needed in technology description to be briefly described, it should be apparent that, the accompanying drawings in the following description is this hair
Some bright embodiments, for those of ordinary skill in the art, without creative efforts, can be with root
Other attached drawings are obtained according to these attached drawings.
Fig. 1 is the flow diagram of full dose method of data synchronization provided in an embodiment of the present invention；
Fig. 2 is the flow diagram of full dose Data Data deriving method provided in an embodiment of the present invention；
Fig. 3 is the flow diagram of full dose data transactions data export method provided in an embodiment of the present invention；
Fig. 4 is the method flow schematic diagram for the full dose data synchronization that further embodiment of this invention provides；
Fig. 5 is the structure diagram of full dose data synchronization unit provided in an embodiment of the present invention；
Fig. 6 is the structure diagram of computer equipment provided in an embodiment of the present invention.
Specific embodiment
To make the purpose, technical scheme and advantage of the embodiment of the present invention clearer, below in conjunction with the embodiment of the present invention
In attached drawing, the technical solution in the embodiment of the present invention is clearly and completely described, it is clear that described embodiment is
Part of the embodiment of the present invention, instead of all the embodiments.Based on the embodiments of the present invention, those of ordinary skill in the art
All other embodiments obtained without creative efforts belong to the scope of protection of the invention.
Fig. 1 is the flow diagram of full dose method of data synchronization provided in an embodiment of the present invention, as shown in Figure 1, the side
Method includes：
S101, the data file for reading source database, and parse the data dictionary of the data file, database object
And Transaction Information；
S102, the data framework that the data dictionary is formed with the database object is sent by the form of transaction file
To the data receiver of target database；
S103, data receiver is sent to by the form transmission Transaction Information of transaction file；
The data framework and Transaction Information that receive are loaded into number of targets by S104, the data receiver in order
According in storehouse, realizing that the source database is synchronous with the target database.
Specifically, the data file in source database is synchronized in target database, the embodiment of the present invention is mainly pair
Full dose data synchronize, and the data structure of data file in source database is imported into target database first, then by source
Transaction Information in database is one by one or batch is imported into target database, completes the data of source database and target database
The synchronization of file.
First, the data file of source database is read, and parses the data dictionary of the data file, database object
And Transaction Information；
Then, the data dictionary and the data framework of database object composition are sent by the form of transaction file
To the data receiver of target database；It, will be first by tree to hand over for example, table in database is there are one tree-shaped structure
The destination end that the form of easy file is sent；
The Transaction Information is sent by the form of transaction file again and is sent to data receiver；For example, by tree-shaped knot
After structure is sent to destination end, then Transaction Information is sent to destination end；
The data framework and Transaction Information that receive are loaded into target database by data receiver in order, i.e.,
First data framework is loaded in target database, then Transaction Information is filled into data framework, realizes the source database
It is synchronous with the target database.
It should be noted that S102, S103 can be performed in order, it can also perform, not do in embodiments of the present invention simultaneously
It is specific to limit.
It during above-mentioned send and receive, is transmitted by wireless network, WAN, LAN etc. can be used.
Full dose method of data synchronization provided in an embodiment of the present invention, by the way that the data framework parsed in source database is first
Target database is sent to, then the Transaction Information being loaded in data framework is sent to target database, realizes source database
With the real-time synchronization of target database, SQL statement need not be directly invoked to realize the synchronization of database, promoted using the present invention
Database synchronization efficiency reduces database loads, reduces the resources occupation rate of database.
Optionally, the transaction file form is a kind of internal data format in data conversion process, is specially：xf1/
Xdt data modes.
Specifically, the data file in source database is parsed into data dictionary, database object and Transaction Information, by institute
It states data dictionary, database object and Transaction Information all to store and transmit in the form of transaction file, the embodiment of the present invention can use
The data format of xf1/xdt.It should be noted that xf1/xdt data formats provided in an embodiment of the present invention are not specifically limited,
Also other data modes can be selected.
Optionally, it is described to read the data file of source database, and parse the data dictionary of the data file, data
Storehouse object and Transaction Information are specially：
Mode is exported using directapath, reads the data file of the source database；
The data dictionary is obtained by inquiry system table, the data dictionary is the SQL statement for creating object；
The database object is obtained by inquiring about the system table, the database object includes at least：Table, index,
View, trigger；
By way of directapath export, the Transaction Information is exported.
Fig. 2 is the flow diagram of full dose data export method provided in an embodiment of the present invention, as shown in Figure 2.Using straight
Mode derived from path is connect, reads the data file of the source database；
Data dictionary is obtained by inquiry system table (schema.sys), for describing the xf1/xdt structures of the definition of table,
Build the SQL statement of table；
Database object is obtained by inquiry system table, the database object includes at least：
A. table information：Schema, table name, internal indicator, table space etc..
B. field information：Field name, type, precision etc..
C.LOB fields can be saved in different table spaces, it is therefore desirable to some special information：Internal indicator, table are empty
Between, storage format (basic/secure-file) etc..
D. the relevant database object of table, such as index.
E. partition information：Divisional type, zone boundary etc..
F. data segment information.
By way of directapath export, the Transaction Information is exported.Transaction Information export is the synchronization of full dose data
Major function and a most time-consuming part.
Full dose method of data synchronization provided in an embodiment of the present invention, by the way that the data framework parsed in source database is first
Target database is sent to, then the Transaction Information being loaded in data framework is sent to target database, realizes source database
With the real-time synchronization of target database, SQL statement need not be directly invoked to realize the synchronization of database, promoted using the present invention
Database synchronization efficiency reduces database loads, reduces the resources occupation rate of database, to greatest extent reduces manually dry
In advance, realize and do not shut down, the business that do not stop doing business, do not change the original parameter of database, the full dose data synchronizing function of a key is met
The applications such as migration, inquiry separation, this long-distance disaster, dual-active switching, dual-active duplication.
Optionally, it is described by way of directapath export, the Transaction Information is exported, is specially：
The initial data block of the data segment of the Transaction Information is parsed, obtains data block cluster information；
According to the data block cluster information, block-by-block parses data Transaction Information in the block and exports.
Specifically, Fig. 3 is the flow diagram of full dose data transactions data export method provided in an embodiment of the present invention, such as
Shown in Fig. 3, data space management is divided into table space (tablespace), data file (datafile), number by database
According to the multistage segmentation paging system management such as section (segment), data block cluster (extent), block (block).
Table space is made of one or more data files.All data are all stored in some table space.Each table
Space can preserve multiple database objects (table, index, LOB etc.).One table can be by one or more data segments (segment)
Composition.
Since a table is made of one or more data segments.So the directapath of the Transaction Information of a table is derived
Journey is exactly the process for sequentially exporting each data segment.
It is specifically that the Transaction Information export process of a data segment is as follows：
The initial data block of the data segment of the Transaction Information is parsed, obtains data block cluster information；
According to the data block cluster information, block-by-block parses data Transaction Information in the block and exports, until by table
In all Transaction Informations all export.
The initial position of each data segment is had been saved in data dictionary, so, the directapath export of data segment from
This data segment initial position starts.
During derived from data segment, as soon as data segment may have thousands of a data blocks, this resolving
It is likely to require several hours or even time a couple of days.This period, user's application all worked in Transaction Information always, it is possible to
Data loss problem, such as transaction rollback is caused to be possible to because database is transaction journal mark to have deleted,
But without submission, but move on in roll-back segment.Once such case affairs rollback is possible to lose data.
Method provided by the invention further includes supplement export (supplemental export), is mainly parsing number
After section, according to the suspicious record information preserved in resolving, judge that record is to replicate also with reference to the data of roll-back segment
It is to abandon, so as to ensure that data will not lose.
Optionally, the data framework and Transaction Information that receive are loaded into target by the data receiver in order
It in database, realizes that the source database is synchronous with the target database, is specially：
Receive the data framework and the Transaction Information that the source database is sent in the form of transaction file；
Received transaction file is stored in disk, the transaction file in the disk is resolved to SQL statement, institute
Data framework and Transaction Information is stated to be loaded into order in the target database.
Specifically, source database exports data dictionary, database object and Transaction Information, and with the shape of transaction file
Formula is sent to target database receiving terminal by network, and target database receiving terminal receives the transaction file of source database transmission
Data；And be saved in disk file, the data load module of target database reads the transaction file in queue from disk, makes
It is loaded into database api in target database.
Specifically, disk file is loaded, disk file is resolved into SQL statement, the Transaction Information that will be parsed
One by one or batch is inserted into target database, completes data load process.
Full dose method of data synchronization provided in an embodiment of the present invention, by the way that the data framework parsed in source database is first
Target database is sent to, then the Transaction Information being loaded in data framework is sent to target database, realizes source database
With the real-time synchronization of target database, SQL statement need not be directly invoked to realize the synchronization of database, promoted using the present invention
Database synchronization efficiency reduces database loads, reduces the resources occupation rate of database, to greatest extent reduces manually dry
In advance, realize and do not shut down, the business that do not stop doing business, do not change the original parameter of database, the full dose data synchronizing function of a key is met
The applications such as migration, inquiry separation, this long-distance disaster, dual-active switching, dual-active duplication.
Fig. 4 is the method flow schematic diagram for the full dose data synchronization that further embodiment of this invention provides, as shown in figure 4, right
The step of full dose data synchronize transmission in source database and target database is as follows：
Mode is exported using directapath, the data file of the source database is read, by data dictionary, database object
Definition and Transaction Information export, by data dictionary, the data framework of database object constructing definitions and thing by way of network
Business data send to obtain the receiving terminal of target database in the form of transaction file, and the receiving terminal of target database receives source data
The transaction file data that storehouse is sent, parse it, and the data after parsing are loaded using data upload program, complete
It is synchronous into full dose data.
Full dose method of data synchronization provided in an embodiment of the present invention, by the way that the data framework parsed is first sent to target
End, then Transaction Information is sent to destination end, so as to improve database synchronization efficiency, the resources occupation rate of database is reduced,
Reduce manual intervention to greatest extent, realize and do not shut down, the business that do not stop doing business, do not change the original parameter of database, a key
Full dose data synchronizing function meets the applications such as migration, inquiry separation, this long-distance disaster, dual-active switching, dual-active duplication.
Fig. 5 is the structure diagram of full dose data synchronization unit provided in an embodiment of the present invention, as shown in figure 5, the dress
Put including：File module 10 is read, data framework module 20 is sent, sends Transaction Information module 30 and load document module 40,
Wherein：
It reads file module 10 and is used to read the data file of source database, and parse the data word of the data file
Allusion quotation, database object and Transaction Information；
Data framework module 20 is sent to be used to send the data dictionary and the database by the form of transaction file
Object form data framework to target database data receiver；
Transaction Information module 30 is sent to connect for being sent to data by the form of the transaction file transmission Transaction Information
Receiving end；
Load document module 40 is for the data receiver by the data framework and Transaction Information that receive by suitable
Sequence is loaded into target database, realizes that the source database is synchronous with the target database.
Specifically, read file module 10 and read the data file of source database, and parse the number of the data file
According to dictionary, database object and Transaction Information；It sends data framework module 20 and the data is sent by the form of transaction file
Dictionary and the data framework that the database object is formed to the data receiver of target database；Send Transaction Information module 30
The Transaction Information is sent by the form of transaction file and is sent to data receiver；Data receiver described in load document module 40
End the data framework and Transaction Information that receive are loaded into target database in order, realize the source database and
The synchronization of the target database.
Full dose data synchronization unit provided in an embodiment of the present invention, by the way that the data framework parsed in source database is first
Target database is sent to, then the Transaction Information being loaded in data framework is sent to target database, realizes source database
With the real-time synchronization of target database, SQL statement need not be directly invoked to realize the synchronization of database, promoted using the present invention
Database synchronization efficiency reduces database loads, reduces the resources occupation rate of database.Optionally, the reading file mould
Block, specially：
Mode is exported using directapath, reads the data file of the source database；
According to the data file of reading, the data dictionary is obtained by inquiry system table, the data dictionary is establishment
The SQL statement of object；
According to the data file of reading, the database object, the database pair are obtained by inquiring about the system table
As including at least：Table, index, view, trigger；
By way of directapath export, exported from read data file by the Transaction Information.
Specifically, using mode derived from directapath, the data file of the source database is read；
For read data file, data dictionary is obtained by inquiry system table (schema.sys), for describing
The xf1/xdt structures of the definition of table build the SQL statement of table；
Database object is obtained by inquiry system table, the database object includes at least：
A. table information：Schema, table name, internal indicator, table space etc..
B. field information：Field name, type, precision etc..
C.LOB fields can be saved in different table spaces, it is therefore desirable to some special information：Internal indicator, table are empty
Between, storage format (basic/secure-file) etc..
D. the relevant database object of table, such as index.
E. partition information：Divisional type, zone boundary etc..
F. data segment information.
By way of directapath export, the Transaction Information is exported.Transaction Information export is the synchronization of full dose data
Major function and a most time-consuming part.
Full dose data synchronization unit provided in an embodiment of the present invention, by the way that the data framework parsed in source database is first
Target database is sent to, then the Transaction Information being loaded in data framework is sent to target database, realizes source database
With the real-time synchronization of target database, SQL statement need not be directly invoked to realize the synchronization of database, promoted using the present invention
Database synchronization efficiency reduces database loads, reduces the resources occupation rate of database, to greatest extent reduces manually dry
In advance, realize and do not shut down, the business that do not stop doing business, do not change the original parameter of database, the full dose data synchronizing function of a key is met
The applications such as migration, inquiry separation, this long-distance disaster, dual-active switching, dual-active duplication.
Optionally, the load document module, specially：
Receive the data framework and the Transaction Information that the source database is sent in the form of transaction file；
Received transaction file is stored in disk, the transaction file in the disk is resolved to SQL statement, institute
Data framework and Transaction Information is stated to be loaded into order in the target database.
Specifically, source database exports data dictionary, database object and Transaction Information, and with the shape of transaction file
Formula is sent to target database receiving terminal by communication network, and target database receiving terminal receives the transaction of source database transmission
File data；And be saved in disk file, the data load module of target database reads the transaction text in queue from disk
Part is loaded into using database api in target database.
Specifically, disk file is loaded, is specially：Transaction file is resolved into SQL statement, the thing that will be parsed
Business data are one by one or batch is inserted into target database, complete data load process.
Full dose data synchronization unit provided in an embodiment of the present invention, by the way that the data framework parsed is first sent to target
End, then Transaction Information is sent to destination end, so as to improve database synchronization efficiency, the resources occupation rate of database is reduced,
Reduce manual intervention to greatest extent, realize and do not shut down, the business that do not stop doing business, do not change the original parameter of database, a key
Full dose data synchronizing function meets the applications such as migration, inquiry separation, this long-distance disaster, dual-active switching, dual-active duplication.
The structure diagram of Fig. 6 computer equipments provided in an embodiment of the present invention, as shown in fig. 6, the computer equipment bag
It includes：Processor (processor) 601, memory (memory) 602 and bus 603；
Wherein, the processor 601 and the memory 602 complete mutual communication by the bus 603；
The processor 601 is used to call the program instruction in the memory 602, to perform above-mentioned each method embodiment
The method provided, such as including：
The data file of source database is read, and parses the data dictionary, database object and thing of the data file
Business data；
The data dictionary and the data framework that the database object is formed to mesh are sent by the form of transaction file
Mark the data receiver of database；
The Transaction Information is sent by the form of transaction file and is sent to data receiver；
The data framework and Transaction Information that receive are loaded into target database by the data receiver in order
In, realize that the source database is synchronous with the target database.
The embodiment of the present invention discloses a kind of computer program product, and the computer program product includes being stored in non-transient
Computer program on computer readable storage medium, the computer program include program instruction, when described program instructs quilt
When computer performs, computer is able to carry out the method that above-mentioned each method embodiment is provided, such as including：
The data file of source database is read, and parses the data dictionary, database object and thing of the data file
Business data；
The data dictionary and the data framework that the database object is formed to mesh are sent by the form of transaction file
Mark the data receiver of database；
The Transaction Information is sent by the form of transaction file and is sent to data receiver；
The data framework and Transaction Information that receive are loaded into target database by the data receiver in order
In, realize that the source database is synchronous with the target database.
The embodiment of the present invention provides a kind of non-transient computer readable storage medium storing program for executing, the non-transient computer readable storage
Medium storing computer instructs, and the computer instruction makes the computer perform the side that above-mentioned each method embodiment is provided
Method, such as including：
The data file of source database is read, and parses the data dictionary, database object and thing of the data file
Business data；
The data dictionary and the data framework that the database object is formed to mesh are sent by the form of transaction file
Mark the data receiver of database；
The Transaction Information is sent by the form of transaction file and is sent to data receiver；
The data framework and Transaction Information that receive are loaded into target database by the data receiver in order
In, realize that the source database is synchronous with the target database.
Through the above description of the embodiments, those skilled in the art can be understood that each embodiment can
It is realized by the mode of software plus required general hardware platform, naturally it is also possible to pass through hardware.Based on such understanding, on
Technical solution is stated substantially in other words to embody the part that the prior art contributes in the form of software product, it should
Computer software product can store in a computer-readable storage medium, such as ROM/RAM, magnetic disc, CD, including several fingers
Order, which is used, so that computer equipment (can be personal computer, server or the network equipment etc.) performs each implementation
Method described in some parts of example or embodiment.
Device and system embodiment described above is only schematical, wherein described be used as separating component explanation
Unit may or may not be physically separate, the component shown as unit may or may not be
Physical location, you can be located at a place or can also be distributed in multiple network element.It can be according to the actual needs
Some or all of module therein is selected to realize the purpose of this embodiment scheme.Those of ordinary skill in the art are not paying
In the case of performing creative labour, you can to understand and implement.
Claims (10)
1. a kind of synchronous method of full dose data, which is characterized in that including：
The data file of source database is read, and parses the data dictionary, database object and number of transactions of the data file
According to；
The data dictionary and the data framework that the database object is formed to number of targets are sent by the form of transaction file
According to the data receiver in storehouse；
The Transaction Information is sent by the form of transaction file and is sent to the data receiver；
The data framework and Transaction Information that receive are loaded into target database by the data receiver in order, real
The existing source database is synchronous with the target database.
2. according to the method described in claim 1, it is characterized in that, the transaction file form is one in data conversion process
Kind internal data format, specially：Xf1/xdt data modes.
3. according to the method described in claim 1, it is characterized in that, the data file for reading source database, and parses
Data dictionary, database object and the Transaction Information of the data file, specially：
Mode is exported using directapath, reads the data file of the source database；
According to the data file of reading, the data dictionary is obtained by inquiry system table, the data dictionary is establishment object
SQL statement；
According to the data file of reading, the database object is obtained by inquiring about the system table, the database object is extremely
Include less：Table, index, view, trigger；
By way of directapath export, the Transaction Information is exported from the data file of reading.
4. according to the method described in claim 3, it is characterized in that, described by way of directapath export, by the affairs
Data export, and are specially：
The initial data block of the data segment of the Transaction Information is parsed, obtains data block cluster information；
According to the data block cluster information, block-by-block parses data Transaction Information in the block and exports.
5. the according to the method described in claim 1, it is characterized in that, data framework that the data receiver will receive
It is loaded into order in target database with Transaction Information, realizes that the source database is synchronous with the target database, had
Body is：
Receive the data framework and the Transaction Information that the source database is sent in the form of transaction file；
Received transaction file is stored in disk, and the transaction file in the disk is resolved to SQL statement, the number
It is loaded into order in the target database according to frame and Transaction Information.
6. a kind of sychronisation of full dose data, which is characterized in that including：
File module is read, for reading the data file of source database, and parses the data dictionary of the data file, number
According to storehouse object and Transaction Information；
Data framework module is sent, the data dictionary and the database object structure are sent for passing through the form of transaction file
Into data framework to target database data receiver；
Transaction Information module is sent, data receiver is sent to for passing through the form of the transaction file transmission Transaction Information；
Load document module loads the data framework and Transaction Information that receive for the data receiver in order
Into target database, realize that the source database is synchronous with the target database.
7. device according to claim 6, which is characterized in that the reading file module, specially：
Mode is exported using directapath, reads the data file of the source database；
According to the data file of reading, the data dictionary is obtained by inquiry system table, the data dictionary is establishment object
SQL statement；
According to the data file of reading, the database object is obtained by inquiring about the system table, the database object is extremely
Include less：Table, index, view, trigger；
By way of directapath export, the Transaction Information is exported from the data file of reading.
8. device according to claim 6, which is characterized in that the load document module, specially：
Receive the data framework and the Transaction Information that the source database is sent in the form of transaction file；
Received transaction file is stored in disk, and the transaction file in the disk is resolved to SQL statement, the number
It is loaded into order in the target database according to frame and Transaction Information.
9. a kind of computer equipment, which is characterized in that including memory and processor, the processor and the memory pass through
Bus completes mutual communication；The memory storage has the program instruction that can be performed by the processor, the processor
Described program instruction is called to be able to carry out the method as described in claim 1 to 5 is any.
10. a kind of computer readable storage medium, is stored thereon with computer program, which is characterized in that the computer program quilt
The method as described in claim 1 to 5 is any is realized when processor performs.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810037013.4A CN108121827B (en) | 2018-01-15 | 2018-01-15 | Full data synchronization method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201810037013.4A CN108121827B (en) | 2018-01-15 | 2018-01-15 | Full data synchronization method and device |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN108121827A true CN108121827A (en) | 2018-06-05 |
| CN108121827B CN108121827B (en) | 2020-10-16 |
Family
ID=62232903
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201810037013.4A Active CN108121827B (en) | 2018-01-15 | 2018-01-15 | Full data synchronization method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN108121827B (en) |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108776710A (en) * | 2018-06-28 | 2018-11-09 | 农信银资金清算中心有限责任公司 | A kind of concurrent stowage and device of database data |
| CN109165263A (en) * | 2018-10-16 | 2019-01-08 | 农信银资金清算中心有限责任公司 | Log analysis processing method and processing device, DDL synchronous method and the device of database |
| CN109635040A (en) * | 2018-11-27 | 2019-04-16 | 中国建设银行股份有限公司 | Real time data migration method, apparatus and storage medium |
| CN109783571A (en) * | 2018-12-13 | 2019-05-21 | 平安科技（深圳）有限公司 | Data processing method, device, computer equipment and the storage medium of isolation environment |
| CN110263028A (en) * | 2019-04-26 | 2019-09-20 | 石化盈科信息技术有限责任公司 | A kind of full dose synchronous method applied to search service |
| CN110489483A (en) * | 2019-07-08 | 2019-11-22 | 中国平安人寿保险股份有限公司 | Method of data synchronization, device, computer equipment and storage medium |
| CN111352766A (en) * | 2018-12-21 | 2020-06-30 | 中国移动通信集团山东有限公司 | Database double-activity implementation method and device |
| CN112015804A (en) * | 2019-05-28 | 2020-12-01 | 阿里巴巴集团控股有限公司 | Data synchronization method, device, equipment and storage medium |
Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101110706A (en) * | 2007-07-23 | 2008-01-23 | 中兴通讯股份有限公司 | Configuration data synchronization method |
| EP2058738A1 (en) * | 2006-08-31 | 2009-05-13 | NTT DoCoMo, Inc. | Database synchronization system and database synchronization method |
| CN101639865A (en) * | 2009-08-28 | 2010-02-03 | 金蝶软件（中国）有限公司 | Method and system for synchronizing and processing data and single-point login system |
| CN102629268A (en) * | 2012-03-09 | 2012-08-08 | 华为技术有限公司 | Data synchronization method, system and date access device |
| CN102999600A (en) * | 2012-11-20 | 2013-03-27 | 中兴通讯股份有限公司 | Method and system for automatically generating embedded database |
| CN103369660A (en) * | 2012-03-26 | 2013-10-23 | 大唐联诚信息系统技术有限公司 | Network-element data synchronization method and network-element device |
| CN103491137A (en) * | 2013-09-02 | 2014-01-01 | 用友软件股份有限公司 | Data synchronizing system and data synchronizing method |
| CN105426373A (en) * | 2014-09-19 | 2016-03-23 | 阿里巴巴集团控股有限公司 | Database synchronization method and device |
| CN105933446A (en) * | 2016-06-28 | 2016-09-07 | 中国农业银行股份有限公司 | Service dual-active implementation method and system of big data platform |
| US20170169053A1 (en) * | 2007-09-12 | 2017-06-15 | Netapp Inc. | Mechanism for converting one type of mirror to another type of mirror on a storage system without transferring data |
| US9811542B1 (en) * | 2013-06-30 | 2017-11-07 | Veritas Technologies Llc | Method for performing targeted backup |
| CN107391635A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Data synchronous system and method |
-
2018
- 2018-01-15 CN CN201810037013.4A patent/CN108121827B/en active Active
Patent Citations (12)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| EP2058738A1 (en) * | 2006-08-31 | 2009-05-13 | NTT DoCoMo, Inc. | Database synchronization system and database synchronization method |
| CN101110706A (en) * | 2007-07-23 | 2008-01-23 | 中兴通讯股份有限公司 | Configuration data synchronization method |
| US20170169053A1 (en) * | 2007-09-12 | 2017-06-15 | Netapp Inc. | Mechanism for converting one type of mirror to another type of mirror on a storage system without transferring data |
| CN101639865A (en) * | 2009-08-28 | 2010-02-03 | 金蝶软件（中国）有限公司 | Method and system for synchronizing and processing data and single-point login system |
| CN102629268A (en) * | 2012-03-09 | 2012-08-08 | 华为技术有限公司 | Data synchronization method, system and date access device |
| CN103369660A (en) * | 2012-03-26 | 2013-10-23 | 大唐联诚信息系统技术有限公司 | Network-element data synchronization method and network-element device |
| CN102999600A (en) * | 2012-11-20 | 2013-03-27 | 中兴通讯股份有限公司 | Method and system for automatically generating embedded database |
| US9811542B1 (en) * | 2013-06-30 | 2017-11-07 | Veritas Technologies Llc | Method for performing targeted backup |
| CN103491137A (en) * | 2013-09-02 | 2014-01-01 | 用友软件股份有限公司 | Data synchronizing system and data synchronizing method |
| CN105426373A (en) * | 2014-09-19 | 2016-03-23 | 阿里巴巴集团控股有限公司 | Database synchronization method and device |
| CN105933446A (en) * | 2016-06-28 | 2016-09-07 | 中国农业银行股份有限公司 | Service dual-active implementation method and system of big data platform |
| CN107391635A (en) * | 2017-06-30 | 2017-11-24 | 北京奇虎科技有限公司 | Data synchronous system and method |
Non-Patent Citations (1)
| Title |
|---|
| DANIEL J.WORDEN: "《Sybase开发者指南》", 30 November 1995, 清华大学出版社 * |
Cited By (13)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN108776710B (en) * | 2018-06-28 | 2020-06-30 | 农信银资金清算中心有限责任公司 | Concurrent loading method and device for database data |
| CN108776710A (en) * | 2018-06-28 | 2018-11-09 | 农信银资金清算中心有限责任公司 | A kind of concurrent stowage and device of database data |
| CN109165263A (en) * | 2018-10-16 | 2019-01-08 | 农信银资金清算中心有限责任公司 | Log analysis processing method and processing device, DDL synchronous method and the device of database |
| CN109635040A (en) * | 2018-11-27 | 2019-04-16 | 中国建设银行股份有限公司 | Real time data migration method, apparatus and storage medium |
| CN109635040B (en) * | 2018-11-27 | 2022-03-11 | 中国建设银行股份有限公司 | Real-time data migration method and device and storage medium |
| CN109783571A (en) * | 2018-12-13 | 2019-05-21 | 平安科技（深圳）有限公司 | Data processing method, device, computer equipment and the storage medium of isolation environment |
| CN109783571B (en) * | 2018-12-13 | 2023-10-27 | 平安科技（深圳）有限公司 | Data processing method, device, computer equipment and storage medium for isolated environment |
| CN111352766A (en) * | 2018-12-21 | 2020-06-30 | 中国移动通信集团山东有限公司 | Database double-activity implementation method and device |
| CN110263028B (en) * | 2019-04-26 | 2021-06-15 | 石化盈科信息技术有限责任公司 | Full-scale synchronization method applied to search service |
| CN110263028A (en) * | 2019-04-26 | 2019-09-20 | 石化盈科信息技术有限责任公司 | A kind of full dose synchronous method applied to search service |
| CN112015804A (en) * | 2019-05-28 | 2020-12-01 | 阿里巴巴集团控股有限公司 | Data synchronization method, device, equipment and storage medium |
| CN110489483A (en) * | 2019-07-08 | 2019-11-22 | 中国平安人寿保险股份有限公司 | Method of data synchronization, device, computer equipment and storage medium |
| CN110489483B (en) * | 2019-07-08 | 2024-02-06 | 中国平安人寿保险股份有限公司 | Data synchronization method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN108121827B (en) | 2020-10-16 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN108121827A (en) | A kind of synchronous method and device of full dose data | |
| CN110032604B (en) | Data storage device, translation device and database access method | |
| CN110908997B (en) | Data blood relationship construction method and device, server and readable storage medium | |
| EP2344959B1 (en) | Index compression in databases | |
| CN111445236B (en) | Block chain transaction verification method, device and storage medium | |
| CN105518641B (en) | Point-to-Point Data clone method, equipment and system and host node switching method, equipment and system | |
| CN111858760B (en) | Data processing method and device for heterogeneous database | |
| CN103631924B (en) | A kind of application process and system of distributive database platform | |
| WO2012068110A1 (en) | Safely transporting legacy data to an object semantic form data grid | |
| CN101673289B (en) | Method and device for constructing distributed file storage framework | |
| US20130191328A1 (en) | Standardized framework for reporting archived legacy system data | |
| CN112182099A (en) | Transaction verification method and device | |
| US10664459B2 (en) | Database managing method, database managing system, and database tree structure | |
| CN107562882A (en) | A kind of method of data synchronization and device based on log analysis | |
| CN112579695A (en) | Data synchronization method and device | |
| CN109947759A (en) | A kind of data directory method for building up, indexed search method and device | |
| CN111753141B (en) | Data management method and related equipment | |
| CN108418746A (en) | A kind of mail synchronization method, device and computer readable storage medium | |
| CN111221861A (en) | Method and device for synchronizing local cache and database data | |
| CN116775712A (en) | Method, device, electronic equipment, distributed system and storage medium for inquiring linked list | |
| CN114127707A (en) | System, computing node and method for processing write requests | |
| CN111881110A (en) | Data migration method and device | |
| CN115687290A (en) | Data transfer method, device and equipment | |
| CN112291312B (en) | ETL data synchronization method and device, electronic equipment and storage medium | |
| CN104869056A (en) | Institution personnel data synchronization method based on relational data separation |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |