CN104504012A - Database data migration method based on inverse topology - Google Patents
Database data migration method based on inverse topology Download PDFInfo
- Publication number
- CN104504012A CN104504012A CN201410758140.5A CN201410758140A CN104504012A CN 104504012 A CN104504012 A CN 104504012A CN 201410758140 A CN201410758140 A CN 201410758140A CN 104504012 A CN104504012 A CN 104504012A
- Authority
- CN
- China
- Prior art keywords
- node
- database
- data
- data migration
- method based
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a database data migration method based on an inverse topology. The database data migration method based on the inverse topology comprises the following steps of acquiring a topological relationship graph according to a database requiring to be migrated; obtaining an extended sequential sequence during a migration process according to the topological relationship graph and a mode transformation model; immigrating data and modes into NoSQL according to the obtained extended sequential sequence. According to the database data migration method based on the inverse topology, mode migration is carried out while data migration is carried out, so that dependent data in a data table and inverted data are correctly integrated into the migrated NoSQL database, and any one join inquiry in a SQL (Structured Query Language) database before migration can find a correct result in an individual table in the NoSQL database after migration, and access to multiple tables is not needed, and the inquiry efficiency of the migrated NoSQL database is effectively improved. The database data migration method based on the inverse topology can be widely applied in the field of computers.
Description
Technical field
The present invention relates to field of computer technology, particularly relate to a kind of database data migration method based on inverse topology.
Background technology
In current existing correlative study work, mainly contain following three aspects: denormalization, data batchmove and patten transformation.Denormalization work mainly pointer, to the normal constraint of traditional Relational DataBase, is carried out suitable redundant storage and is carried out design database to improve the modes such as search efficiency.Data batchmove work mainly refers to it is the resettlement being only data in database from SQL database to NoSQL database migration, such as, in SQL database, the data of certain table are dumped to merely certain Collection in MongoDB data or the table of certain in other NoSQL databases, do not carry out patten transformation, the dependence of the table in SQL database is not moved.Patten transformation work mainly refers to is not only data dump in SQL database in NoSQL database, and corresponding pattern is also moved.
In the data migration method of traditional Relational DataBase to NoSQL database, a lot of research work is at present the content (data batchmove) of merely migration table, do not carry out patten transformation, namely in transition process, lost the incidence relation between the SQL database table before migration.NoSQL database so like this after migration can not be supported in the result of the join query statement drawing original SQL database in single table, needs to carry out separately inquiry and repeatedly, causes search efficiency low.
Summary of the invention
In order to solve the problems of the technologies described above, the object of this invention is to provide one can carry out patten transformation, improves a kind of database data migration method based on inverse topology of the rear search efficiency of migration.
The technical solution adopted in the present invention is:
Based on a database data migration method for inverse topology, comprise the following steps:
A, the database moved as required, obtain its topological relation figure;
B, according to topological relation figure and patten transformation model, draw the extended order sequence in transition process;
Data and pattern are migrated in NoSQL by the extended order sequence that C, basis obtain.
As the further improvement of described a kind of database data migration method based on inverse topology, described step B comprises:
B1, from the leaf node topological relation figure concentrate appoint get a node as start node;
B2, to process with the limit of this start node for arc tail node all, obtain the integration order of its corresponding sides;
B3, this start node is shifted out leaf node collection, put into isolated node collection;
B4, integration order processing procedure obtained are added to extended order sequence;
B5, return perform step B1, until leaf node integrates as empty set.
As the further improvement of described a kind of database data migration method based on inverse topology, described step B2 comprises:
B21, appoint arbitrary the limit that to get with this start node be arc tail node, and expansion integration is carried out to the arc head node on this limit, obtain the integration order on this limit;
B22, judge whether this arc head node is leaf node when not comprising this limit, if so, then this starting point is put into leaf node collection; Otherwise, perform step B23;
B23, return perform step B21, until all limits that this start node is arc tail node are all disposed.
As the further improvement of described a kind of database data migration method based on inverse topology, described step C comprises:
C1, the data only appearing at the node of arc tail are moved to respectively in tables of data corresponding in NoSQL database;
C2, according to extended order sequence, according to the pattern after conversion successively by tables of data corresponding to NoSQL database for the Data Migration of residue node.
As the further improvement of described a kind of database data migration method based on inverse topology, described step C2 comprises:
Integration order in C21, successively taking-up extended order sequence;
C22, the information of arc tail node in the integration order of taking-up incorporated in arc head node and arc head node is expanded;
C23, by tables of data corresponding to NoSQL database for the Data Migration of arc head node after expansion.
The invention has the beneficial effects as follows:
A kind of database data migration method based on inverse topology of the present invention is by carrying out data batchmove simultaneously, carry out mode shifts, make the data of the tables of data having dependence and drive in the wrong direction that is correctly incorporated in the NoSQL database after migration, thus ensure that any one join in the SQL database before migration inquires about and can find correct result in the single table in the NoSQL database after migration, no longer need to conduct interviews to multiple table, effectively improve the search efficiency to NoSQL database after migration.
Accompanying drawing explanation
Below in conjunction with accompanying drawing, the specific embodiment of the present invention is described further:
Fig. 1 is the flow chart of steps of a kind of database data migration method based on inverse topology of the present invention;
Fig. 2 is the patten transformation schematic diagram of a kind of database data migration method embodiment one based on inverse topology of the present invention;
Fig. 3 is the topological relation schematic diagram of a kind of database data migration method embodiment two based on inverse topology of the present invention.
Embodiment
With reference to figure 1, a kind of database data migration method based on inverse topology of the present invention, comprises the following steps:
A, the database moved as required, obtain its topological relation figure;
B, according to topological relation figure and patten transformation model, draw the extended order sequence in transition process;
Data and pattern are migrated in NoSQL by the extended order sequence that C, basis obtain.
As the further improvement of described a kind of database data migration method based on inverse topology, described step B comprises:
B1, from the leaf node topological relation figure concentrate appoint get a node as start node;
B2, to process with the limit of this start node for arc tail node all, obtain the integration order of its corresponding sides;
B3, this start node is shifted out leaf node collection, put into isolated node collection;
B4, integration order processing procedure obtained are added to extended order sequence;
B5, return perform step B1, until leaf node integrates as empty set.
As the further improvement of described a kind of database data migration method based on inverse topology, described step B2 comprises:
B21, appoint arbitrary the limit that to get with this start node be arc tail node, and expansion integration is carried out to the arc head node on this limit, obtain the integration order on this limit;
B22, judge whether this arc head node is leaf node when not comprising this limit, if so, then this starting point is put into leaf node collection; Otherwise, perform step B23;
B23, return perform step B21, until all limits that this start node is arc tail node are all disposed.
Wherein, being specifically summarized as follows of step B:
(1) in the leaf node collection P topological relation figure, taking-up node u is appointed;
(2) take out arbitrary the limit <v that be arc tail with this node u, u>, expands the arc head node v on this limit, namely the full detail of node u is injected in node v, circular treatment complete all take u as the limit of arc tail;
(3) when node v expands complete, namely not with this limit for arc head, just node is put into leaf node collection P.When leaf node u is no longer relied on by any node, namely not with this node for arc tail, just this node is shifted out set of node P, puts into isolated node collection T;
(4) circular treatment leaf node collection P like this, until leaf node integrates P as empty set.
The specific embodiment one of step B of the present invention is as follows:
With reference to figure 2, the relevant database that such as will carry out moving has table T3, T2, T1, and dependence has that T3 relies on the external key of T2, T2 relies on the external key of T1.The table of database and dependence have T3, T2, T1 tri-nodes and < T3, T2>, < T2, T1> two table after being expressed as corresponding directed acyclic graph.Here is all concrete steps of present case:
S1, from leaf node collection P={ T1 } appoint taking-up node T1;
S2, arbitrary the limit <T2 that to take out with this node T1 be arc tail, T1>, carry out expansion to the arc head node T2 on this limit and integrate, namely the full detail of node T1 is injected in node T2.Node T2 expands complete, namely not with this limit for arc head, just node is put into leaf node collection P, P={T1, T2}.When leaf node T1 is no longer relied on by any node, namely not with this node for arc tail, just this node is shifted out set of node P, P={T2}, puts into isolated node collection T, T={T1};
S3, integration order <T2, T1> are added in the sequence S of integration order result of tables of data, S={<T2, T1>};
S4, continue from leaf node collection P={ T2 } appoint taking-up node T2;
S5, arbitrary the limit <T3 that to take out with this node T2 be arc tail, T2>, carry out expansion to the arc head node T3 on this limit and integrate, namely the full detail of node T2 is injected in node T3.Node T3 expands complete, namely not with this limit for arc head, just node is put into leaf node collection P, P={T2, T3}.When leaf node T2 is no longer relied on by any node, namely not with this node for arc tail, just this node is shifted out set of node P, P={T3}, puts into isolated node collection T, T={T1, T2};
S6, integration order <T3, T2> are added in the sequence S of integration order result of tables of data, S={<T2, T1>, <T3, T2>};
S7, continue to appoint taking-up node T3 from leaf node collection P={T3}, the limit being not arc tail with this node T2, shifts out set of node P this node, P={}, puts into isolated node collection T, T={T1, T2, T3};
S8, now, P={}, P are empty set;
The sequence S of the integration order result of S9, output tables of data, S={<T2, T1>, <T3, T2>}.
As the further improvement of described a kind of database data migration method based on inverse topology, described step C comprises:
C1, the data only appearing at the node of arc tail are moved to respectively in tables of data corresponding in NoSQL database;
C2, according to extended order sequence, according to the pattern after conversion successively by tables of data corresponding to NoSQL database for the Data Migration of residue node.
As the further improvement of described a kind of database data migration method based on inverse topology, described step C2 comprises:
Integration order in C21, successively taking-up extended order sequence;
C22, the information of arc tail node in the integration order of taking-up incorporated in arc head node and arc head node is expanded;
C23, by tables of data corresponding to NoSQL database for the Data Migration of arc head node after expansion.
The specific embodiment two of step C of the present invention is as follows:
With reference to figure 3, wherein, extended order sequence:
S=[<T2,T1>,<T3,T1>,<T6,T5>,<T3,T2>,<T6,T2>]。
1, first the data batchmove in all leaf node T1, T5 in the table T1 ' in NoSQL database and T5 ';
2, taking out tuple first tuple <T2, the T1> in S, the full content of T1 is incorporated in T2, is T2 after T2 expansion ", T2 " data batchmove be put in the T2 ' of NoSQL database;
3, sequence spreading [<T3 remaining in the extended method treatment S of step 2 is repeated, T1>, <T6, T5>, <T3, T2>, <T6, T2>] until S is empty set.
More than that better enforcement of the present invention is illustrated, but the invention is not limited to described embodiment, those of ordinary skill in the art also can make all equivalent variations or replacement under the prerequisite without prejudice to spirit of the present invention, and these equivalent distortion or replacement are all included in the application's claim limited range.
Claims (5)
1., based on a database data migration method for inverse topology, it is characterized in that: comprise the following steps:
A, the database moved as required, obtain its topological relation figure;
B, according to topological relation figure and patten transformation model, draw the extended order sequence in transition process;
Data and pattern are migrated in NoSQL by the extended order sequence that C, basis obtain.
2. a kind of database data migration method based on inverse topology according to claim 1, is characterized in that: described step B comprises:
B1, from the leaf node topological relation figure concentrate appoint get a node as start node;
B2, to process with the limit of this start node for arc tail node all, obtain the integration order of its corresponding sides;
B3, this start node is shifted out leaf node collection, put into isolated node collection;
B4, integration order processing procedure obtained are added to extended order sequence;
B5, return perform step B1, until leaf node integrates as empty set.
3. a kind of database data migration method based on inverse topology according to claim 2, is characterized in that: described step B2 comprises:
B21, appoint arbitrary the limit that to get with this start node be arc tail node, and expansion integration is carried out to the arc head node on this limit, obtain the integration order on this limit;
B22, judge whether this arc head node is leaf node when not comprising this limit, if so, then this starting point is put into leaf node collection; Otherwise, perform step B23;
B23, return perform step B21, until all limits that this start node is arc tail node are all disposed.
4. a kind of database data migration method based on inverse topology according to claim 1, is characterized in that: described step C comprises:
C1, the data only appearing at the node of arc tail are moved to respectively in tables of data corresponding in NoSQL database;
C2, according to extended order sequence, according to the pattern after conversion successively by tables of data corresponding to NoSQL database for the Data Migration of residue node.
5. a kind of database data migration method based on inverse topology according to claim 4, is characterized in that: described step C2 comprises:
Integration order in C21, successively taking-up extended order sequence;
C22, the information of arc tail node in the integration order of taking-up incorporated in arc head node and arc head node is expanded;
C23, by tables of data corresponding to NoSQL database for the Data Migration of arc head node after expansion.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410758140.5A CN104504012A (en) | 2014-12-10 | 2014-12-10 | Database data migration method based on inverse topology |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201410758140.5A CN104504012A (en) | 2014-12-10 | 2014-12-10 | Database data migration method based on inverse topology |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN104504012A true CN104504012A (en) | 2015-04-08 |
Family
ID=52945410
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201410758140.5A Pending CN104504012A (en) | 2014-12-10 | 2014-12-10 | Database data migration method based on inverse topology |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN104504012A (en) |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106294665A (en) * | 2016-08-05 | 2017-01-04 | 浪潮软件股份有限公司 | Method and device for storing student status data |
| CN107832461A (en) * | 2017-11-28 | 2018-03-23 | 中国银行股份有限公司 | A kind of data migration method and system |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102799570A (en) * | 2012-07-20 | 2012-11-28 | 中山大学 | Data sheet association analysis method |
| US20140172789A1 (en) * | 2006-01-31 | 2014-06-19 | International Business Machines Corporation | Migration of data from a legacy system |
-
2014
- 2014-12-10 CN CN201410758140.5A patent/CN104504012A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20140172789A1 (en) * | 2006-01-31 | 2014-06-19 | International Business Machines Corporation | Migration of data from a legacy system |
| CN102799570A (en) * | 2012-07-20 | 2012-11-28 | 中山大学 | Data sheet association analysis method |
Non-Patent Citations (1)
| Title |
|---|
| GANSEN ZHAO 等: "Schema Conversion Model of SQL Database to NoSQL", 《2014 NINTH INTERNATIONAL CONFERENCE ON P2P, PARALLEL, GRID, CLOUD AND INTERNET COMPUTING》 * |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN106294665A (en) * | 2016-08-05 | 2017-01-04 | 浪潮软件股份有限公司 | Method and device for storing student status data |
| CN107832461A (en) * | 2017-11-28 | 2018-03-23 | 中国银行股份有限公司 | A kind of data migration method and system |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017096939A1 (en) | Method for establishing index on hdfs-based spark-sql big-data processing system | |
| CN103761318B (en) | A kind of method and system of relationship type synchronization of data in heterogeneous database | |
| JP6964384B2 (en) | Methods, programs, and systems for the automatic discovery of relationships between fields in a mixed heterogeneous data source environment. | |
| US20180060389A1 (en) | Query optimization over distributed heterogeneous execution engines | |
| CN104809190B (en) | A kind of database access method of tree structure data | |
| CN105205105A (en) | Data ETL (Extract Transform Load) system based on storm and treatment method based on storm | |
| WO2022143045A1 (en) | Method and apparatus for determining data blood relationship, and storage medium and electronic apparatus | |
| US20150149472A1 (en) | For all entries processing | |
| CN103092996B (en) | Document data acquisition device and bills data acquisition methods | |
| CN102135995A (en) | Extract transform and load (ETL) data cleaning design method | |
| CN104462351B (en) | A kind of data query model and method towards MapReduce patterns | |
| CN105938479B (en) | A kind of structure conversion method of relation table and non-relation table | |
| CN105740264A (en) | Distributed XML database sorting method and apparatus | |
| CN104731922A (en) | System and method for rapidly retrieving structural data based on distributed type database HBase | |
| WO2014094331A1 (en) | Virtual table indexing mechanism and method capable of realizing multi-attribute compound condition query | |
| WO2016070529A1 (en) | Method and device for achieving duplicated data deletion | |
| CN103678621A (en) | SQL statement optimization method based on constant substitution | |
| WO2017088666A1 (en) | Data storage method and coordinator node | |
| CN103617273A (en) | SOL script objectification method and system | |
| CN107818114A (en) | A kind of data processing method, device and database | |
| EP3513306A1 (en) | N-way hash join | |
| CN105447105A (en) | NoSQL-based single field section index query method for distributed Internet of things data | |
| TWI539309B (en) | Synchronization system for transforming database and method thereof | |
| WO2018090557A1 (en) | Method and device for querying data table | |
| CN104504012A (en) | Database data migration method based on inverse topology |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20150408 |