CN101739436A - XML-based flexible data migration method - Google Patents
XML-based flexible data migration method Download PDFInfo
- Publication number
- CN101739436A CN101739436A CN 200910152748 CN200910152748A CN101739436A CN 101739436 A CN101739436 A CN 101739436A CN 200910152748 CN200910152748 CN 200910152748 CN 200910152748 A CN200910152748 A CN 200910152748A CN 101739436 A CN101739436 A CN 101739436A
- Authority
- CN
- China
- Prior art keywords
- data
- document
- xml
- xml document
- create
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a data processing method, which aims to utilize the characteristics of the expandability, structuring, self-description and cross-platform performance of XML to provide an XML-based data migration model to extract, process and exchange the data of each conventional application system or the application systems to be developed to realize data exchange and integration, solve the problem of the migration and identification of the historical data of a database when a database application system is changed, and migrate key historical data to a new-version system or the changed system without damage. The technical scheme of a flexible data migration method comprises three parts which are data extraction, data conversion and data importing. Through the XML-based flexible data migration method of the invention, an enterprise can expand the usable range of an information system on the basis of the conventional information system and make full use of conventional information resources; or the enterprise securely and accurately migrate the conventional data resources of value in use into the new system when redesigning the information system.
Description
Technical field
The present invention relates to a kind of data processing method, particularly a kind of flexible data migration method based on XML.
Background technology
Along with development of computer, human various social activitieies more and more be unable to do without information.Effective information is most important to making a strategic decision, and what have is many more, will be in more advantageously position in competition.Data are as the carrier of information, and its management tool database more and more obtains people's attention for importance of information technology.The application of database technology and nearly more than the 40 years so far history of development, it is one of computer realm subject with fastest developing speed, is a very practical technology, also be one involve a wide range of knowledge, subject that research range is wide.Construction is the information application system of core with the database, and for the benefit that improves enterprise, the management that improves enterprise is significant, along with expanding economy, various infosystems are arisen at the historic moment, and they have not only brought benefit for enterprise, have also brought convenience for the client of enterprise simultaneously.
In the informatization process of enterprise, because the complicacy and the development of computer of business event, making up operation system at different times may be based on different operating system platform and database technology, the data communicative effect is poor each other, thereby, can not realize information sharing and mutual between each system effectively, form " information island " phenomenon, be faced with the problem of a data migration.Original old system replaces from enabling by new system, has accumulated a large amount of historical datas during use, and wherein many historical datas all are that new system is necessary, and these historical datas also are the important evidence of carrying out decision analysis simultaneously.Data migtation cleans these historical datas, change exactly, and is loaded into the process in the new system.Data migtation is applicable to that mainly the old system of a cover switches to another set of new system, or overlaps old system when switching to same set of new system more, the historical data in the old system need be transformed into the situation in the new system.When field generation systems such as bank, telecommunications, the tax, industry and commerce, insurance and sale switch, generally all need to carry out data migtation.Yet in the application system escalation process of reality, some usually can take place the new and old edition database table structure changes, and can not use ready-made DB Backup, operation such as imports and exports and finishes data migtation.Therefore,, the most important thing is that the source database and the destination data storehouse both sides that carry out data migtation will reach unified understanding to the pattern of data, could realize functions such as the automatic circulation of data and processing like this for data migtation.
Summary of the invention
The present invention is intended to utilize extensibility, structuring, self-described and the cross-platform characteristics of XML, proposition is based on the data migtation model of XML, data to each existing application system that maybe will have extract, handle and exchange, realize exchanges data and integrated, problem is distinguished in database historical data migration when solving the change database application system, and the historical data of key is not moved in the system of redaction or the system after changing with damaging.
Flexible data migration method is based on XML data migtation model and realizes, because the data of XML are tree structures, is semi-structured, and data may be nested mixed types.And relational database is structurized, and field numerical value is some the most basic indivisible entity attributes.The theoretical foundation of relational database is a relational model, and it is represented by the form of table; And the document of XML foundation is XML Schemas or DTD.Corresponding relation how to set up these two kinds of models is the matter of utmost importance that data migtation will solve.Draw thus, the technical scheme of flexible data migration method comprises three partial contents: data pick-up, data-switching and data importing.
A kind of flexible data migration method based on XML may further comprise the steps:
One, data pick-up
(1) at first by the data in the JDBC-ODBC bridge extraction source database;
(2) mapping is set, adjusts mapped file, make that all Useful Informations can both be mapped in the tables of data, the retrieval mapping;
(3) the source inquiry of retrieve stored in attribute of an element; The data storage that extracts in transient document, and is become this data map the data structure of XML document; Create the XML document of a blank then, data element in the transient document is added in XML document successively, and in XML document, create a corresponding new element with correct title, retrieve corresponding data then and also these data are arranged to the content of this element, after finishing the element loading, in XML document, add respective attributes again;
Two, data-switching
(1) sets up the object of a parsing, XML document is imported the object of resolving;
(2) set up the object that another one is resolved, in the XSL document of data importing with the extraction of JDBC-ODBC bridge, then the XSL document is imported the object of resolving;
(3) object with above-mentioned two parsings merges the final XML document of generation;
Three, data importing
(1) at first creates a class A, create a resolver, read each element in the XML document then, and create the element that a transient document storage is read with class A;
(2) retrieve the root element that comprises all data in the transient document, create a class B then, pass through the masurium retrieve data, and return a node tabulation, use the node tabulation to visit each element at last with class B; If comprise daughter element in the element, then need to create Multiple Cycle and realize; At last data are inserted in the database.
By the flexible data migration method based on XML of the present invention, enterprise can be on the basis of original information system, and the usable range of extend information system makes full use of the existing information resource; Perhaps when the redesign infosystem, the data resource safety and precise that originally had value is moved in the new system.
Description of drawings
Fig. 1 is the data pick-up schematic flow sheet in the specific embodiments of the invention;
Fig. 2 is the data-switching schematic flow sheet in the specific embodiments of the invention;
Fig. 3 is the data importing schematic flow sheet in the specific embodiments of the invention.
Embodiment
In conjunction with the accompanying drawings, by the present invention will be further described.
One, data pick-up
This method mainly is to utilize the data pick-up device to carry out data pick-up by JDBC access originator database.It extracts principle is by the data in the JDBC extraction disparate databases, and they are converted to the XML document data by JDOM, then several XML document are become a document according to certain rule set, realize the conversion of unified XML document in each heterogeneous database.XML describes data from the different pieces of information source with a kind of unified data pattern, thereby the isomerism of applied environment and data structure in the shadow data source is realized the migration of heterogeneous database.As shown in Figure 1:
(1) first step of creating new document is the retrieval mapping, finishes by resolving mapped file, because this file also comprises quoting of data retrieved the most at last, therefore must resolve it before can carrying out any database manipulation.
(2) the source inquiry of retrieve stored in the sql of data element attribute: determine root element earlier, retrieve the data node then, can also utilize the several inquiring structuring files of similar techniques from moving successively, the coercion of type with Node converts Element to again, so that can use the Attribute value.
(3) with data storage in the Document object, successfully after database has extracted data, these data just are stored among the interim Document.Promptly earlier create an empty file, create a row element for each line data again, each tabulation is shown as an element with this row name, the content of data itself as this element with root element data.
(4) retrieve data mapping: after having these data, just it should be mapped to new construction, from resolving mapping document retrieval mapping.At first, retrieve the information of relevant root element and row element, retrieve element map itself then, so just can make up needed new XML document.
(5) circular treatment primary data: each initial provisional capital is stored in the transient document with the form of ROW element, it is retrieved as NodeList, in XML document, add new element then, promptly create a new element, retrieve the content that suitable data also is arranged to these data this element then.After finishing the element loading, just can in new document, add attribute.
(6) generate final XML document.
Two, data-switching
(1) set up the object that a Microsoft Parser (XMLDOM) resolves, and the interim XML document that the data that the JDBC extraction is come form is read in internal memory;
(2) set up the object that another one Microsoft Parser (XMLDOM) resolves, and import predefine XSL document;
(3) interim XML document is changed with the XSL document.
Its process as shown in Figure 2.
Three, data importing
The importing process and the extraction process of data are similar, utilize data writing device to carry out data importing by JDBC visit destination data storehouse.Its process is as shown in Figure 3:
(1) reads the XML file.At first, resolution file also obtains original XML document, reads each element then creating a Document object, and the circular treatment element.
(2) circular treatment data.At first retrieval comprises the root element of all data, and by the masurium retrieve data, and return a Nodelist, use it to visit each element then.If comprise daughter element in the element, then need to create Multiple Cycle and realize.
(3) retrieve data and data are inserted in the database.
Claims (1)
1. the flexible data migration method based on XML is characterized in that, may further comprise the steps:
One, data pick-up
(1) at first by the data in the JDBC-ODBC bridge extraction source database;
(2) mapping is set, adjusts mapped file, make that all Useful Informations can both be mapped in the tables of data, the retrieval mapping;
(3) the source inquiry of retrieve stored in attribute of an element; The data storage that extracts in transient document, and is become this data map the data structure of XML document; Create the XML document of a blank then, data element in the transient document is added in XML document successively, and in XML document, create a corresponding new element with correct title, retrieve corresponding data then and also these data are arranged to the content of this element, after finishing the element loading, in XML document, add respective attributes again;
Two, data-switching
(1) sets up the object of a parsing, XML document is imported the object of resolving;
(2) set up the object that another one is resolved, in the XSL document of data importing with the extraction of JDBC-ODBC bridge, then the XSL document is imported the object of resolving;
(3) object with above-mentioned two parsings merges the final XML document of generation;
Three, data importing
(1) at first creates a class A, create a resolver, read each element in the XML document then, and create the element that a transient document storage is read with class A;
(2) retrieve the root element that comprises all data in the transient document, create a class B then, pass through the masurium retrieve data, and return a node tabulation, use the node tabulation to visit each element at last with class B; If comprise daughter element in the element, then need to create Multiple Cycle and realize; At last data are inserted in the database.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910152748 CN101739436B (en) | 2009-09-28 | 2009-09-28 | XML-based flexible data migration method |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 200910152748 CN101739436B (en) | 2009-09-28 | 2009-09-28 | XML-based flexible data migration method |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101739436A true CN101739436A (en) | 2010-06-16 |
| CN101739436B CN101739436B (en) | 2012-12-19 |
Family
ID=42462926
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 200910152748 Active CN101739436B (en) | 2009-09-28 | 2009-09-28 | XML-based flexible data migration method |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101739436B (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081661A (en) * | 2011-01-19 | 2011-06-01 | 吉林大学 | Data integration method and system of heterogeneous relational database based on XML (Extensive Makeup Language) |
| CN102662705A (en) * | 2012-03-31 | 2012-09-12 | 中国工商银行股份有限公司 | System and method for upgrading system environment of computer cluster |
| CN102799570A (en) * | 2012-07-20 | 2012-11-28 | 中山大学 | Data sheet association analysis method |
| CN105320763A (en) * | 2015-10-22 | 2016-02-10 | 江苏国泰新点软件有限公司 | Method and apparatus for importing contents of XML file into database |
| CN105354211A (en) * | 2015-09-24 | 2016-02-24 | 四川长虹电器股份有限公司 | MongoDB based method for realizing XBRL instance document access |
| CN105512237A (en) * | 2015-11-30 | 2016-04-20 | 用友网络科技股份有限公司 | Data introduction system with complex structure |
| CN105740462A (en) * | 2016-03-02 | 2016-07-06 | 上海新炬网络信息技术有限公司 | Method for supporting data migration between different environments |
| CN105808539A (en) * | 2014-12-29 | 2016-07-27 | 北京航天测控技术有限公司 | Data import and export method and device |
| CN109597917A (en) * | 2018-10-17 | 2019-04-09 | 中国工程物理研究院计算机应用研究所 | The method that XML Schema document is converted into XSL document |
| CN109840243A (en) * | 2018-12-28 | 2019-06-04 | 北京航天测控技术有限公司 | A kind of digitlization O&M support process data conversion method based on multistage rule |
| CN110019139A (en) * | 2017-12-29 | 2019-07-16 | 中国移动通信集团内蒙古有限公司 | Method, apparatus, electronic equipment and the storage medium of Data Migration |
| CN110110151A (en) * | 2018-01-05 | 2019-08-09 | 北大方正集团有限公司 | Method, system, computer equipment and the storage medium of resource Packet analyzing storage |
| WO2020000725A1 (en) * | 2018-06-29 | 2020-01-02 | 平安科技（深圳）有限公司 | Electronic apparatus, data migration method, and storage medium |
| CN111104556A (en) * | 2019-11-19 | 2020-05-05 | 泰康保险集团股份有限公司 | Service processing method and device |
| CN111352918A (en) * | 2018-12-21 | 2020-06-30 | 卓望数码技术（深圳）有限公司 | Terminal database data migration method and system |
Family Cites Families (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1217270C (en) * | 2002-03-14 | 2005-08-31 | 上海网上乐园信息技术有限公司 | System for backing up isomerous data in same network and its realization method |
| CN100458793C (en) * | 2007-05-10 | 2009-02-04 | 浪潮集团山东通用软件有限公司 | Mapping conversion method between data access level Xml format data and relational data |
| CN101282345B (en) * | 2008-03-03 | 2011-12-07 | 北京航空航天大学 | Method for converting XML based on formalization description and state ontroller system |
-
2009
- 2009-09-28 CN CN 200910152748 patent/CN101739436B/en active Active
Cited By (21)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102081661A (en) * | 2011-01-19 | 2011-06-01 | 吉林大学 | Data integration method and system of heterogeneous relational database based on XML (Extensive Makeup Language) |
| CN102662705A (en) * | 2012-03-31 | 2012-09-12 | 中国工商银行股份有限公司 | System and method for upgrading system environment of computer cluster |
| CN102662705B (en) * | 2012-03-31 | 2015-06-03 | 中国工商银行股份有限公司 | System and method for upgrading system environment of computer cluster |
| CN102799570A (en) * | 2012-07-20 | 2012-11-28 | 中山大学 | Data sheet association analysis method |
| CN105808539A (en) * | 2014-12-29 | 2016-07-27 | 北京航天测控技术有限公司 | Data import and export method and device |
| CN105354211A (en) * | 2015-09-24 | 2016-02-24 | 四川长虹电器股份有限公司 | MongoDB based method for realizing XBRL instance document access |
| CN105320763A (en) * | 2015-10-22 | 2016-02-10 | 江苏国泰新点软件有限公司 | Method and apparatus for importing contents of XML file into database |
| CN105320763B (en) * | 2015-10-22 | 2019-06-07 | 江苏国泰新点软件有限公司 | A kind of method and apparatus that the content by XML file imports database |
| CN105512237A (en) * | 2015-11-30 | 2016-04-20 | 用友网络科技股份有限公司 | Data introduction system with complex structure |
| CN105512237B (en) * | 2015-11-30 | 2018-11-16 | 用友网络科技股份有限公司 | A kind of data import system of labyrinth |
| CN105740462A (en) * | 2016-03-02 | 2016-07-06 | 上海新炬网络信息技术有限公司 | Method for supporting data migration between different environments |
| CN110019139A (en) * | 2017-12-29 | 2019-07-16 | 中国移动通信集团内蒙古有限公司 | Method, apparatus, electronic equipment and the storage medium of Data Migration |
| CN110110151A (en) * | 2018-01-05 | 2019-08-09 | 北大方正集团有限公司 | Method, system, computer equipment and the storage medium of resource Packet analyzing storage |
| WO2020000725A1 (en) * | 2018-06-29 | 2020-01-02 | 平安科技（深圳）有限公司 | Electronic apparatus, data migration method, and storage medium |
| CN109597917A (en) * | 2018-10-17 | 2019-04-09 | 中国工程物理研究院计算机应用研究所 | The method that XML Schema document is converted into XSL document |
| CN109597917B (en) * | 2018-10-17 | 2022-03-01 | 中国工程物理研究院计算机应用研究所 | Method for converting XML Schema document into XSL document |
| CN111352918A (en) * | 2018-12-21 | 2020-06-30 | 卓望数码技术（深圳）有限公司 | Terminal database data migration method and system |
| CN111352918B (en) * | 2018-12-21 | 2023-05-23 | 卓望数码技术（深圳）有限公司 | Terminal database data migration method and system |
| CN109840243A (en) * | 2018-12-28 | 2019-06-04 | 北京航天测控技术有限公司 | A kind of digitlization O&M support process data conversion method based on multistage rule |
| CN111104556A (en) * | 2019-11-19 | 2020-05-05 | 泰康保险集团股份有限公司 | Service processing method and device |
| CN111104556B (en) * | 2019-11-19 | 2023-09-15 | 泰康保险集团股份有限公司 | Service processing method and device |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101739436B (en) | 2012-12-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101739436B (en) | XML-based flexible data migration method | |
| CN100458793C (en) | Mapping conversion method between data access level Xml format data and relational data | |
| US7805341B2 (en) | Extraction, transformation and loading designer module of a computerized financial system | |
| CN107122443A (en) | A kind of distributed full-text search system and method based on Spark SQL | |
| US9317518B2 (en) | Data synchronization | |
| US7539701B2 (en) | Generic infrastructure for migrating data between applications | |
| CN102760165B (en) | Full text retrieval method using bitmap index and device | |
| US9563650B2 (en) | Migrating federated data to multi-source universe database environment | |
| CN103365852A (en) | Concurrency control method and system for document library systems | |
| CN104504008B (en) | A kind of Data Migration algorithm based on nested SQL to HBase | |
| CN102054034A (en) | Implementation method for business basic data persistence of enterprise information system | |
| CN106528898A (en) | Method and device for converting data of non-relational database into relational database | |
| CN101706801A (en) | Method and system for managing memory database data | |
| CN106055678A (en) | Hadoop-based panoramic big data distributed storage method | |
| CN103034703A (en) | Method for data exchange among multiple systems based on rule configuration | |
| CN112001021A (en) | BIM-based classification coding method and system | |
| CN111061679A (en) | Method and system for rapid configuration of technological innovation policy based on rete and drools rules | |
| CN101963993B (en) | Method for fast searching database sheet table record | |
| CN102760164B (en) | A kind of method of exchanges data between relational database management system and XML data base management system | |
| CN110955658B (en) | Data organization and storage method based on Java intelligent contract | |
| CN101916260A (en) | Method for establishing semantic mapping between disaster body and relational database | |
| US7574329B1 (en) | Object model for decision and issue tracking | |
| CN116049193A (en) | Data storage method and device | |
| US20080243876A1 (en) | Creation of generic hierarchies | |
| CN104834742A (en) | ETL architecture management method based on SCA |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| ASS | Succession or assignment of patent right |
Free format text: FORMER OWNER: WANG NA YOU GANG ZHENG LILI CHEN WEI Effective date: 20121022 Owner name: ZHEJIANG UNIVERSITY OF INDUSTRY AND COMMERCE Free format text: FORMER OWNER: SUN BIN Effective date: 20121022 |
|
| C41 | Transfer of patent application or patent right or utility model | ||
| C53 | Correction of patent for invention or patent application | ||
| CB03 | Change of inventor or designer information |
Inventor after: Ju Chunhua Inventor after: Sun Bin Inventor after: Wang Na Inventor after: You Gang Inventor after: Zheng Lili Inventor after: Chen Wei Inventor before: Sun Bin Inventor before: Wang Na Inventor before: You Gang Inventor before: Zheng Lili Inventor before: Chen Wei |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: INVENTOR; FROM: SUN BIN WANG NA YOU GANG ZHENG LILI CHEN WEI TO: JU CHUNHUA SUN BIN WANG NA YOU GANG ZHENG LILI CHEN WEI |
|
| TA01 | Transfer of patent application right |
Effective date of registration: 20121022 Address after: Hangzhou City, Zhejiang province 310018 Xiasha Higher Education Park is 18 street. Applicant after: Zhejiang Gongshang University Address before: 310018 School of information, Zhejiang Gongshang University, 18 Jiao Cheng street, Xiasha Higher Education Zone, Hangzhou, Zhejiang Applicant before: Sun Bin Applicant before: Wang Na Applicant before: You Gang Applicant before: Zheng Lili Applicant before: Chen Wei |
|
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant |