CN110532244A - A kind of lightweight legacy system data transfer device and its system - Google Patents
A kind of lightweight legacy system data transfer device and its system Download PDFInfo
- Publication number
- CN110532244A CN110532244A CN201910770679.5A CN201910770679A CN110532244A CN 110532244 A CN110532244 A CN 110532244A CN 201910770679 A CN201910770679 A CN 201910770679A CN 110532244 A CN110532244 A CN 110532244A
- Authority
- CN
- China
- Prior art keywords
- data
- conversion
- historical data
- rule
- storage
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
- 238000012546 transfer Methods 0.000 title claims abstract description 16
- 238000006243 chemical reaction Methods 0.000 claims abstract description 101
- 238000013507 mapping Methods 0.000 claims abstract description 56
- 238000013500 data storage Methods 0.000 claims abstract description 18
- 238000012937 correction Methods 0.000 claims description 9
- 238000013481 data capture Methods 0.000 claims description 7
- 238000000034 method Methods 0.000 claims description 7
- 230000008569 process Effects 0.000 claims description 6
- 230000009466 transformation Effects 0.000 claims description 3
- 238000013508 migration Methods 0.000 description 4
- 230000005012 migration Effects 0.000 description 4
- 238000012360 testing method Methods 0.000 description 3
- 238000010276 construction Methods 0.000 description 2
- 238000011161 development Methods 0.000 description 2
- 238000010586 diagram Methods 0.000 description 2
- 238000005516 engineering process Methods 0.000 description 2
- 238000012986 modification Methods 0.000 description 2
- 230000004048 modification Effects 0.000 description 2
- 238000012544 monitoring process Methods 0.000 description 2
- 230000009286 beneficial effect Effects 0.000 description 1
- 238000004364 calculation method Methods 0.000 description 1
- 230000008859 change Effects 0.000 description 1
- 238000004891 communication Methods 0.000 description 1
- 230000007547 defect Effects 0.000 description 1
- 230000007812 deficiency Effects 0.000 description 1
- 238000001914 filtration Methods 0.000 description 1
- 238000012216 screening Methods 0.000 description 1
- 238000012800 visualization Methods 0.000 description 1
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/219—Managing data history or versioning
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/25—Integrating or interfacing systems involving database management systems
- G06F16/258—Data format conversion from or to a database
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a kind of lightweight legacy system data transfer devices, including the data conversion system that the legacy system data storage with old system is connected, data transfer device includes following steps: S1. data conversion system obtains the historical data in the storage of historical data data；S2. data conversion system is regular according to the Mapping and Converting of acquired historical data creation control mapping relations and configuration field grade；S3. data conversion system is successively read the historical data in the storage of legacy system data according to control mapping relations, to filter out the historical data for closing rule in real time, and the historical data for closing rule is successively transformed into the system data storage of new system based on Mapping and Converting rule；Tracing to the source after realizing automation, irregularity correcting data error and the data conversion of historical data conversion is traced.
Description
Technical field
The present invention relates to the technical field of data conversion, refer in particular to a kind of lightweight legacy system data transfer device and
Its system.
Background technique
With the development of informationization technology, a large amount of operation system of enterprise construction, and the operation system of original construction, In
In the case of Technical Architecture continues to develop, occur being not suitable with Developing trend of new technology, in the case where business demand constantly changes, original system
The demand that new business proposes cannot be responded very well, and in the case, enterprise needs to rebuild new operation system or the old system of reconstruct
System, but after building new system, it would be desirable to be able to the seamless switching of business is realized with old system, old system business data need to be turned from the background
Enter into new system, to guarantee the continuity of business.But the data structure of newly-built system is planned and is built according to business new demand
If there are biggish othernesses with old system in data structure.
In the historical data conversion of new-old system, commonly using two kinds of solutions: one, SQL script is write, it is real
Existing library to library migration, i.e., from old system copies data, by filtering, historical data is transferred to newly by the rule such as merging, calculating
In system；Another kind is migrated using correlation ETL tool, that is, passes through intuitive visualization interface, configuration related service rule
Then, old system and new system are connected, historical data is carried out to move to new system.In practical applications, above-mentioned two scheme is all
In some defects:
Scheme one carries out writing transcode for each table, the migration for not only increasing written in code amount, while writing every time
Code is difficult to accomplish to reuse, and because of system complex, data are complicated, is related to largely migrating test, migration test can because
One misregistration causes all records that can not migrate, and increases test complexity, increases conversion operability, brings a large amount of
Unnecessary development cost, while code is also difficult to safeguard；
Scheme two is implemented using related ETL tool, on the one hand needs input cost to buy tool, while because of complex data
Handle regular bring deployment cost, and flexibility is poor, and implement to be disposable historical data migration, can not accomplish limit at
Originally successively decrease.
Summary of the invention
It is an object of the invention to overcome the deficiencies of the prior art and provide a kind of simplicity, accurate lightweight departments of history
System data transfer device and its system
In order to achieve the above purpose, a kind of lightweight legacy system data transfer device provided by the present invention, including with it is old
The data conversion system that the legacy system data storage of system is connected, data transfer device includes following steps:
S1. data conversion system obtains the historical data in the storage of historical data data；
S2. data conversion system compares the mapping of mapping relations and configuration field grade according to acquired historical data creation
Transformation rule；
S3. data conversion system is successively read the historical data in the storage of legacy system data according to control mapping relations, with reality
When filter out close rule historical data, and based on Mapping and Converting rule by close rule historical data be successively transformed into new system be
In data of uniting storage.
Further, in step s3, the conversion progress for closing the historical data of rule is recorded in real time based on control mapping relations, with
It can be continuous turn of breakpoint according to conversion progress for data conversion system.
Further, in step s3, the history number being successively read according to control mapping relations in the storage of legacy system data
According to, and record generates the error in data log of the historical data about irregularity in real time, wherein it can be to the historical data of irregularity
After being modified, converted based on Mapping and Converting rule into the system data storage of new system.
Further, in data conversion process in step s3, record generates the historical data converted old in real time
Major key mapping relations between system and new system.
A kind of lightweight legacy system data conversion system is connected including the legacy system data storage with old system
Data conversion system, the data conversion system include:
Data capture unit, for obtaining the historical data in the storage of legacy system data；
Map unit creates control mapping relations and configuration words for the historical data according to acquired in data capture unit
The Mapping and Converting rule of section grade；
Date Conversion Unit, for being successively read historical data acquired in acquiring unit according to control mapping relations with real-time
The historical data for closing rule is sifted out, and the historical data for closing rule is successively transformed into the system number of new system based on Mapping and Converting rule
According in storage.
Further, the data conversion system further include conversion recording unit, wherein the conversion recording unit based on pair
It records the conversion progress for closing the historical data of rule in real time according to mapping relations, enables the Date Conversion Unit based on the conversion recorded
Progress makees continuous turn of breakpoint.
Further, the conversion recording unit is also used to record to generate in real time and not conform to about what Date Conversion Unit was sifted out
The error log of historical data is advised, and historical data corresponding to error log is carried out manually configured with data correction unit
Or automatic amendment, revised historical data are successively transformed into the system data storage of new system by Date Conversion Unit.
Further, in data conversion process, the conversion recording unit is also used to record in real time and generate about converting
At major key mapping relations of the historical data between old system and new system.
The present invention uses above-mentioned scheme, the beneficial effect is that: pass through creation control mapping relations and configuration field grade
Mapping and Converting rule, in order to legacy system data storage historical data successively convert to new system system data storage
In, meanwhile, also have the function of recording conversion progress, error log, major key mapping relations, realizes oneself of historical data conversion
Tracing to the source after dynamicization, irregularity correcting data error and data conversion is traced.
Detailed description of the invention
Fig. 1 is the flow diagram of data transfer device of the invention.
Fig. 2 is the schematic diagram of data conversion system of the invention.
Specific embodiment
To facilitate the understanding of the present invention, with reference to the accompanying drawings to invention is more fully described.It is given in attached drawing
Better embodiment of the invention.But the invention can be realized in many different forms, however it is not limited to described herein
Embodiment.The purpose of providing these embodiments is that making to understand more thorough and comprehensive to the disclosure.
Referring to shown in attached drawing 1 and attached drawing 2, in the present embodiment, a kind of lightweight legacy system data conversion system, data
Converting system is connected with the legacy system data storage of old system, in addition, data conversion system includes the data being in communication with each other
Acquiring unit, map unit, Date Conversion Unit and conversion recording unit.
In the present embodiment, data capture unit and the legacy system data storage of old system communicate, and go through for obtaining
Historical data in the storage of history system data, wherein historical data acquired in data capture unit for Date Conversion Unit and
Map unit makees data application.Map unit is according to acquired historical data creation control mapping relations and configuration field grade
Mapping and Converting rule, wherein the Mapping and Converting rule of the present embodiment include automatically generate sequence number, code mapping, field cut
It takes, field merging, field format conversion, the transformation rules such as logic calculation, this Mapping and Converting rule can be according to practical old system
Historical data situation and independently set.
In the present embodiment, Date Conversion Unit is successively read history acquired in acquiring unit for compareing mapping relations
Data, and data rule is advised in presetting conjunction, to sift out the historical data for closing rule or irregularity in historical data, wherein right
In the historical data for closing rule, Date Conversion Unit is successively transformed into the system data storage of new system based on Mapping and Converting rule.
In the present embodiment, turned come Real-time Monitoring Data converting unit in data screening and data by conversion recording unit
Data conversion situation during changing, wherein the monitoring function of the conversion recording unit of the present embodiment includes the note of conversion progress
Record, the record of irregularity data, major key mapping relations record.
For converting the writing function of progress, the conversion recording unit of the present embodiment is based on control mapping relations and records in real time
The conversion progress of the historical data of rule is closed, the conversion progress recorded makees data application for Date Conversion Unit, that is, data conversion
Unit makees continuous turn of breakpoint according to the conversion progress recorded, makes conversion process after pause is interrupted, can be in conversion next time by it
Preceding conversion progress continues data conversion.
For the writing function of irregularity data, the conversion recording unit of the present embodiment records generation in real time and turns about data
Change the error log for the irregularity historical data that unit is sifted out.Conversion recording unit error log generated is looked into for user
It sees.It is also configured with the data correction unit communicated with conversion recording unit and data correction unit, data correction unit is used for
Manual or automatic amendment is carried out to the historical data of irregularity corresponding to error log, that is, modification rule can be preset for number
Irregularity can be gone through by data correction unit by making automatic amendment or user according to historical data of the amending unit to irregularity
History data make artificial correction.Turn in addition, being based on mapping by Date Conversion Unit through the revised historical data of data correction unit
It changes rule to be successively transformed into the system data storage of new system, realizes the amendment conversion of irregularity data.
For the writing function of major key mapping relations, the conversion recording unit of the present embodiment records generation about conversion in real time
Major key (being also " the primary keyword ") mapping relations of the historical data of completion between old system and new system, in order to history number
According to having one-to-one mapping relations in new-old system, it is more convenient for tracing to the source to the data after data conversion and traces.
For the ease of the understanding of those skilled in the art, spy in conjunction with specific embodiments makes further data transfer device
It illustrates.
In the present embodiment, data transfer device includes following steps:
S1. the data capture unit of data conversion system obtains the historical data in the storage of historical data data；
S2. the map unit of data conversion system historical data according to acquired in acquiring unit creation control mapping relations, with
And the Mapping and Converting rule of configuration field grade；
S3. the Date Conversion Unit of data conversion system is successively read in the storage of legacy system data according to control mapping relations
Historical data to filter out the historical data for closing rule in real time, and is successively turned the historical data for closing rule based on Mapping and Converting rule
It changes in the system data storage of new system.
Further, in step s3, conversion recording unit records the historical data for closing rule based on control mapping relations in real time
Conversion progress, so that Date Conversion Unit is according to conversion progress can breakpoint be continuous turns.
Further, in step s3, the history number being successively read according to control mapping relations in the storage of legacy system data
According to filter out the historical data of irregularity in real time, and converting recording unit and record in real time and generate history about irregularity
The error in data log of data, wherein the historical data of irregularity can be based on Mapping and Converting after data correction unit is corrected
Rule is converted by Date Conversion Unit into the system data storage of new system.
Further, in data conversion process in step s3, record generates the historical data converted old in real time
Major key mapping relations between system and new system.
The embodiment of the above is only presently preferred embodiments of the present invention, not does limit in any form to the present invention
System.Anyone skilled in the art utilizes the skill of the disclosure above without departing from the scope of the technical proposal of the invention
Art content makes more possible variations and retouching to technical solution of the present invention, or modification is equivalent embodiment of the invention.
Therefore all contents without departing from technical solution of the present invention, the equivalent equivalence changes that thinking according to the present invention is done should all be covered by
In protection scope of the present invention.
Claims (8)
1. a kind of lightweight legacy system data transfer device, the number being connected including the legacy system data storage with old system
According to converting system, it is characterised in that: data transfer device includes following steps:
S1. data conversion system obtains the historical data in the storage of historical data data；
S2. data conversion system compares the mapping of mapping relations and configuration field grade according to acquired historical data creation
Transformation rule；
S3. data conversion system is successively read the historical data in the storage of legacy system data according to control mapping relations, with reality
When filter out close rule historical data, and based on Mapping and Converting rule by close rule historical data be successively transformed into new system be
In data of uniting storage.
2. a kind of lightweight legacy system data transfer device according to claim 1, it is characterised in that: in step S3
In, the conversion progress for closing the historical data of rule is recorded, in real time based on control mapping relations so that data conversion system is according to conversion
Progress can be continuous turn of breakpoint.
3. a kind of lightweight legacy system data transfer device according to claim 1, it is characterised in that: in step S3
In, it is successively read the historical data in the storage of legacy system data according to control mapping relations, and record is generated about not in real time
Close the error in data log of the historical data of rule, wherein after being modified to the historical data of irregularity, be based on Mapping and Converting
Rule conversion is into the system data storage of new system.
4. a kind of lightweight legacy system data transfer device according to claim 1, it is characterised in that: in step s3
Data conversion process in, record generates the major key mapping of the historical data that converts between old system and new system in real time
Relationship.
5. a kind of a kind of lightweight legacy system data conversion system as described in claim 1-4 any one, including with it is old
The data conversion system that the legacy system data storage of system is connected, it is characterised in that: the data conversion system includes:
Data capture unit, for obtaining the historical data in the storage of legacy system data；
Map unit creates control mapping relations and configuration words for the historical data according to acquired in data capture unit
The Mapping and Converting rule of section grade；
Date Conversion Unit, for being successively read historical data acquired in acquiring unit according to control mapping relations with real-time
The historical data for closing rule is sifted out, and the historical data for closing rule is successively transformed into the system number of new system based on Mapping and Converting rule
According in storage.
6. a kind of lightweight legacy system data conversion system according to claim 5, it is characterised in that: the data turn
The system of changing further includes conversion recording unit, wherein the conversion recording unit is recorded in real time based on control mapping relations and closes rule
The conversion progress of historical data, enabling the Date Conversion Unit make based on the conversion progress recorded, breakpoint is continuous to be turned.
7. a kind of lightweight legacy system data conversion system according to claim 6, it is characterised in that: the conversion note
Record unit is also used to record the error log for generating the irregularity historical data sifted out about Date Conversion Unit in real time, and
Manual or automatic amendment, revised history number are carried out to historical data corresponding to error log configured with data correction unit
According in the system data storage for being successively transformed into new system by Date Conversion Unit.
8. a kind of lightweight legacy system data conversion system according to claim 6, it is characterised in that: in data conversion
In the process, the conversion recording unit is also used to record in real time and generate about the historical data converted in old system and new system
Major key mapping relations between system.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910770679.5A CN110532244A (en) | 2019-08-20 | 2019-08-20 | A kind of lightweight legacy system data transfer device and its system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910770679.5A CN110532244A (en) | 2019-08-20 | 2019-08-20 | A kind of lightweight legacy system data transfer device and its system |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN110532244A true CN110532244A (en) | 2019-12-03 |
Family
ID=68662295
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201910770679.5A Pending CN110532244A (en) | 2019-08-20 | 2019-08-20 | A kind of lightweight legacy system data transfer device and its system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110532244A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111641679A (en) * | 2020-04-30 | 2020-09-08 | 广东艾诗凯奇智能科技有限公司 | Data transmission method of wearable massage instrument and electronic equipment |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008293229A (en) * | 2007-05-24 | 2008-12-04 | Hitachi Ltd | History data processor and history data processing method |
| AU2011213842A1 (en) * | 2010-09-03 | 2012-03-22 | Tata Consultancy Services Limited | A system and method of managing mapping information |
| CN105005618A (en) * | 2015-07-21 | 2015-10-28 | 杭州合众数据技术有限公司 | Data synchronization method and system among heterogeneous databases |
-
2019
- 2019-08-20 CN CN201910770679.5A patent/CN110532244A/en active Pending
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2008293229A (en) * | 2007-05-24 | 2008-12-04 | Hitachi Ltd | History data processor and history data processing method |
| AU2011213842A1 (en) * | 2010-09-03 | 2012-03-22 | Tata Consultancy Services Limited | A system and method of managing mapping information |
| CN105005618A (en) * | 2015-07-21 | 2015-10-28 | 杭州合众数据技术有限公司 | Data synchronization method and system among heterogeneous databases |
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN111641679A (en) * | 2020-04-30 | 2020-09-08 | 广东艾诗凯奇智能科技有限公司 | Data transmission method of wearable massage instrument and electronic equipment |
| CN111641679B (en) * | 2020-04-30 | 2023-06-20 | 未来穿戴技术有限公司 | Data transmission method of wearable massager and electronic equipment |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| Marbán et al. | A data mining & knowledge discovery process model | |
| RU2767149C2 (en) | Method and configuration for automated testing system | |
| CN109656820B (en) | Intelligent automatic test system based on CBTC | |
| CN111176613B (en) | Collaborative Task Automatic Decomposition System Based on Architecture Model | |
| CN104077232B (en) | Testing device and method based on use case and source code two-way tracing | |
| CN116090360B (en) | RPA flow recommendation method based on multi-modal entity alignment | |
| CN103605512A (en) | System and method for data verification based on GTechnology platform | |
| CN109902117A (en) | Operation system analysis method and device | |
| CN113987398A (en) | Software self-defined form content web development system and method | |
| CN103530134B (en) | A kind of configurable software platform structure | |
| CN110532244A (en) | A kind of lightweight legacy system data transfer device and its system | |
| CN107229478A (en) | A kind of task distribution modeling method of credible flight control system co-development | |
| CN109614079A (en) | A kind of coding rule inspection system towards avionics field | |
| CN104615540A (en) | Code standard management system | |
| CN107122408A (en) | Information association and its database audit method, auditing system | |
| CN114911773B (en) | Universal meta-model design method | |
| Vaandering | Transitioning CMS to Rucio Data Managment | |
| CN115564373A (en) | Project information data processing method, system, device and medium | |
| CN107589966A (en) | The method and apparatus of management code in software development system | |
| CN105956735A (en) | Construction quality supervision method in engineering supervision | |
| CN111026361A (en) | Software development method and development system | |
| CN115374080B (en) | Method for realizing data pipeline technology based on model drive | |
| CN117708100A (en) | Business big data analysis system based on artificial intelligence | |
| Hacks et al. | Optimizing enterprise architectures using linear integer programming techniques | |
| CN111190802A (en) | Method for automatically generating performance test configuration |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20191203 |