CN106776876A - Data migration method and data mover system - Google Patents

Data migration method and data mover system Download PDF

Info

Publication number
CN106776876A
CN106776876A CN201611074474.6A CN201611074474A CN106776876A CN 106776876 A CN106776876 A CN 106776876A CN 201611074474 A CN201611074474 A CN 201611074474A CN 106776876 A CN106776876 A CN 106776876A
Authority
CN
China
Prior art keywords
data
migration
source system
system data
goal systems
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN201611074474.6A
Other languages
Chinese (zh)
Inventor
万多乐
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Yonyou Network Technology Co Ltd
Original Assignee
Yonyou Network Technology Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Yonyou Network Technology Co Ltd filed Critical Yonyou Network Technology Co Ltd
Priority to CN201611074474.6A priority Critical patent/CN106776876A/en
Publication of CN106776876A publication Critical patent/CN106776876A/en
Pending legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/27Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/21Design, administration or maintenance of databases
    • G06F16/214Database migration support

Landscapes

  • Engineering & Computer Science (AREA)
  • Databases & Information Systems (AREA)
  • Theoretical Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Management, Administration, Business Operations System, And Electronic Commerce (AREA)

Abstract

The invention provides a kind of data migration method and data mover system.Wherein, a kind of data migration method, including:Migrated to before goal systems by source system data, check that rule carries out validity checking to source system data according to default, source system data is processed according to inspection result;And source system data structure is carried out into control configuration with goal systems data structure according to configuration mode is compareed;When source system data is by after validity checking, source system data being migrated to goal systems;When source system data does not pass through validity checking, source system data is repaired and adjusted, until source system data passes through validity checking;The log information produced in record validity checking, control configuration and transition process.By technical scheme, logic is migrated using configurable mode processing data, Data Migration is worked becomes more flexible, easy-to-use, heterogeneous system Data Migration is become simpler, efficiently.

Description

Data migration method and data mover system
Technical field
The present invention relates to a kind of Data Transference Technology field, in particular to a kind of data migration method, one is further related to Plant data mover system.
Background technology
Enterprise resource planning (Enterprise Resource Planning, ERP) refers to set up in information technology On the basis of, with the management thought of systematization, for business decision layer and employee provide the management platform that decision-making runs means.In information Change all-round developing today, increasing enterprise have selected ERP system to manage each side resource and business of enterprise.
But it is so swift and violent in economic development, ecommerce enters development climax, existing under the background of Intranet surprise attack Some ERP systems tend not to meet the demand of all business of client, and substantial amounts of enterprise all does on the basis of ERP system is existing Different degrees of customization development, the thing followed is the demand that ERP system updates.And ERP system has its intrinsic Complexity, update not a duck soup, as the development of big data technology is increasingly faster, data are that client's highest value is produced Thing, so model change will certainly produce the demand to Data Migration, perhaps problem can't be too big for isomorphism system upgrade, But it is related to the upgrading of heterogeneous system, just into one piece of worry of enterprise, either the user of ERP is still for Data Migration The provider of ERP, heterogeneous system data liter is moved the capital to another place and will become not only headache but also a thing for active demand.
Correlation technique will do data structure control between legacy system and new system from code aspect, then by origin system Data query does certain treatment in internal memory in internal memory, processes data into the requirement for meeting goal systems, preserves data Storehouse.Although correlation technique completes the promotion of heterogeneous system data, there is following some deficiency:1st, before Data Migration There is no source system data to check rule, it is impossible to be assessed before migration and avoided risk;2nd, origin system and goal systems data structure Control can not accomplish flexible configuration, if run into field control need modification, can only by change source code solve run into ask Topic;3rd, the data of project customization cannot be solved by existing capability configuration mode, can only be developed by customized project;4th, visitor Family business rule cannot dynamic expansion in existing product, can only be developed by customized project and solved;5th, target is moved to The later data of system do not have origin system to be processed with goal systems reconciliation, it is impossible to the report of automatically generated data verification of correctness, item It is too big that mesh pays error in data potential risk.
Therefore, a kind of data migration method how is provided to solve the above problems, and is asked as technology urgently to be resolved hurrily at present Topic
The content of the invention
It is contemplated that at least solving one of technical problem present in prior art or correlation technique.
Therefore, it is an object of the present invention to propose a kind of data migration method.
It is another object of the present invention to propose a kind of data mover system.
In view of this, the present invention proposes a kind of data migration method, including:Migrated to target system by source system data Before system, check that rule carries out validity checking to source system data according to default, source system data is entered according to inspection result Row treatment;And source system data structure is carried out into control configuration with goal systems data structure according to configuration mode is compareed;When Source system data is by after validity checking, source system data being migrated to goal systems;When source system data is not by legal Property check when, source system data is repaired and adjusted, until source system data pass through validity checking;Record legitimacy inspection Look into, compare the log information produced in configuration and transition process.
Data migration method of the invention, moves by by each traffic table or business paper in origin system Before moving to goal systems, validity checking is first carried out, only check that the data for passing through can just carry out follow-up Data Migration, such as Fruit inspection does not pass through, and source system data is repaired and adjusted, and rechecking is then carried out again, until passing through validity checking Untill, can to greatest extent find and avoid risk, so that target system after improving the stability of data migration process and migrating The correctness of data of uniting;Compareed according to control configuration mode by by source system data structure and goal systems data structure Configuration, realizes flexibly control configuration data structure, supports the Data Migration between various heterogeneous systems, supports customization development number According to structural transfer, so as to reduce the development cost of data migration process, autgmentability, stability and the safety of migratory system are improved Property;By recording the log information produced in validity checking, control configuration and data migration process, can in time check and move The problem run into during shifting, analyzes illegal data and is repaired and adjustment so as to follow-up, and then the data after reparation are moved again Move, be prevented effectively from Data Migration interruption, goal systems data after the further stability for improving data migration process and migration Correctness.
In addition, according to the above-mentioned data migration method of the present invention, can also have following additional technical characteristic:
In the above-mentioned technical solutions, it is preferable that when source system data does not pass through validity checking, source system data is entered Row is repaired and adjustment is specifically included:Generate illegal data report;Fault-tolerant checking is carried out to source system data;Work as source system data During by fault-tolerant checking, source system data is migrated to goal systems;When source system data is not by fault-tolerant checking, analysis is not Valid data is reported, source system data is repaired and adjusted according to analysis result, until source system data passes through legitimacy Check;Delete illegal data report.
In the technical scheme, if source system data is not over validity checking, illegal data report is generated, Illegal data with Update log information, and are carried out fault-tolerant checking by insertion backstage log sheet, when illegal data are by fault-tolerant After checking, by the Data Migration after checking to goal systems, the stability that system is continuously run is greatly strengthen;When illegal number During according to not over fault-tolerant checking, by analyzing illegal data report, the illegal reason of data is found out, with to illegal number According to being repaired and being adjusted, rechecking is then carried out again, untill inspection passes through, and the illegal data in synchronization removal backstage Log information, can to greatest extent find and avoid risk, so that mesh after improving the stability of data migration process and migrating The correctness of mark system data.
In the above-mentioned technical solutions, it is preferable that when source system data is by after validity checking, will be by validity checking Source system data migrate to goal systems and specifically include:Judge whether to need Extension migratory rule;If the determination result is YES, insert Enter new migration rules, replace original migration rules or increase rule newly on the basis of original;Otherwise selection migration model and right The migration pattern answered;Source system data is carried out into data conversion treatment according to default transformation rule;After data conversion treatment Data are saved in goal systems.
In the technical scheme, Extension migratory rule be in order to be directed to the personalisation process of customization development data structure, When client carries out customization development, it is possible to special assignment treatment logic can be done to the field of customization development, is led to Cross and insert new migration rules, replace original migration rules or increase rule newly on the basis of original, effectively solve client and melt The problem of hair, realizes that minimum cost supports the data structure migration of customer personalized exploitation, can fast and effectively support isomery Data Migration between system, while can also be used as Data Migration or data transfer tool between isomorphism system;If client The personalisation process of customization development data structure need not be carried out, then by selecting suitable migration model and corresponding moving Shifting mode, is migrated to source system data, and the data of origin system are processed as meeting goal systems by default transformation rule The data of rule, the data after data conversion treatment are saved in the corresponding storehouse table of goal systems.
In the above-mentioned technical solutions, it is preferable that reconciliation treatment is carried out to goal systems data and source system data.
In the technical scheme, after the migration of completion source system data to goal systems data, by goal systems Data carry out reconciliation treatment with source system data, can more quickly generate data verification report, free by manpower checking Labor problem and operating efficiency problem, there is provided more comprehensive data verification report, shortening project is reached the standard grade time of switching.
In the above-mentioned technical solutions, it is preferable that goal systems data and source system data are carried out with the specific bag of reconciliation treatment Include:Selection needs the document and default reconciliation dimension or self-defined reconciliation dimension of reconciliation;Document reconciliation is performed, data verification is generated Report.
In the technical scheme, document reconciliation is carried out by selecting document and reconciliation dimension, generation data verification report is The reconciliation dimension of preset some standards of system, if client has individual demand, can be with oneself configuration reconciliation dimension, it is right then to perform Account, generation checking report, flexible configuration reconciliation rule meets the demand that different clients are reported data verification of correctness.
In the above-mentioned technical solutions, it is preferable that default to check that rule has scalability.
In the technical scheme, according to different systems, the default check gauge of migration demand dynamic expansion of different clients Then, do not influence standard check rule on the premise of flexible expansion different system different clients personalized demands on examination, in source Find to greatest extent and avoid risk before system data migration, target after improving the stability of data migration process and migrating The correctness of system data.
In the above-mentioned technical solutions, it is preferable that control configuration mode includes following at least any one:Excel configurations are imported Pattern, interfaceization configuration normal control mode.
In the technical scheme, source system data structure can accomplish with goal systems data structure in data migration process Flexible configuration, configuration mode includes that excel configures introduction model and interfaceization configuration normal control mode.By flexible configuration data knot Structure is compareed, and accomplishes to support the Data Migration between various heterogeneous systems, supports the migration of customization development data structure.This area skill Art personnel are it should be understood that control configuration mode includes but is not limited to following at least any one:Excel configurations introduction model, interface Change configuration normal control mode.
In the above-mentioned technical solutions, it is preferable that migration model includes following at least any one:Smooth migration, control migration; Migration pattern includes following at least any one:Static data migration, incremental data migration.
In the technical scheme, smooth migration is exactly as its name suggests that the data of origin system are moved to according to certain rule Goal systems;Control migration refer in goal systems some data, it is necessary to by source system data and goal systems Data do contrast relationship, migration are then performed again, for example:There is an organization name in origin system " A001 ", goal systems has One tissue is " B001 ", and " A001 " tissue is compareed into migration with " B001 " tissue, then the attribute of origin system " A001 " tissue Can compare in moving to goal systems " B001 " tissue.It should be appreciated by those skilled in the art that migration model includes but does not limit In smooth migration with compare migration both of which.
Migration pattern supports that static data migration migrates both of which with incremental data, and static data migration refers to user Data are no longer done and increase, delete, changing the data of operation and migrated;Incremental data migration refers to the source system of the user during migration System is also being continuing with, and also in lasting change, incremental data migration is in order to user is doing system upgrade Data Migration to data During, do not interrupt the current business of user as far as possible, the influence to user is minimized.Those skilled in the art should manage Solution, migration pattern includes but is not limited to static data migration and incremental data migrates two ways.
In any of the above-described technical scheme, it is preferable that log information includes following at least any one or its combination:It is illegal Data, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment migration, incremental data contrast.
In the technical scheme, it should be appreciated by those skilled in the art that log information includes but is not limited to following at least appoint One or its combination:Illegal data, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment are moved Move, incremental data is contrasted.By log information, the problem run into transition process can be in time checked, analyzed illegal Data are repaired and adjustment so as to follow-up, and then the data after reparation are migrated again, are prevented effectively from Data Migration interruption, further The correctness of goal systems data after improving the stability of data migration process and migrating.
The present invention also proposes a kind of data mover system, including:Data validity checking module, for by origin system number According to migrating to before goal systems, check that rule carries out validity checking to source system data according to default, according to inspection result Source system data is processed;And data structure control configuration module, for by source system data structure and goal systems Data structure carries out control configuration according to control configuration mode;Data Migration core processing module, when source system data is by closing After method is checked, source system data is migrated to goal systems;Data validity checking module, is additionally operable to work as source system data not During by validity checking, source system data is repaired and adjusted, until source system data passes through validity checking;Daily record Management module, for recording the log information produced in validity checking, control configuration and transition process.
Data mover system of the invention, moves by by each traffic table or business paper in origin system Before moving to goal systems, validity checking is first carried out, only check that the data for passing through can just carry out follow-up Data Migration, such as Fruit inspection does not pass through, and source system data is repaired and adjusted, and rechecking is then carried out again, until passing through validity checking Untill, can to greatest extent find and avoid risk, so that target system after improving the stability of data migration process and migrating The correctness of data of uniting;Compareed according to control configuration mode by by source system data structure and goal systems data structure Configuration, realizes flexibly control configuration data structure, supports the Data Migration between various heterogeneous systems, supports customization development number According to structural transfer, so as to reduce the development cost of data migration process, autgmentability, stability and the safety of migratory system are improved Property;By recording the log information produced in validity checking, control configuration and data migration process, can in time check and move The problem run into during shifting, analyzes illegal data and is repaired and adjustment so as to follow-up, and then the data after reparation are moved again Move, be prevented effectively from Data Migration interruption, goal systems data after the further stability for improving data migration process and migration Correctness.
In addition, according to the above-mentioned data mover system of the present invention, can also have following additional technical characteristic:
In the above-mentioned technical solutions, it is preferable that when source system data does not pass through validity checking, source system data is entered Row is repaired and adjustment is specifically included:Generation unit, for generating illegal data report;Fault-tolerant unit, for origin system number According to carrying out fault-tolerant checking;Data Migration core processing module, is additionally operable to when source system data is by fault-tolerant checking, by origin system Data Migration is to goal systems;Log management module, is additionally operable to, when source system data is not by fault-tolerant checking, analyze illegal Data report;Adjustment unit is repaired, for source system data to be repaired and adjusted according to analysis result, until origin system number According to by validity checking;Log management module, is additionally operable to delete illegal data report.
In the technical scheme, if source system data is not over validity checking, illegal data report is generated, Illegal data with Update log information, and are carried out fault-tolerant checking by insertion backstage log sheet, when illegal data are by fault-tolerant After checking, by the Data Migration after checking to goal systems, the stability that system is continuously run is greatly strengthen;When illegal number During according to not over fault-tolerant checking, by analyzing illegal data report, the illegal reason of data is found out, with to illegal number According to being repaired and being adjusted, rechecking is then carried out again, untill inspection passes through, and the illegal data in synchronization removal backstage Log information, can to greatest extent find and avoid risk, so that mesh after improving the stability of data migration process and migrating The correctness of mark system data.
In the above-mentioned technical solutions, it is preferable that when source system data is by after validity checking, will be by validity checking Source system data migrate to goal systems and specifically include:Judging unit, for judging whether to need Extension migratory rule;Rule Expanding element, for if the determination result is YES, inserting new migration rules, replaces original migration rules or on the basis of original Newly-increased rule;Select unit is otherwise migrated, for selecting migration model and corresponding migration pattern;Date Conversion Unit, is used for Source system data is carried out into data conversion treatment according to default transformation rule;Memory cell, for by after data conversion treatment Data are saved in goal systems.
In the technical scheme, Extension migratory rule be in order to be directed to the personalisation process of customization development data structure, When client carries out customization development, it is possible to special assignment treatment logic can be done to the field of customization development, is led to Cross and insert new migration rules, replace original migration rules or increase rule newly on the basis of original, effectively solve client and melt The problem of hair, realizes that minimum cost supports the data structure migration of customer personalized exploitation, can fast and effectively support isomery Data Migration between system, while can also be used as Data Migration or data transfer tool between isomorphism system;If client The personalisation process of customization development data structure need not be carried out, then by selecting suitable migration model and corresponding moving Shifting mode, is migrated to source system data, and the data of origin system are processed as meeting goal systems by default transformation rule The data of rule, the data after data conversion treatment are saved in the corresponding storehouse table of goal systems.
In the above-mentioned technical solutions, it is preferable that data reconciliation processing module, for goal systems data and origin system number According to carrying out reconciliation treatment.
In the technical scheme, after the migration of completion source system data to goal systems data, by goal systems Data carry out reconciliation treatment with source system data, can more quickly generate data verification report, free by manpower checking Labor problem and operating efficiency problem, there is provided more comprehensive data verification report, shortening project is reached the standard grade time of switching.
In the above-mentioned technical solutions, it is preferable that goal systems data and source system data are carried out with the specific bag of reconciliation treatment Include:Reconciliation select unit, the document and default reconciliation dimension or self-defined reconciliation dimension of reconciliation are needed for selection;Perform list Unit, for performing document reconciliation, and generates data verification report.
In the technical scheme, document reconciliation is carried out by selecting document and reconciliation dimension, generation data verification report is The reconciliation dimension of preset some standards of system, if client has individual demand, can be with oneself configuration reconciliation dimension, it is right then to perform Account, generation checking report, flexible configuration reconciliation rule meets the demand that different clients are reported data verification of correctness.
In the above-mentioned technical solutions, it is preferable that default to check that rule has scalability.
In the technical scheme, according to different systems, the default check gauge of migration demand dynamic expansion of different clients Then, do not influence standard check rule on the premise of flexible expansion different system different clients personalized demands on examination, in source Find to greatest extent and avoid risk before system data migration, target after improving the stability of data migration process and migrating The correctness of system data.
In the above-mentioned technical solutions, it is preferable that control configuration mode includes following at least any one:Excel configurations are imported Pattern, interfaceization configuration normal control mode.
In the technical scheme, source system data structure can accomplish with goal systems data structure in data migration process Flexible configuration, configuration mode includes that excel configures introduction model and interfaceization configuration normal control mode.By flexible configuration data knot Structure is compareed, and accomplishes to support the Data Migration between various heterogeneous systems, supports the migration of customization development data structure.This area skill Art personnel are it should be understood that control configuration mode includes but is not limited to following at least any one:Excel configurations introduction model, interface Change configuration normal control mode.
In the above-mentioned technical solutions, it is preferable that migration model includes following at least any one:Smooth migration, control migration; Migration pattern includes following at least any one:Static data migration, incremental data migration.
In the technical scheme, smooth migration is exactly as its name suggests that the data of origin system are moved to according to certain rule Goal systems;Control migration refer in goal systems some data, it is necessary to by source system data and goal systems Data do contrast relationship, migration are then performed again, for example:There is an organization name in origin system " A001 ", goal systems has One tissue is " B001 ", and " A001 " tissue is compareed into migration with " B001 " tissue, then the attribute of origin system " A001 " tissue Can compare in moving to goal systems " B001 " tissue.It should be appreciated by those skilled in the art that migration model includes but does not limit In smooth migration with compare migration both of which.
Migration pattern supports that static data migration migrates both of which with incremental data, and static data migration refers to user Data are no longer done and increase, delete, changing the data of operation and migrated;Incremental data migration refers to the source system of the user during migration System is also being continuing with, and also in lasting change, incremental data migration is in order to user is doing system upgrade Data Migration to data During, do not interrupt the current business of user as far as possible, the influence to user is minimized.Those skilled in the art should manage Solution, migration pattern includes but is not limited to static data migration and incremental data migrates two ways.
In any of the above-described technical scheme, it is preferable that log information includes following at least any one or its combination:It is illegal Data, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment migration, incremental data contrast.
In the technical scheme, it should be appreciated by those skilled in the art that log information includes but is not limited to following at least appoint One or its combination:Illegal data, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment are moved Move, incremental data is contrasted.By log information, the problem run into transition process can be in time checked, analyzed illegal Data are repaired and adjustment so as to follow-up, and then the data after reparation are migrated again, are prevented effectively from Data Migration interruption, further The correctness of goal systems data after improving the stability of data migration process and migrating.
Additional aspect of the invention and advantage will become obvious in following description section, or by practice of the invention Recognize.
Brief description of the drawings
Of the invention above-mentioned and/or additional aspect and advantage will become from description of the accompanying drawings below to embodiment is combined Substantially and be readily appreciated that, wherein:
Fig. 1 shows the schematic flow sheet of data migration method according to an embodiment of the invention;
Fig. 2 shows the schematic flow sheet of data validity checking according to an embodiment of the invention;
Fig. 3 a show that Data Migration according to an embodiment of the invention processes the schematic flow sheet of logic;
Fig. 3 b show the schematic flow sheet of Data Migration treatment logic according to still a further embodiment;
Fig. 4 shows the schematic flow sheet of data migration method according to still a further embodiment;
Fig. 5 shows the schematic flow sheet of reconciliation treatment according to an embodiment of the invention;
Fig. 6 shows the schematic block diagram of data mover system according to an embodiment of the invention;
Fig. 7 shows the schematic block diagram of data mover system according to still a further embodiment;
Fig. 8 shows the schematic block diagram of Data Migration core processing module according to an embodiment of the invention;
Fig. 9 shows the schematic block diagram of data mover system according to another embodiment of the invention;
Figure 10 shows the schematic block diagram of data reconciliation processing module according to an embodiment of the invention;
Figure 11 shows the configuration diagram of data mover system according to an embodiment of the invention.
Specific embodiment
It is below in conjunction with the accompanying drawings and specific real in order to be more clearly understood that the above objects, features and advantages of the present invention Mode is applied to be further described in detail the present invention.It should be noted that in the case where not conflicting, the implementation of the application Feature in example and embodiment can be mutually combined.
Many details are elaborated in the following description in order to fully understand the present invention, but, the present invention may be used also Implemented with being different from other modes described here using other, therefore, protection scope of the present invention is not by described below Specific embodiment limitation.
As shown in figure 1, the schematic flow sheet of data migration method according to an embodiment of the invention:
Step 102, migrates to before goal systems by source system data, checks rule to source system data according to default Validity checking is carried out, source system data is processed according to inspection result;
Step 104, source system data structure and goal systems data structure are matched somebody with somebody according to compareing configuration mode and carry out control Put;
Step 106, when source system data is by after validity checking, source system data being migrated to goal systems;
Step 108, source system data when source system data does not pass through validity checking, repaired and adjusted, directly Pass through validity checking to source system data;
Step 110, the log information produced in record validity checking, control configuration and transition process.
In this embodiment, migrated to goal systems by by each traffic table or business paper in origin system Before, validity checking is first carried out, only checks that the data for passing through can just carry out follow-up Data Migration, if checked obstructed Cross, source system data is repaired and adjusted, rechecking is then carried out again, untill by validity checking, can Find to greatest extent and avoid risk, thus improve data migration process stability and migration after goal systems data just True property;Control configuration is carried out according to control configuration mode by by source system data structure and goal systems data structure, is realized Flexibly control configuration data structure, supports the Data Migration between various heterogeneous systems, supports that customization development data structure is moved Move, so as to reduce the development cost of data migration process, improve autgmentability, stability and the security of migratory system;By note The log information produced in record validity checking, control configuration and data migration process, in can in time checking transition process The problem for running into, analyzes illegal data and is repaired and adjustment so as to follow-up, and then the data after reparation are migrated again, effectively keeps away Exempt from Data Migration interruption, the correctness of goal systems data after further improving the stability of data migration process and migrating.
As shown in Fig. 2 the schematic flow sheet of data validity checking according to an embodiment of the invention:
Step 202, validity checking is carried out to source system data;When source system data does not pass through validity checking, hold Row step 204;When source system data passes through validity checking, step 208 is performed;
Step 204, generates illegal data report;
Step 206, fault-tolerant checking is carried out to source system data;When source system data is by fault-tolerant checking, step is performed 208;When source system data is not by fault-tolerant checking, step 210 is performed;
Step 208, source system data is migrated to goal systems;
Step 210, analyzes illegal data report, and source system data is repaired and adjusted according to analysis result, weight Multiple step 202, until source system data passes through validity checking
Step 212, the log information produced in record validity checking and transition process;
Step 214, deletes illegal data report.
In this embodiment, if source system data is not over validity checking, illegal data report is generated, is inserted Enter backstage log sheet, with Update log information, and fault-tolerant checking is carried out to illegal data, when illegal data are tested by fault-tolerant After card, by the Data Migration after checking to goal systems, the stability that system is continuously run is greatly strengthen;When illegal data During not over fault-tolerant checking, by analyzing illegal data report, the illegal reason of data is found out, with to illegal data Repaired and adjusted, rechecking is then carried out again, untill inspection passes through, and the illegal data day in synchronization removal backstage Will information, can to greatest extent find and avoid risk, so that target after improving the stability of data migration process and migrating The correctness of system data.Data validity checking demonstration effect such as table 1:
The data validity checking demonstration effect of table 1
As shown in Figure 3 a, Data Migration according to an embodiment of the invention processes the schematic flow sheet of logic:
Step 302, judges whether to need Extension migratory rule;If the determination result is YES, step 304 is performed;Otherwise perform Step 306;
Step 304, inserts new migration rules, replaces original migration rules or increases rule newly on the basis of original;
Step 306, selects migration model and corresponding migration pattern;
Step 308, data conversion treatment is carried out by source system data according to default transformation rule;
Data after data conversion treatment are saved in goal systems by step 310
Step 312, the log information produced in record transition process.
In this embodiment, Extension migratory rule be in order to be directed to the personalisation process of customization development data structure, when When client carries out customization development, it is possible to special assignment treatment logic can be done to the field of customization development, is passed through The new migration rules of insertion, replace original migration rules or increase rule newly on the basis of original, effectively solve customization development Problem, realize that minimum cost supports the data structure migration of customer personalized exploitation, can fast and effectively support isomery system Data Migration between system, while can also be used as Data Migration or data transfer tool between isomorphism system;If client is not Need to carry out the personalisation process of customization development data structure, then by selecting suitable migration model and corresponding migration Mode, is migrated to source system data, and the data of origin system are processed as by default transformation rule to meet goal systems rule Data then, the data after data conversion treatment are saved in the corresponding storehouse table of goal systems.
As shown in Figure 3 b, Data Migration according to still a further embodiment processes the schematic flow sheet of logic:
Step 302, judges whether to need Extension migratory rule;If the determination result is YES, step 304 is performed;Otherwise perform Step 306;
Step 304, inserts new migration rules, replaces original migration rules or increases rule newly on the basis of original;
Step 306, selects migration model and corresponding migration pattern;
Step 308, source data backup;
Step 310, inquires about backup source data;
Step 312, data conversion treatment is carried out by source system data according to default transformation rule;
Data after data conversion treatment are saved in goal systems by step 314
Step 316, the log information produced in record transition process.
In this embodiment, Data Migration treatment logic includes:Selection migration model, migration rules extension, source data are standby Part (choosing is done), inquiry source data, data conversion treatment, preservation database, wherein source data backup, data conversion treatment, preserve The traveling locus of generation all recorded backstage daily record by the process of database, subsequently through analysis log information, process and solve Migration problem.Source data backup be the database table of origin system is first backed up it is a in goal systems database, then from standby Read data in part table to move in goal systems in corresponding storehouse table, this migration model is migrated suitable for static data.When Complete to avoid the need for operating source system data storehouse again after backup source database table, data and the target database table of backup exist Same database, it is convenient to be processed followed by Data Migration error analysis and reconciliation.
As shown in figure 4, the schematic flow sheet of data migration method according to still a further embodiment:
Step 402, migrates to before goal systems by source system data, checks rule to source system data according to default Validity checking is carried out, source system data is processed according to inspection result;
Step 404, source system data structure and goal systems data structure are matched somebody with somebody according to compareing configuration mode and carry out control Put;
Step 406, when source system data is by after validity checking, source system data being migrated to goal systems;
Step 408, source system data when source system data does not pass through validity checking, repaired and adjusted, directly Pass through validity checking to source system data;
Step 410, the log information produced in record validity checking, control configuration and transition process;
Goal systems data and source system data are carried out reconciliation treatment by step 412.
In this embodiment, after the migration of completion source system data to goal systems data, by goal systems number Reconciliation treatment is carried out according to source system data, data verification report can be more quickly generated, freed by manpower checking Labor problem and operating efficiency problem, there is provided more comprehensive data verification report, shortening project are reached the standard grade time of switching.
As shown in figure 5, the schematic flow sheet of reconciliation treatment according to an embodiment of the invention:
Step 502, selection needs the document of reconciliation;
Step 504, the default reconciliation dimension of selection;Or
Step 506, self-defined reconciliation dimension;
Step 508, performs document reconciliation, generation data verification report.
In this embodiment, document reconciliation, generation data verification report, system are carried out by selecting document and reconciliation dimension The reconciliation dimension of preset some standards, if client has individual demand, can be with oneself configuration reconciliation dimension, it is right then to perform Account, generation checking report, flexible configuration reconciliation rule meets the demand that different clients are reported data verification of correctness.
In the above embodiment, it is preferable that default check that rule has scalability.
In this embodiment, according to different systems, the migration demand dynamic expansion of different clients is default to check rule, The personalized demands on examination of flexible expansion different system different clients on the premise of not influenceing standard to check rule, in origin system Found to greatest extent before Data Migration and avoided risk, goal systems after improving the stability of data migration process and migrating The correctness of data.
In the above embodiment, it is preferable that control configuration mode includes following at least any one:Excel configurations import mould Formula, interfaceization configuration normal control mode.
In this embodiment, source system data structure can accomplish spirit with goal systems data structure in data migration process Configuration living, configuration mode includes that excel configures introduction model and interfaceization configuration normal control mode.By flexibly control configuration data Structure, accomplishes to support the Data Migration between various heterogeneous systems, the migration of customization development data structure is supported, to meet difference The demand of customization development.It should be appreciated by those skilled in the art that control configuration mode include but is not limited to it is following at least any :Excel configurations introduction model, interfaceization configuration normal control mode.
Specific embodiment one, the data structure contrast relationship of origin system and goal systems is configured by excel, and configuration is completed Source system data is imported into goal systems via foreground import feature later, is closed according to control in follow-up migration logic Source system data storehouse literary name segment value is moved to the database table field value of corresponding goal systems for system.Excel configurations import mould Formula contrast template such as table 2:
Table 2excel configures introduction model contrast template
Specific embodiment two, foreground interfaceization configuration normal control mode, the left side is goal systems database table structure (data dictionary), The right is origin system database table structure (data dictionary), and the database table structure of origin system and goal systems is all by storehouse table name from source Load out in system library and target database.Target database database table structure is sorted alphabetically and is set forth in interface, source system System correspondence document database table structure is designed to reference pattern and manually enters both of which, is selected to use according to different rule types Reference pattern or pattern is manually entered, if rule type is assignment, using manual typing pattern, if other are regular Type, then by referring to the database table structure for enumerating correspondence document table, then field name corresponding with goal systems is referred to as control, Advantage of this is that avoid manually entering causing mistake.Origin system can more flexibly be realized by the configuration of foreground interface Field between goal systems is compareed, and is capable of the data structure migration of minimum cost support customization development, is reduced data and is moved Customization development cost is moved, autgmentability, stability and the security of migratory system is improved.Interfaceization configuration normal control mode template is such as Table 3:
The interfaceization of table 3 configures normal control mode template
Object table Property Name Target Table Properties Target data type Rule type Source Table Properties title Source Table Properties Source data type Remarks
Buying storage bill master meter ID cbillid char Directly compare Document is identified cbillid char Directly compare
Printing times iprintcount int Directly compare Printing times iprintcount int Conversion int types
Founder creator varchar PK is translated The single mark of system coperatorid varchar Translation
Warehouse cstordocid varchar PK is translated Warehouse is identified cwarehouse varchar Translation
Inventory organization latest edition cstockorgid varchar PK is translated Inventory organization is identified crdcenterid varchar Translation
Document date dbilldate char Control Document date dbilldate char Date conversion is directly compareed
Accounting period caccountperiod char Splicing Accounting month caccountmd varchar Fiscal year+accounting month
Accounting period caccountperiod char Splicing Fiscal year caccountye varchar Fiscal year+accounting month
In the above embodiment, it is preferable that migration model includes following at least any one:Smooth migration, control migration;Move Shifting mode includes following at least any one:Static data migration, incremental data migration.
In this embodiment, smooth migration is exactly as its name suggests that the data of origin system are moved into mesh according to certain rule Mark system;Control migration refer in goal systems some data, it is necessary to by source system data and goal systems number According to contrast relationship is done, migration is then performed again, for example:There is an organization name in origin system " A001 ", goal systems has one Individual tissue is " B001 ", and " A001 " tissue is compareed into migration with " B001 " tissue, then the attribute meeting of origin system " A001 " tissue Control is moved in " B001 " tissue of goal systems.It should be appreciated by those skilled in the art that migration model is included but is not limited to Smooth migration with compare migration both of which.
Migration pattern supports that static data migration migrates both of which with incremental data, and static data migration refers to user Data are no longer done and increase, delete, changing the data of operation and migrated;Incremental data migration refers to the source system of the user during migration System is also being continuing with, and also in lasting change, incremental data migration is in order to user is doing system upgrade Data Migration to data During, do not interrupt the current business of user as far as possible, the influence to user is minimized.Those skilled in the art should manage Solution, migration pattern includes but is not limited to static data migration and incremental data migrates two ways.
In any of the above-described embodiment, it is preferable that log information includes following at least any one or its combination:Illegal number Contrasted according to, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment migration, incremental data.
In this embodiment, it should be appreciated by those skilled in the art that log information include but is not limited to it is following at least any Item or its combination:Illegal data, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment are moved Move, incremental data is contrasted.By log information, the problem run into transition process can be in time checked, analyzed illegal Data are repaired and adjustment so as to follow-up, and then the data after reparation are migrated again, are prevented effectively from Data Migration interruption, further The correctness of goal systems data after improving the stability of data migration process and migrating.
As shown in fig. 6, the schematic block diagram of data mover system according to an embodiment of the invention:
Data validity checking module 602, for being migrated to before goal systems by source system data, according to default inspection Look into rule carries out validity checking to source system data, and source system data is processed according to inspection result;And
Data structure compare configuration module 604, for by source system data structure with goal systems data structure according to right Control configuration is carried out according to configuration mode;
Data Migration core processing module 606, when source system data is by after validity checking, source system data being migrated To goal systems;
Data validity checking module 602, is additionally operable to when source system data does not pass through validity checking, by origin system number According to being repaired and being adjusted, until source system data passes through validity checking;
Log management module 608, for recording the daily record letter produced in validity checking, control configuration and transition process Breath.
In this embodiment, migrated to goal systems by by each traffic table or business paper in origin system Before, validity checking is first carried out, only checks that the data for passing through can just carry out follow-up Data Migration, if checked obstructed Cross, source system data is repaired and adjusted, rechecking is then carried out again, untill by validity checking, can Find to greatest extent and avoid risk, thus improve data migration process stability and migration after goal systems data just True property;Control configuration is carried out according to control configuration mode by by source system data structure and goal systems data structure, is realized Flexibly control configuration data structure, supports the Data Migration between various heterogeneous systems, supports that customization development data structure is moved Move, so as to reduce the development cost of data migration process, improve autgmentability, stability and the security of migratory system;By note The log information produced in record validity checking, control configuration and data migration process, in can in time checking transition process The problem for running into, analyzes illegal data and is repaired and adjustment so as to follow-up, and then the data after reparation are migrated again, effectively keeps away Exempt from Data Migration interruption, the correctness of goal systems data after further improving the stability of data migration process and migrating.
As shown in fig. 7, the schematic block diagram of data mover system according to still a further embodiment:
Data validity checking module 702 includes:Generation unit 7022, for generating illegal data report;
Data structure compare configuration module 704, for by source system data structure with goal systems data structure according to right Control configuration is carried out according to configuration mode;
Data Migration core processing module 706, is additionally operable to when source system data is by fault-tolerant checking, by source system data Migrate to goal systems;
Log management module 708, is additionally operable to, when source system data is not by fault-tolerant checking, analyze illegal datagram Accuse;
Data validity checking module 702 also includes:Repair adjustment unit 7024, for according to analysis result to origin system Data are repaired and adjusted, until source system data passes through validity checking;
Log management module 708, is additionally operable to delete illegal data report.
In this embodiment, if source system data is not over validity checking, illegal data report is generated, is inserted Enter backstage log sheet, with Update log information, and fault-tolerant checking is carried out to illegal data, when illegal data are tested by fault-tolerant After card, by the Data Migration after checking to goal systems, the stability that system is continuously run is greatly strengthen;When illegal data During not over fault-tolerant checking, by analyzing illegal data report, the illegal reason of data is found out, with to illegal data Repaired and adjusted, rechecking is then carried out again, untill inspection passes through, and the illegal data day in synchronization removal backstage Will information, can to greatest extent find and avoid risk, so that target after improving the stability of data migration process and migrating The correctness of system data.
As shown in figure 8, the schematic block diagram of Data Migration core processing module according to an embodiment of the invention:
Data Migration core processing module 800 includes:Judging unit 802, for judging whether to need Extension migratory rule;
Rule Extended unit 804, for if the determination result is YES, inserting new migration rules, replaces original migration rules;
Migration select unit 806, for selecting migration model and corresponding migration pattern;
Date Conversion Unit 808, for source system data to be carried out into data conversion treatment according to default transformation rule;
Memory cell 810, for the data after data conversion treatment to be saved in into goal systems.
In this embodiment, Extension migratory rule be in order to be directed to the personalisation process of customization development data structure, when When client carries out customization development, it is possible to special assignment treatment logic can be done to the field of customization development, is passed through The new migration rules of insertion, replace original migration rules, effectively solve the problems, such as customization development, realize that minimum cost supports visitor The data structure migration of the personalized exploitation in family, can fast and effectively support the Data Migration between heterogeneous system, while also may be used As Data Migration or data transfer tool between isomorphism system;If client need not carry out customization development data structure Personalisation process, then by selecting suitable migration model and corresponding migration pattern, source system data is migrated, The data of origin system are processed as meeting the data of goal systems rule by default transformation rule, after data conversion treatment Data are saved in the corresponding storehouse table of goal systems.
As Fig. 9 shows, the schematic block diagram of data mover system according to another embodiment of the invention:
Data validity checking module 902, for being migrated to before goal systems by source system data, according to default inspection Look into rule carries out validity checking to source system data, and source system data is processed according to inspection result;Wherein, data are closed Method checks that module 902 includes:Generation unit 9022, for generating illegal data report;Adjustment unit 9024 is repaired, is used for Source system data is repaired and adjusted according to analysis result, until source system data passes through validity checking;
Data structure compare configuration module 904, for by source system data structure with goal systems data structure according to right Control configuration is carried out according to configuration mode;
Data Migration core processing module 906, when source system data is by after validity checking, source system data being migrated To goal systems;Wherein, Data Migration core processing module 906 includes:Judging unit 9062, for judging whether to need extension Migration rules;Rule Extended unit 9064, for if the determination result is YES, inserting new migration rules, replaces original migration rule Then;Migration select unit 9066, for selecting migration model and corresponding migration pattern;Date Conversion Unit 9068, for inciting somebody to action Source system data carries out data conversion treatment according to default transformation rule;Memory cell 9070, for by after data conversion treatment Data be saved in goal systems;
Data validity checking module 902, is additionally operable to when source system data does not pass through validity checking, by origin system number According to being repaired and being adjusted, until source system data passes through validity checking;
Log management module 908, for recording the daily record letter produced in validity checking, control configuration and transition process Breath;
Data reconciliation processing module 910, for carrying out reconciliation treatment to goal systems data and source system data.
In this embodiment, after the migration of completion source system data to goal systems data, by goal systems number Reconciliation treatment is carried out according to source system data, data verification report can be more quickly generated, freed by manpower checking Labor problem and operating efficiency problem, there is provided more comprehensive data verification report, shortening project are reached the standard grade time of switching.
As shown in Figure 10, the schematic block diagram of data reconciliation processing module according to an embodiment of the invention:
Data reconciliation processing module 1000 includes:Reconciliation select unit 1002, for selecting to need the document of reconciliation and pre- If reconciliation dimension or self-defined reconciliation dimension;Execution unit 1004, for performing document reconciliation, and generates data verification report.
In this embodiment, document reconciliation, generation data verification report, system are carried out by selecting document and reconciliation dimension The reconciliation dimension of preset some standards, if client has individual demand, can be with oneself configuration reconciliation dimension, it is right then to perform Account, generation checking report, flexible configuration reconciliation rule meets the demand that different clients are reported data verification of correctness.
In the above embodiment, it is preferable that default check that rule has scalability.
In this embodiment, according to different systems, the migration demand dynamic expansion of different clients is default to check rule, The personalized demands on examination of flexible expansion different system different clients on the premise of not influenceing standard to check rule, in origin system Found to greatest extent before Data Migration and avoided risk, goal systems after improving the stability of data migration process and migrating The correctness of data.
In the above embodiment, it is preferable that control configuration mode includes following at least any one:Excel configurations import mould Formula, interfaceization configuration normal control mode.
In this embodiment, source system data structure can accomplish spirit with goal systems data structure in data migration process Configuration living, configuration mode includes that excel configures introduction model and interfaceization configuration normal control mode.By flexible configuration data structure Control, accomplishes to support the Data Migration between various heterogeneous systems, supports the migration of customization development data structure.Art technology Personnel are it should be understood that control configuration mode includes but is not limited to following at least any one:Excel configurations introduction model, interface Configuration normal control mode.
In the above embodiment, it is preferable that migration model includes following at least any one:Smooth migration, control migration;Move Shifting mode includes following at least any one:Static data migration, incremental data migration.
In this embodiment, smooth migration is exactly as its name suggests that the data of origin system are moved into mesh according to certain rule Mark system;Control migration refer in goal systems some data, it is necessary to by source system data and goal systems number According to contrast relationship is done, migration is then performed again, for example:There is an organization name in origin system " A001 ", goal systems has one Individual tissue is " B001 ", and " A001 " tissue is compareed into migration with " B001 " tissue, then the attribute meeting of origin system " A001 " tissue Control is moved in " B001 " tissue of goal systems.It should be appreciated by those skilled in the art that migration model is included but is not limited to Smooth migration with compare migration both of which.
Migration pattern supports that static data migration migrates both of which with incremental data, and static data migration refers to user Data are no longer done and increase, delete, changing the data of operation and migrated;Incremental data migration refers to the source system of the user during migration System is also being continuing with, and also in lasting change, incremental data migration is in order to user is doing system upgrade Data Migration to data During, do not interrupt the current business of user as far as possible, the influence to user is minimized.Those skilled in the art should manage Solution, migration pattern includes but is not limited to static data migration and incremental data migrates two ways.
In any of the above-described embodiment, it is preferable that log information includes following at least any one or its combination:Illegal number Contrasted according to, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment migration, incremental data.
In this embodiment, it should be appreciated by those skilled in the art that log information include but is not limited to it is following at least any Item or its combination:Illegal data, migrating data amount statistics, the time-consuming statistics of migration, the treatment of illegal data exception, increment are moved Move, incremental data is contrasted.By log information, the problem run into transition process can be in time checked, analyzed illegal Data are repaired and adjustment so as to follow-up, and then the data after reparation are migrated again, are prevented effectively from Data Migration interruption, further The correctness of goal systems data after improving the stability of data migration process and migrating.
As shown in figure 11, the configuration diagram of data mover system according to an embodiment of the invention:Data Migration System 1100 is included comprising 5 big modules:Data validity checking module 1102, data conjunction was carried out to origin system DB before migration Method checks, analyzes and avoid risk before system data migration is done as far as possible;Data structure compares configuration module 1104, makes source System DB structures can accomplish flexible configuration with goal systems DB structures, by flexible configuration data structure, realize supporting various Data Migration between heterogeneous system, supports the migration of customization development data structure;Data Migration core processing module 1106, leads to Dynamic expansion Data Migration rule is crossed, the Data Migration between heterogeneous system can be fast and effectively supported, and effectively solves visitor Family melts the problem of hair, while can also be used as Data Migration or data transfer tool between isomorphism system;Log management Module 1108, by recording the log information produced in validity checking, control configuration and data migration process, can be timely The problem that runs into examination transition process, analyzes illegal data and repairs and adjust so as to follow-up, and then by the data after reparation Again migrate, be prevented effectively from Data Migration interruption, goal systems after further improving the stability of data migration process and migrating The correctness of data;Data reconciliation processing module 1110, can be with flexible configuration reconciliation rule, to meet different clients to data just The demand of true property checking report, performs the data reconciliation of origin system DB and goal systems DB, generation after Data Migration completion Data verification report, frees the labour by manpower checking and efficiency, there is provided more comprehensive data verification report, contracting Short project is reached the standard grade time of switching.
Data mover system 1100, logic is migrated using configurable mode processing data, and Data Migration is worked becomes It is more flexible, easy-to-use, heterogeneous system Data Migration is become simpler, efficiently;Meanwhile, the Data Migration system of present invention design System can also more flexibly support the Data Migration between isomorphism system, it is also possible to support the Data Migration of same system to not With in the database of version.
The preferred embodiments of the present invention are the foregoing is only, is not intended to limit the invention, for the skill of this area For art personnel, the present invention can have various modifications and variations.It is all within the spirit and principles in the present invention, made any repair Change, equivalent, improvement etc., should be included within the scope of the present invention.

Claims (18)

1. a kind of data migration method, it is characterised in that including:
Migrated to before goal systems by source system data, the source system data is carried out according to default inspection rule legal Property check, the source system data is processed according to inspection result;And
Source system data structure is carried out into control configuration with goal systems data structure according to configuration mode is compareed;
When the source system data is by after the validity checking, the source system data being migrated to the goal systems;
When the source system data is not by the validity checking, the source system data repaired and adjusted, directly To the source system data by the validity checking;
Record the log information produced in the validity checking, control configuration and the transition process.
2. data migration method according to claim 1, it is characterised in that described when the source system data does not pass through institute When stating validity checking, the source system data is repaired and adjustment is specifically included:
Generate illegal data report;
Fault-tolerant checking is carried out to the source system data;
When the source system data is by the fault-tolerant checking, the source system data is migrated to the goal systems;
When the source system data is not by the fault-tolerant checking, the illegal data report is analyzed, according to analysis result The source system data is repaired and adjusted, until the source system data is by the validity checking;
Delete the illegal data report.
3. data migration method according to claim 1, it is characterised in that described when the source system data is by described After validity checking, specifically included being migrated to the goal systems by the source system data of the validity checking:
Judge whether to need Extension migratory rule;
If the determination result is YES, new migration rules are inserted, original migration rules is replaced or is increased rule newly on the basis of original; Otherwise
Selection migration model and corresponding migration pattern;
The source system data is carried out into data conversion treatment according to default transformation rule;
Data after data conversion treatment are saved in the goal systems.
4. data migration method according to claim 1, it is characterised in that also include:
Reconciliation treatment is carried out to the goal systems data and the source system data.
5. data migration method according to claim 4, it is characterised in that it is described to the goal systems data with it is described Source system data carries out reconciliation treatment and specifically includes:
Selection needs the document and default reconciliation dimension or self-defined reconciliation dimension of reconciliation;
Perform document reconciliation, generation data verification report.
6. data migration method according to claim 1, it is characterised in that
The default inspection rule has scalability.
7. data migration method according to claim 1, it is characterised in that
The control configuration mode includes following at least any one:Excel configurations introduction model, interfaceization configuration normal control mode.
8. data migration method according to claim 3, it is characterised in that
The migration model includes following at least any one:Smooth migration, control migration;
The migration pattern includes following at least any one:Static data migration, incremental data migration.
9. the data migration method according to any one of claim 1 to 8, it is characterised in that
The log information includes following at least any one or its combination:Illegal data, migrating data amount statistics, migration are time-consuming Statistics, the treatment of illegal data exception, increment migration, incremental data contrast.
10. a kind of data mover system, it is characterised in that including:
Data validity checking module, for being migrated to before goal systems by source system data, rule is checked according to default Validity checking is carried out to the source system data, the source system data is processed according to inspection result;And
Data structure compares configuration module, for source system data structure and goal systems data structure to be configured into mould according to compareing Formula carries out control configuration;
Data Migration core processing module, when the source system data is by after the validity checking, by the origin system number According to migrating to the goal systems;
The data validity checking module, is additionally operable to when the source system data is not by the validity checking, by institute State source system data to be repaired and adjusted, until the source system data is by the validity checking;
Log management module, for recording the validity checking, control configuration and generation in the transition process Log information.
11. data mover systems according to claim 10, it is characterised in that described when the source system data does not pass through During the validity checking, the source system data is repaired and adjustment is specifically included:
Generation unit, for generating illegal data report;
Fault-tolerant unit, for carrying out fault-tolerant checking to the source system data;
Data Migration core processing module, is additionally operable to when the source system data is by the fault-tolerant checking, by the source system System Data Migration is to the goal systems;
The log management module, is additionally operable to when the source system data is not by the fault-tolerant checking, does not conform to described in analysis Method data report;
Adjustment unit is repaired, for the source system data to be repaired and adjusted according to analysis result, until the source system System data are by the validity checking;
The log management module, is additionally operable to delete the illegal data report.
12. data mover systems according to claim 10, it is characterised in that described when the source system data passes through institute After stating validity checking, specifically included being migrated to the goal systems by the source system data of the validity checking:
Judging unit, for judging whether to need Extension migratory rule;
Rule Extended unit, for if the determination result is YES, inserting new migration rules, replaces original migration rules or original On the basis of increase newly rule;Otherwise
Migration select unit, for selecting migration model and corresponding migration pattern;
Date Conversion Unit, for the source system data to be carried out into data conversion treatment according to default transformation rule;
Memory cell, for the data after data conversion treatment to be saved in into the goal systems.
13. data mover systems according to claim 10, it is characterised in that also include:
Data reconciliation processing module, for carrying out reconciliation treatment to the goal systems data and the source system data.
14. data mover systems according to claim 13, it is characterised in that described to the goal systems data and institute State source system data carry out reconciliation treatment specifically include:
Reconciliation select unit, the document and default reconciliation dimension or self-defined reconciliation dimension of reconciliation are needed for selection;
Execution unit, for performing document reconciliation, and generates data verification report.
15. data mover systems according to claim 10, it is characterised in that
The default inspection rule has scalability.
16. data mover systems according to claim 10, it is characterised in that
The control configuration mode includes following at least any one:Excel configurations introduction model, interfaceization configuration normal control mode.
17. data mover systems according to claim 12, it is characterised in that
The migration model includes following at least any one:Smooth migration, control migration;
The migration pattern includes following at least any one:Static data migration, incremental data migration.
18. data mover system according to any one of claim 10 to 17, it is characterised in that
The log information includes following at least any one or its combination:Illegal data, migrating data amount statistics, migration are time-consuming Statistics, the treatment of illegal data exception, increment migration, incremental data contrast.
CN201611074474.6A 2016-11-29 2016-11-29 Data migration method and data mover system Pending CN106776876A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN201611074474.6A CN106776876A (en) 2016-11-29 2016-11-29 Data migration method and data mover system

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN201611074474.6A CN106776876A (en) 2016-11-29 2016-11-29 Data migration method and data mover system

Publications (1)

Publication Number Publication Date
CN106776876A true CN106776876A (en) 2017-05-31

Family

ID=58900612

Family Applications (1)

Application Number Title Priority Date Filing Date
CN201611074474.6A Pending CN106776876A (en) 2016-11-29 2016-11-29 Data migration method and data mover system

Country Status (1)

Country Link
CN (1) CN106776876A (en)

Cited By (18)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107506451A (en) * 2017-08-28 2017-12-22 泰康保险集团股份有限公司 abnormal information monitoring method and device for data interaction
CN108647270A (en) * 2018-04-28 2018-10-12 尚谷科技(天津)有限公司 A method of the Data Migration based on fault-tolerant time daily record
CN108959205A (en) * 2018-06-27 2018-12-07 成都泰盟软件有限公司 A kind of method of ERP system export data fast transferring
CN109639468A (en) * 2018-11-29 2019-04-16 威富通科技有限公司 A kind of data mutual-conductance method, public cloud and server
CN109639778A (en) * 2018-11-29 2019-04-16 威富通科技有限公司 A kind of data transmission method, private clound and server
CN109684298A (en) * 2018-09-07 2019-04-26 平安科技(深圳)有限公司 Database migration method, apparatus, equipment and storage medium
CN109905384A (en) * 2019-02-19 2019-06-18 北京深思数盾科技股份有限公司 Data migration method and system
CN110297813A (en) * 2019-05-22 2019-10-01 平安银行股份有限公司 Data migration method, device, computer equipment and storage medium
CN111159150A (en) * 2019-12-19 2020-05-15 北京文安智能技术股份有限公司 Data expansion method and device
CN111597197A (en) * 2020-06-29 2020-08-28 腾讯科技(深圳)有限公司 Data reconciliation method and device between databases, storage medium and electronic equipment
CN112597128A (en) * 2020-12-17 2021-04-02 广东南方通信建设有限公司 Data migration method and device
CN112860666A (en) * 2021-02-08 2021-05-28 土巴兔集团股份有限公司 Data migration method and device, computer equipment and storage medium
CN113064906A (en) * 2021-04-21 2021-07-02 杭州天谷信息科技有限公司 Binlog log data adaptive migration method and system
CN113239019A (en) * 2021-06-03 2021-08-10 锋泾(中国)建材集团有限公司 Data migration method and system between ERP systems, electronic device and medium
CN113409015A (en) * 2021-06-21 2021-09-17 浙江省能源集团有限公司 ERP system and use method of ERP system in double-activity deployment of double data centers
CN113626412A (en) * 2021-08-13 2021-11-09 徐世亮 Database migration method and device, readable storage medium and computer equipment
CN116107993A (en) * 2022-12-26 2023-05-12 北京万里开源软件有限公司 Data migration evaluation method and system in MySQL protocol database
CN116756247A (en) * 2023-08-21 2023-09-15 腾讯科技(深圳)有限公司 Data restoration method, device, computer equipment and storage medium

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101706803A (en) * 2009-11-27 2010-05-12 天津工业大学 Method for migrating production data of weaving workshop
CN103176988A (en) * 2011-12-21 2013-06-26 上海博腾信息科技有限公司 Data migration system based on software-as-a-service (SaaS)
CN104424283A (en) * 2013-08-30 2015-03-18 阿里巴巴集团控股有限公司 Data migration system and data migration method
CN104461516A (en) * 2014-11-21 2015-03-25 北京锐安科技有限公司 Data processing method and device
CN105354314A (en) * 2015-11-10 2016-02-24 中国建设银行股份有限公司 Data migration method and device
CN105740411A (en) * 2016-01-30 2016-07-06 武汉大学 SOA (Service-Oriented Architecture) and WebService based data migration method
CN105739929A (en) * 2016-01-29 2016-07-06 哈尔滨工业大学深圳研究生院 Data center selection method for big data to migrate to cloud
CN105868343A (en) * 2016-03-28 2016-08-17 上海携程商务有限公司 Database migration method and system

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN101706803A (en) * 2009-11-27 2010-05-12 天津工业大学 Method for migrating production data of weaving workshop
CN103176988A (en) * 2011-12-21 2013-06-26 上海博腾信息科技有限公司 Data migration system based on software-as-a-service (SaaS)
CN104424283A (en) * 2013-08-30 2015-03-18 阿里巴巴集团控股有限公司 Data migration system and data migration method
CN104461516A (en) * 2014-11-21 2015-03-25 北京锐安科技有限公司 Data processing method and device
CN105354314A (en) * 2015-11-10 2016-02-24 中国建设银行股份有限公司 Data migration method and device
CN105739929A (en) * 2016-01-29 2016-07-06 哈尔滨工业大学深圳研究生院 Data center selection method for big data to migrate to cloud
CN105740411A (en) * 2016-01-30 2016-07-06 武汉大学 SOA (Service-Oriented Architecture) and WebService based data migration method
CN105868343A (en) * 2016-03-28 2016-08-17 上海携程商务有限公司 Database migration method and system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
朱爱华: ""广电行业台内分布式异构网络化系统的集成方法探讨和实践"", 《中国电影电视技市学会影视技术文集》 *
秦学勇: ""数据仓库中数据集成的研究"", 《农业网络信息》 *

Cited By (24)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107506451B (en) * 2017-08-28 2020-11-03 泰康保险集团股份有限公司 Abnormal information monitoring method and device for data interaction
CN107506451A (en) * 2017-08-28 2017-12-22 泰康保险集团股份有限公司 abnormal information monitoring method and device for data interaction
CN108647270A (en) * 2018-04-28 2018-10-12 尚谷科技(天津)有限公司 A method of the Data Migration based on fault-tolerant time daily record
CN108959205A (en) * 2018-06-27 2018-12-07 成都泰盟软件有限公司 A kind of method of ERP system export data fast transferring
CN109684298A (en) * 2018-09-07 2019-04-26 平安科技(深圳)有限公司 Database migration method, apparatus, equipment and storage medium
CN109684298B (en) * 2018-09-07 2024-02-20 平安科技(深圳)有限公司 Database migration method, device, equipment and storage medium
CN109639468A (en) * 2018-11-29 2019-04-16 威富通科技有限公司 A kind of data mutual-conductance method, public cloud and server
CN109639778A (en) * 2018-11-29 2019-04-16 威富通科技有限公司 A kind of data transmission method, private clound and server
CN109905384A (en) * 2019-02-19 2019-06-18 北京深思数盾科技股份有限公司 Data migration method and system
CN109905384B (en) * 2019-02-19 2020-11-20 北京纬百科技有限公司 Data migration method and system
CN110297813A (en) * 2019-05-22 2019-10-01 平安银行股份有限公司 Data migration method, device, computer equipment and storage medium
CN111159150A (en) * 2019-12-19 2020-05-15 北京文安智能技术股份有限公司 Data expansion method and device
CN111597197A (en) * 2020-06-29 2020-08-28 腾讯科技(深圳)有限公司 Data reconciliation method and device between databases, storage medium and electronic equipment
CN112597128A (en) * 2020-12-17 2021-04-02 广东南方通信建设有限公司 Data migration method and device
CN112860666A (en) * 2021-02-08 2021-05-28 土巴兔集团股份有限公司 Data migration method and device, computer equipment and storage medium
CN113064906A (en) * 2021-04-21 2021-07-02 杭州天谷信息科技有限公司 Binlog log data adaptive migration method and system
CN113239019A (en) * 2021-06-03 2021-08-10 锋泾(中国)建材集团有限公司 Data migration method and system between ERP systems, electronic device and medium
CN113409015A (en) * 2021-06-21 2021-09-17 浙江省能源集团有限公司 ERP system and use method of ERP system in double-activity deployment of double data centers
CN113626412A (en) * 2021-08-13 2021-11-09 徐世亮 Database migration method and device, readable storage medium and computer equipment
CN113626412B (en) * 2021-08-13 2024-09-24 徐世亮 Database migration method and device, readable storage medium and computer equipment
CN116107993A (en) * 2022-12-26 2023-05-12 北京万里开源软件有限公司 Data migration evaluation method and system in MySQL protocol database
CN116107993B (en) * 2022-12-26 2023-08-29 北京万里开源软件有限公司 Data migration evaluation method and system in MySQL protocol database
CN116756247A (en) * 2023-08-21 2023-09-15 腾讯科技(深圳)有限公司 Data restoration method, device, computer equipment and storage medium
CN116756247B (en) * 2023-08-21 2023-11-17 腾讯科技(深圳)有限公司 Data restoration method, device, computer equipment and storage medium

Similar Documents

Publication Publication Date Title
CN106776876A (en) Data migration method and data mover system
US11210456B2 (en) Method relating to preparation of a report
US11144308B2 (en) System and method for predicting defects in a computer program
US20080193008A1 (en) System and Method for Generating Magnetic Ink Character Recognition (MICR) Testing Documents
CN101523413A (en) Automated generation of form definitions from hard-copy forms
CN107220757A (en) A kind of system and method for rule configuration and parsing
CN105426419B (en) Data promotion system and method between heterogeneous system
US20090055341A1 (en) Regulatory Survey Automation System (RSAS)
CN103077019B (en) A kind of graphical parameter batch maintenance method and device
CN101996380A (en) Method, device and system for searching incorrect account
CN104573053B (en) A kind of configuration item template dynamic customization method based on XML
CN110045981A (en) A method of analytical technology is compared based on source code and establishes demand tracking relationship
CN101221564A (en) Universal independent data acquisition method
CN107688593A (en) Script file generation method and device
CN106528088A (en) Method and device for addling control in online form
CN113538626A (en) Workflow generation method and device
CN103399744B (en) A kind of method building business gate platform
CN108133341A (en) A kind of generation method and system of the material alternative condition based on ERP system
CN109426576B (en) Fault-tolerant processing method and fault-tolerant assembly
KR101120989B1 (en) Method and system for generating standardized source etl program, recording medium and data-etl-method using the method
CN106383778A (en) Method for improving test case reuse rate
Wang et al. Using workflow patterns to model and validate service requirements
Platonov et al. Development of a methodology for cost optimization of software testing for the automatically tests generation
CN101262657A (en) Base station single-board upgrade system
CN105741087A (en) Business flow import method and system

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication

Application publication date: 20170531

RJ01 Rejection of invention patent application after publication