CN101706803A - Method for migrating production data of weaving workshop - Google Patents
Method for migrating production data of weaving workshop Download PDFInfo
- Publication number
- CN101706803A CN101706803A CN200910228796A CN200910228796A CN101706803A CN 101706803 A CN101706803 A CN 101706803A CN 200910228796 A CN200910228796 A CN 200910228796A CN 200910228796 A CN200910228796 A CN 200910228796A CN 101706803 A CN101706803 A CN 101706803A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- subsidiary factory
- statement
- factory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Images
Abstract
The invention relates to a method for migrating production data of a weaving workshop, which comprises a central database, a central data sheet structure, a branch factory database and a branch factory data sheet structure and adopts the following steps: (1) system initialization; (2) data connection configuration, namely providing an editing function for data connection; (3) data synchronization, namely finishing the migration of the production data from the branch factory database to the central database; and (4) data recovery, namely, when a database system of a certain branch factory collapses and cannot operate normally due to an accident, utilizing a branch factory identification to extract the production data of a corresponding branch factory from the central database, recovering the production data to the database of the branch factory and ensuring the completeness of the branch factory data. The migrating method of the invention has the characteristics of reconfigurability, easy expansion, support for transmission resuming at break-point, incremental migration, periodic migration, and support for the data transmission between different databases.
Description
Technical field
The present invention relates to the data processing technique of Industry Control, be specially a kind of moving method of weaving the Workshop Production data.
Background technology
In recent years, the textile industry develop rapidly, the proportion of data message management also progressively increases, various production information data volumes also increase thereupon year by year, because geographic position difference, different division datas under the same company all are stored on separately the server, and interdepartmental data circulation has been caused obstacle.Effectively and accurate data can't be from another department that circulates efficiently an of department, and basic configuration and coded data independent existence in a plurality of systems often, basic data maintenance and synchronous difficulty caused.A medium-sized textile enterprise sets up has 3-5 to weave subsidiary factory, and the loom that turns round simultaneously in each subsidiary factory has hundreds of and even thousands of.Weaving supervisory system is stored in the production data that collects on the production line in the special-purpose separately database server of subsidiary factory; The company information center then needs to pass through in addition the production data recording of the whole enterprise of independent database server stores, and this record is used for the data analysis at company information center on the other hand on the one hand as the data backup of subsidiary factory.Exist in the enterprise information system multisystem of data and reuse problem, also therefore need production data to carry out effectively and efficiently data migtation weaving between subsidiary factory and the company information center.
SQL Server 2000 databases of existing Microsoft provide one to be used for the assembly that data importing is derived, i.e. data-switching service (Data Transformation Services is called for short DTS).DTS can realize the data dump between the SQL Server data source, and also can be the data source of other OLE DB, ODBC driver or the data importing in the text in SQL Server.Utilize the instrument DTS Import/Export Wizard under the DTS can carry out importing and exporting of data, its step is as follows:
(1) opens DTS Import/Export Wizard, select data source, data source type, authentication mode, database;
(2) the target database type, database-name, authentication mode etc. of importing are set;
(3) data importing is set and derives details, appointment is to duplicate one or more table/views from data source, still duplicates Query Result;
(4) select data source table or view, the script that uses when the mapping relations that are listed as between data source table and the purpose table when data-switching is set, conversion, and write transformation rule;
(5) be provided with whether preserve DTS bag, the type of bag and the scheduling mode of bag;
(6), fill in title, description, owner's password, the user cipher of bag and the server location and the verification mode of preservation are set if select to preserve bag;
(7) information that imports and exports setting is confirmed, errors excepted, returned several steps and revise;
(8) behind the validation of information, instrument is finished data importing and is derived.
The data importing derivation of prior art or the shortcoming of moving method are: 1. do not support data source to change during data move; 2. the data amount of movement is big, does not support the difference transmission; 3. do not support periodic data migtation; 4 pairs of concrete data list structure dependences are bigger, need human configuration literary name section.
Summary of the invention
At the deficiencies in the prior art, technical matters to be solved by this invention is that a kind of moving method of weaving the Workshop Production data is provided.This moving method can reduce in the data migtation dependence to the data list structure, realize the data sync of subsidiary factory's database to central database, solve the problem of utilizing the centre data database data to recover subsidiary factory's data simultaneously, reinforcement is to the tracking of data transition process, improve the versatility of data migtation, can adapt to the data migtation between the several data storehouse.
The technical scheme that the present invention solve the technical problem is: design a kind of moving method of weaving the Workshop Production data.This moving method comprises central database, centre data list structure, subsidiary factory's database and subsidiary factory's data list structure, and adopts following steps:
(1) system initialization after system start-up, is set up a unique system configuration example, and the reading system configuration file is given name-value pair information assignment wherein in the corresponding attribute of this example, finishes the initialization of systematic parameter;
(2) data connect configuration, and the editting function that data are connected is provided, and connect by selecting corresponding data, enter edit phase, accept the user and edit; After editor submits to, editor's data are carried out verification, check the legitimacy of data, the reliability that test data connects and the data of configuration connect the requirement that whether meets centre data list structure and subsidiary factory's data list structure; Upgrade current system configuration parameter after the verification succeeds, comprise update system profile instance and new configuration file more;
(3) data sync is finished production data from the migration of subsidiary factory's database to central database;
(4) data are recovered, when certain subsidiary factory causes the Database Systems collapse to run well owing to accident, utilize subsidiary factory to identify the production data that extracts corresponding subsidiary factory from central database, return in the database of this subsidiary factory, guarantee the integrality of subsidiary factory's data.
Wherein, described step (3) data sync comprises:
(31) synchronous initiation obtains the data link information from the system configuration example, set up being connected of subsidiary factory's corresponding tables and center corresponding tables, at subsidiary factory's end creation operation record sheet, creates trigger on subsidiary factory's corresponding tables, the data manipulation on the record subsidiary factory corresponding tables;
(32) cycle synchronisation, regularly from described operation sheet, obtain by the data operation information under the described recording trigger, binding operation type, added field are spliced into SQL statement with this operation information, on the centre data table, carry out described SQL statement, realization is reacted to data variation on the centre data table, finishes data synchronization;
Described step (4) data are recovered to comprise:
(41) utilize subsidiary factory the partial data list structure field information and be identified in conjunction with subsidiary factory and set up one on the centre data table and recover view; Described recovery view provides the total data information of recovering usefulness;
(42) recover thread and from described recovery view, extract data, be spliced into the insertion statement; On subsidiary factory's tables of data, carry out described insertion statement, finish the recovery of data;
(43) move on to next bar data, repeating step (42) recovers to finish up to total data.
Compared with prior art, moving method of the present invention has designed step and implementation method and corresponding processing rules such as initialization, connection configuration, data sync, the data of database migration system are recovered, database is adaptive, have configurable, be easy to expansion, support breakpoint transmission, increment migration, periodically migration and support between disparate databases characteristics such as data transmission, guaranteed the high efficiency and the integrality of data migtation.Specifically, the beneficial effect of moving method of the present invention is: in the process of carrying out data migtation,
1. support the difference incremental mode to carry out the periodic data migration;
2. data migration process does not rely on structure of data table and field type;
3. guarantee to connect the correctness of configuration by the matching test rule of list structure;
4. the data transition process is implemented monitoring and record, be convenient to fault diagnosis;
5. write down the migration progress when data are recovered automatically, have the breakpoint transmission ability;
6. by the data migtation between the database adaptation method support disparate databases;
Description of drawings
Fig. 1 is the subsidiary factory of a kind of embodiment of moving method of the present invention and the graph of a relation between the central database list structure; Wherein,
Fig. 1 (a) is subsidiary factory's database production data list structure synoptic diagram;
Fig. 1 (b) is a central database production data list structure synoptic diagram;
Fig. 2 is the operation sheet visioning procedure synoptic diagram of a kind of embodiment of moving method of the present invention;
Fig. 3 is the trigger visioning procedure synoptic diagram of a kind of embodiment of moving method of the present invention;
Fig. 4 is the trigger statement splicing schematic flow sheet of a kind of embodiment of moving method of the present invention;
Fig. 5 is the cycle synchronisation schematic flow sheet of a kind of embodiment of moving method of the present invention.
Embodiment
Below in conjunction with accompanying drawing and embodiment the present invention is described in further detail:
Moving method of the present invention (be called for short moving method, referring to Fig. 1-5) structurally comprises four parts: 1. system initialization, 2. data connect configuration section, 3. data sync part and 4. data recovered part.Behind the system initialization, the user can connect configuration section by data and revise current systematic parameter, after data connection configuration finishes, can optionally partly start corresponding data sync task, start corresponding data recovery tasks by the data recovered part by data sync." disparate databases is adaptive " technology is incorporated in four parts, makes whole moving method be with good expansibility.
The production data storage environment that moving method embodiment of the present invention weaves the workshop at textile enterprise designs.For the ease of statement, below represent the company information central database with Center, represent with Factory and weave subsidiary factory's database.In this specific memory environment, production data leaves among the Factory after being gathered by " loom production data supervisory system ", is integrated among the Center by this data migtation afterwards, has finished the double copies of data at Factory and Center.When production data is stored in Center and Factory, the list structure of tables of data has certain relation, concrete structural relation be (referring to Fig. 1): Center with Factory in list structure in have identical field i (i=1,2......n), these fields are used for preserving production data; All have in the list structure of Center and Factory from increasing field, but concrete numerical value independent variation separately; Tables of data among the Center has increased an added field collection than the tables of data among the Factory, is used for customizing messages such as expression " subsidiary factory's sign ", " production kind ".
1. system initialization
System initialization is meant after system start-up, sets up a unique system configuration example, and the reading system configuration file is given name-value pair information assignment wherein in the corresponding attribute of this example, finishes the initialization of systematic parameter.The purpose of system initialization is to provide current up-to-date system parameter message for other three parts.
2. data connect configuration section
Data connect configuration section provides the editting function that data are connected. connect by selecting corresponding data, enter edit phase, accepting the user edits. and editor submits to the back that editor's data are carried out the school inspection, check the legitimacy of data, the reliability that test data connects and the data of configuration connect the list structure that whether meets Center and Factory and require (referring to Fig. 1), be except that increasing field certainly and the added field collection partly has the difference in centre data list structure and the subsidiary factory's data list structure, all the other each fields title in two list structures is identical, storage class identical or corresponding (at disparate databases time). the school is examined successfully the back and is upgraded current system configuration parameter, comprises update system profile instance and new configuration file more.
Data migtation will be used the parameter information that configuration file provides system initialization, has wherein write down the relevant information that data connect.There are two kinds of attributes configuration file inside, public attribute and privately owned attribute.Public attribute is used to control whole System Operation, and the synchronous task or the recovery tasks that are connected with concrete certain data are irrelevant; Privately owned attribute is to be subordinated to the attribute that certain particular data connects, and being connected with this of task is in close relations, and all task initialization information all obtains from privately owned attribute.In addition, also have a class specific properties in the privately owned attribute, be used for auxiliary added field and handle, whether this generic attribute the value of other attribute occurs depending on.For example addFieldNameY and addFieldValueY occur when addFieldNum＞0.Concrete configuration file structure is listed in table 1:
Table 1 data migtation configuration file structural table
| The property key title | Implication | Explanation |
| ??Sleep | The sync interval time | The common configuration attribute |
| ??connectNum | The current data linking number | The common configuration attribute |
| ??mssqlX.desc | Data connect to be described | Privately owned attribute |
| ??mssqlX.url | The database connection string | Privately owned attribute, subsidiary factory's end |
| ??mssqlX.DBtype | Type of database | Privately owned attribute, subsidiary factory's end |
| ??mssqlX.driver | The database-driven character string | Privately owned attribute, subsidiary factory's end |
| ??mssqlX.user | Database user name | Privately owned attribute, subsidiary factory's end |
| ??mssqlX.password | Database password | Privately owned attribute, subsidiary factory's end |
| ??mssqlX.together.url | The database connection string | Privately owned attribute, center-side |
| ??mssqlX.together.DBType | Type of database | Privately owned attribute, center-side |
| ??mssqlX.together.driver | The database-driven character string | Privately owned attribute, center-side |
| ??mssqlX.together.user | Database user name | Privately owned attribute, center-side |
| ??mssqlX.together.password | Database password | Privately owned attribute, center-side |
| ??mssqlX.sourceTable | Subsidiary factory's end corresponding tables | Privately owned attribute, subsidiary factory's end |
| The property key title | Implication | Explanation |
| ??mssqlX.targetTable | The center-side corresponding tables | Privately owned attribute, center-side |
| ??mssqlX.addFieldNum | The added field number | Privately owned attribute |
| ??mssqlX.addFieldNameY | The added field title | Privately owned attribute |
| ??mssqlX.addFieldValueY | The add-word segment value | Privately owned attribute |
In the table 1, the hyphen that the X representative data connects, Y are represented the added field numbering, and numbering is since 1.Mode with key-value pair in the configuration file is stored.
3. data sync part
The data sync part will be finished production data from the migration of subsidiary factory's database to central database, specifically is divided into two stages, i.e. synchronous initiation stage and cycle synchronisation stage.The synchronous initiation stage, obtain the data link information from the system configuration example, creation operation record sheet in subsidiary factory's database is used to write down the data manipulation that subsidiary factory shows at establishment trigger on subsidiary factory's table.The cycle synchronisation stage, regularly from subsidiary factory's end record sheet, obtain by the data operation information under the recording trigger, binding operation type, added field are spliced into SQL statement with this operation, on the centre data table, carry out this SQL statement, data variation is reacted on the centre data table, finishes the migration of data.
The synchronous initiation stage is wanted creation operation record sheet and operation note trigger.
The operation that operation sheet is used on the data table memory changes, and its data are provided by the operation note trigger.Each field of operation sheet is described in detail as follows:
Field id: major key increases field, the numbering of sign operation note certainly.
Field optype: this field has three kinds of values: 0: represent the delete operation; 1: represent the update operation; 2: represent the insert operation.
Field statement: be used to store data manipulation statement by under the recording trigger.The data manipulation device that is triggered is noted, and is written in this field with half-done SQL statement form.
Operation sheet is created by Automatic Program in the synchronous initiation process, according to the difference of type of database, adopts different database statements to create, operation sheet visioning procedure following (referring to Fig. 2):
(1) reading of data link information;
(2) obtain synchronisation source table name and type of database;
(3) obtain and being connected of synchronisation source tables of data;
(4) the adaptive selection of statement type;
(5) SQL statement and source table name splicing;
(6) carry out SQL statement at source database and create record sheet.
The operation note trigger is used for writing down automatically the operation to subsidiary factory's tables of data.To inquire about the field information that obtains subsidiary factory's tables of data when creating trigger, after can writing the field name and type extraction of field in this tables of data, be spliced in the SQL statement of creating trigger trigger visioning procedure following (referring to Fig. 3):
(1) reading of data link information;
(2) obtain and being connected of synchronisation source tables of data;
(3) obtain source literary name segment information;
(4) type of database coupling;
(5) update/delete/insert trigger statement splicing;
(6) at the splicing of destination data storehouse execution synchronously statement;
(7) trigger is set up successfully prompting.
The record format of being created out by trigger is:
A. deletion action record: " the synchronous purpose table name of delete from where major key name=Major key [and major key name 2=Major key ...] ", wherein the number of major key is different because of table.
B. upgrade operation note: " update master meter name set field name=field value where major key name=Major key [and major key name 2=Major key ..] ", the field number of renewal is different because of operation.
C. insert operation note: " field value [, field value 2...] ", number is the Field Count of this table.
The statement splicing following (referring to Fig. 4) in (5) step in the trigger constructive process:
(1) selects adaptation rule;
(2) splice with source table table name;
(3) obtain next field information;
(4) judge whether current field is to write field, be then to carry out (5), otherwise carry out (3);
(5) with SQL statement and the splicing of current field;
(6) judged whether next field, be then to carry out (3), otherwise carry out (7);
(7) with the splicing of operation note table name.
In the whole circulation process, increase these two useless fields of field and calculated column field certainly and abandoned automatically, remaining field all is fields of the effective production data of carrying.
The establishment of synchronous initiation stage complete operation record sheet and operation note trigger enters the cycle synchronisation stage afterwards.The flow process in cycle synchronisation stage specifies following (referring to Fig. 5):
(1) start a synchronizing thread, this thread serves as a mark with the data hyphen;
(2) call the flip-flop states administration module, three triggers of synchronisation source table are all enabled;
(3) obtain all records in the operation sheet of subsidiary factory's tables of data correspondence, pointed article one record.As no record, then change (9);
(4) according to the type of database at data connection two ends, call the function that is used for Data Format Transform the value value part of record is carried out format conversion, make new form be suitable for the destination data storehouse; Form is convertData (source format numbering, purpose form numbering, an original character string), and rreturn value is the purpose character string through format conversion;
(5) according to optype field decision operation type, if deletion and upgrade operation then directly and added field partly splice and get final product; If insert operation, the write field of inquiry subsidiary factory table is spliced with the character string among these fields and the statement, then carries out added field again and handles.
Insert statement at last and will be spliced into following form:
" insert into purpose table table name (and can write field [, added field ...]) values (the statement character string [, the add-word segment value ...]) ";
(6) obtain the connection of central database, in this connection, carry out above-mentioned SQL statement of piecing together;
(7) run succeeded, corresponding record in the deletion record table then, and on the synchronous operation counter of corresponding types, add one;
(8) if carry out failure, then doing unusually of generation caught processing, carry out log record.If special unusual, task can't be proceeded, and changes (10), if generally unusual, then next bar record of pointed changes (4);
(9) one subsynchronous execution finish, and log record successfully carries out the value of each synchronous synchronous operation counter, then with counter O reset, and thread dormancy a period of time, wait pending synchronously next time, return (3);
(10) synchronizing thread stops, and synchronous task finishes.
With the data sync that this mode is carried out, the data of transmission only are the production datas that changes at subsidiary factory and center, have improved transport efficiency.
4. data recovered part
Because the unexpected data system that causes collapses when can't run well, utilize subsidiary factory to identify the production data that extracts corresponding subsidiary factory from central database when certain subsidiary factory, return in the database of this subsidiary factory, guarantee subsidiary factory's data integrity.Concrete grammar is to utilize the partial data list structure field information of subsidiary factory and be identified in conjunction with subsidiary factory to set up one on the centre data table and recover view, this view provides the total data information of recovering usefulness, recover thread and from this view, extract data one by one, be spliced into the insertion statement, on subsidiary factory's tables of data, carry out this and insert statement, finish the recovery of data, so carry out next bar, recover to finish up to total data.
Data sync is to carry out data migtation from subsidiary factory's database to central database; It is then just the opposite that data are recovered, and is to carry out data migtation from central database to subsidiary factory's database, thus data recover to carry out data can shared identical configuration item when connecting configuration, to guarantee compatibility of goals.So both saved the time of configuration, guaranteed again that same data connect the uniqueness that goes up data flow, avoided the unreasonable data collision that causes owing to the task setting.For system configuration, adopt unique profile instance to realize, be convenient to the data migtation task to the data link information obtain and to the renewal of link information.Data recovery configuring part idiographic flow is as follows:
(1) connect selection of configuration: from the system configuration example, obtain all data connection names,, select to edit a certain data by the user and connect as the object of selecting editor, newly-built or delete data and connect;
(2) configuration information reads: the details that read the data connection of choosing from the system configuration example, comprise driving name, connection string, username and password with the database join dependency, and information such as the source table name of data migtation, purpose table name and added field, and in editing interface, show these information;
(3) edit configuration information: the information to edit page is edited, change legacy data, perhaps additions and deletions added field etc.;
(4) submit edit file to.If the deleted data attended operation then directly forwards (6) to;
(5) validity checking: before configuration information is submitted to, configuration information is carried out the feasibility inspection of check of data legitimacy and configuration information.Finish the data layout inspection earlier, comprise that whether correct the input content can not be etc. for empty, input format, then do and connect test according to configuration information, if connecting, data set up successfully, proceed the test of service logic feasibility, promptly test this configuration and whether satisfy the basic demand of data migtation, promptly the write field in the structure of subsidiary factory's table all is included in the list structure of center table.Validity checking is not passed through, and changes (3) and updates;
(6) configuration parameter upgrades: about the information of this connection, increase, delete or revise in the update system profile instance, the parameter information in the example after will upgrading simultaneously is rewritten in the configuration file, guarantees the consistance of configuration parameter.
Described view is meant and formally carrying out before data recover, and the write field that adopts subsidiary factory's tables of data is as the basis, sets up the recovery view of this subsidiary factory according to subsidiary factory number from the centre data table, originates as the data data recovered.By adaptation data storehouse type, select different views to create statement and create.
Described data are provided with 3 variablees, recoverNum, totalNum and finish in recovering.RecoverNum represents the record count that current success recovers, and totalNum represents record counts all in the view, and finish represents whether this recovery tasks finishes.Execution in step is as follows:
(1) starts data and recover thread, be connected to mark with data;
(2) data recovery tasks initialization finishes the data sync task that these data connect, and forces to take these data and connects;
(3) close the trigger that subsidiary factory shows, prevent its recording operation;
(4) query log files is recovered breakpoint if data connect to go up to exist, and then reads breakpoint data assignment and gives the recoverNum variable; If there is no recover breakpoint, then recoverNum is changed to 0;
(5) read all data recording in the view, and vernier is jumped to breakpoint location, promptly vernier points to recoverNum+1 bar record;
(6) statement is inserted in splicing: read the list structure of subsidiary factory's corresponding tables, can write the synthetic parenthesis sentence first half of field groups, extract data from Data View, the value of each bar record is inserted the statement latter half, finish splicing;
(7) spliced insertion statement is carried out on subsidiary factory's table, whenever run succeeded one, recoverNum is from increasing one.If take place unusually then go to (8), continue to carry out otherwise return (6), records all in Data View were all carried out;
(8) for doing following processing unusually: do log record earlier, the time that recording exceptional takes place, abnormal information, unusual program description etc.; Preserve current recoverNum value in the breakpoint file;
(9) trigger on the startup subsidiary factory corresponding tables recovers the monitoring to subsidiary factory's table;
(10) finish is set to true, and end data is recovered thread.
For the ease of checking current recovery progress, utilize the number percent of recoverNum and totalNum to be shown to the user as the progress of recovering, if thread normal termination, number percent should be increased to 100%, and the finish parameter is true, and is unusual when occurring, data are recovered to finish, finish is true, but number percent is not 100%, can observe the situation that data are recovered equally intuitively.Utilize recoverNum at this, add 1 after whenever finishing a record, realized the all-the-way tracking that data are recovered effectively.
5. the adaptation method of disparate databases
In order to realize the support to the several data storehouse in data migration method, the present invention has further designed the adaptation method of isomery or disparate databases, is used to support the data migtation between the disparate databases.This adaptation method comprises the adaptation method of driver, the adaptation method of SQL statement, three parts such as adaptation method of data type conversion.Adaptation procedure has adopted four variable: sourceTable, targetTable, sourceType and targetType.Wherein sourceTable represents subsidiary factory's tables of data; TargetTable represents the centre data table; SourceType represents subsidiary factory's type of database; TargetType represents the central database type.The value of variable is 0,1,2 and 3, and implication is respectively: 0 represents SQL Server database, and 1 represents oracle database, and 2 represent sybase database, and 3 represent the DB2 database.The value of variable can be finished the numbering of other data base management system (DBMS) and adaptive by expanded definition, has good adaptability.
Described driver adaptation method is meant to be held for subsidiary factory in the configuration file structure that data connect and center-side has disposed a database device string attribute respectively.Before data connected foundation, the driver character string that connects two ends according to data was carried out the search and the loading of database-driven.
Described SQL statement adaptation method is meant that this part is applicable to the functional based method that produces insertion, upgrades, deletes the splicing statement, in these functional based methods, provide type as an extra suction parameter, the type in the destination data storehouse that is used to represent that spliced SQL statement is suitable for. provide following the statement law that should use in the various types of data storehouse that relates in functional based method inside, to judge according to the type parameter that provides then, select the rule of correspondence database type. the adaptation method of concrete SQL statement is as follows:
switch(type)
{ case MS-SQL type: sql=sql0; Break; // be applicable to the statement of SQL Server
Case ORACLE type: sql=sql1; Break; // be applicable to the statement of Oracle
Case SYBASE type: sql=sql2; Break; // be applicable to the statement of SYBASE
Case DB2 type: sql=sql3; Break; // be applicable to the statement of DB2
……}。
The adaptation method of described data type conversion is meant for the migration between the data of different types storehouse, need partly carry out format conversion to the character string of the presentation data value in the splicing statement before the splicing statement is carried out, and makes it the requirement of adaptation to end database.A special format conversion method is provided for this reason, and form is convertData (source database type, destination data storehouse type, an original character string), and rreturn value is the purpose character string through format conversion.
This adaptation method was called before the statement splicing of data sync, was called after the statement splicing that data are recovered.The adaptive concrete grammar of data type conversion is as follows:
(1) " the character string part behind the value (", first value value of pointed in the query statement;
(2) inquire about the field name of this value correspondence;
(3) this field must be to write field, obtains the SQL types value of this field in source end and destination database respectively, its rreturn value numeral;
(4) with two SQL types value preceding two suction parameters as convertData ();
(5) the value value is called convertData () method as the 3rd suction parameter, obtain rreturn value newValue;
(6) replace current value value with newValue, finish the type conversion of a field;
(7) pointer is skipped a comma and is pointed to next value value, returns (2), ") in running into statement " finish.
The present invention does not address part and is applicable to prior art.
Moving method of the present invention adopts cross-platform technological means, designed a kind of effective implementation method of data migtation, solved and weaved in the enterprise information system multisystem of data and reuse problem, greatly reduced in the data migration process degree of dependence to database table structure, strengthened tracking to the data transition process, improve the dirigibility and the security of data migtation, thereby solved configurable, easy expansion, be easy to integrated data migtation problem, realized that textile industry weaves that production data is correct between each subsidiary factory and enterprise center, effective mobility.Though the embodiment of moving method of the present invention at be to weave the information center of enterprise and effective transmission problem of subsidiary factory's database, it is equally applicable to other enterprise and establishments of identical or similar demand.
Claims (2)
1. moving method of weaving the Workshop Production data, this moving method comprises central database, centre data list structure, subsidiary factory's database and subsidiary factory's data list structure, and adopts following steps:
(1) system initialization after system start-up, is set up a unique system configuration example, and the reading system configuration file is given name-value pair information assignment wherein in the corresponding attribute of this example, finishes the initialization of systematic parameter;
(2) data connect configuration, and the editting function that data are connected is provided, and connect by selecting corresponding data, enter edit phase, accept the user and edit; After editor submits to, editor's data are carried out verification, check the legitimacy of data, the reliability that test data connects and the data of configuration connect the requirement that whether meets centre data list structure and subsidiary factory's data list structure; Upgrade current system configuration parameter after the verification succeeds, comprise update system profile instance and new configuration file more;
(3) data sync is finished production data from the migration of subsidiary factory's database to central database;
(4) data are recovered, when certain subsidiary factory causes the Database Systems collapse to run well owing to accident, utilize subsidiary factory to identify the production data that extracts corresponding subsidiary factory from central database, return in the database of this subsidiary factory, guarantee the integrality of subsidiary factory's data;
Wherein, described step (3) data sync comprises:
(31) synchronous initiation obtains the data link information from the system configuration example, set up being connected of subsidiary factory's corresponding tables and center corresponding tables, at subsidiary factory's end creation operation record sheet, creates trigger on subsidiary factory's corresponding tables, the data manipulation on the record subsidiary factory corresponding tables;
(32) cycle synchronisation, regularly from described operation sheet, obtain by the data operation information under the described recording trigger, binding operation type, added field are spliced into SQL statement with this operation information, on the centre data table, carry out described SQL statement, realization is reacted to data variation on the centre data table, finishes data synchronization;
Described step (4) data are recovered to comprise:
(41) utilize subsidiary factory the partial data list structure field information and be identified in conjunction with subsidiary factory and set up one on the centre data table and recover view; Described recovery view provides the total data information of recovering usefulness;
(42) recover thread and from described recovery view, extract data, be spliced into the insertion statement; On subsidiary factory's tables of data, carry out described insertion statement, finish the recovery of data;
(43) move on to next bar data, repeating step (42) recovers to finish up to total data.
2. the moving method of weaving the Workshop Production data according to claim 1, it is characterized in that described moving method also comprises the adaptation method of disparate databases, this adaptation method comprises the adaptation method of driver, the adaptation method of SQL statement and the adaptation method of data type conversion; Described driver adaptation method is meant to be held for subsidiary factory in the configuration file structure that data connect and center-side has disposed a database device string attribute respectively, before data connected foundation, the driver character string that connects two ends according to data was carried out the search and the loading of database-driven;
Described SQL statement adaptation method is meant that this part is applicable to the functional based method that produces insertion, upgrades, deletes the splicing statement, in these functional based methods, provide type as an extra suction parameter, the type in the destination data storehouse that is used to represent that spliced SQL statement is suitable for, provide following the statement law that should use in the various types of data storehouse that relates in functional based method inside, judge according to the type parameter that provides then, select the rule of correspondence database type;
The adaptation method of described data type conversion is meant for the migration between the data of different types storehouse, need before carrying out, the splicing statement partly carry out format conversion to the character string of the presentation data value in the splicing statement, the form of format conversion method is convertData, and rreturn value is the purpose character string through format conversion;
This adaptation method was called before the statement splicing of data sync, was called after the statement splicing that data are recovered.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102287965A CN101706803B (en) | 2009-11-27 | 2009-11-27 | Method for migrating production data of weaving workshop |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102287965A CN101706803B (en) | 2009-11-27 | 2009-11-27 | Method for migrating production data of weaving workshop |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101706803A true CN101706803A (en) | 2010-05-12 |
| CN101706803B CN101706803B (en) | 2012-07-04 |
Family
ID=42377028
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009102287965A Expired - Fee Related CN101706803B (en) | 2009-11-27 | 2009-11-27 | Method for migrating production data of weaving workshop |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101706803B (en) |
Cited By (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102393860A (en) * | 2011-12-06 | 2012-03-28 | 方正国际软件有限公司 | Database data migration system |
| CN102495906A (en) * | 2011-12-23 | 2012-06-13 | 天津神舟通用数据技术有限公司 | Incremental data migration method capable of realizing breakpoint transmission |
| CN102609337A (en) * | 2012-01-19 | 2012-07-25 | 北京神州数码思特奇信息技术股份有限公司 | Rapid data recovery method for memory database |
| CN102761614A (en) * | 2012-06-29 | 2012-10-31 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing breakpoint resume of network data transmission |
| CN103049550A (en) * | 2012-12-27 | 2013-04-17 | 北京思特奇信息技术股份有限公司 | Method and system for importing common files to database |
| CN103176989A (en) * | 2011-12-21 | 2013-06-26 | 中国银联股份有限公司 | Method and system used for comparing database table levels and based on data dictionary and variable rules |
| CN103491091A (en) * | 2013-09-24 | 2014-01-01 | 长沙裕邦软件开发有限公司 | Method and system for monitoring user operation based on data bank |
| CN104123367A (en) * | 2014-07-24 | 2014-10-29 | 中国农业银行股份有限公司 | Data migration method and system from non-product-factory pattern to product factory pattern |
| CN104484429A (en) * | 2014-12-18 | 2015-04-01 | 山东中创软件工程股份有限公司 | Data contrast analysis method and data contrast analysis system |
| CN105373621A (en) * | 2015-12-07 | 2016-03-02 | 高新兴科技集团股份有限公司 | Rapid database-system-across data increment migration method |
| CN105574141A (en) * | 2015-12-15 | 2016-05-11 | 杭州朗和科技有限公司 | Method and device for migrating data of database |
| CN106776876A (en) * | 2016-11-29 | 2017-05-31 | 用友网络科技股份有限公司 | Data migration method and data mover system |
| CN106815317A (en) * | 2016-12-23 | 2017-06-09 | 中国银联股份有限公司 | A kind of data copy method and system based on disparate databases |
| CN107357885A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Method for writing data and device, electronic equipment, computer-readable storage medium |
| CN107480224A (en) * | 2017-09-11 | 2017-12-15 | 爱普（福建）科技有限公司 | The configuration data of control station realizes the device of data sharing with third party database |
| CN107644122A (en) * | 2017-08-29 | 2018-01-30 | 深圳市兴森快捷电路科技股份有限公司 | A kind of ODB++ file modifications method, apparatus and readable storage medium storing program for executing |
| CN108255984A (en) * | 2017-12-28 | 2018-07-06 | 浪潮通用软件有限公司 | A kind of data analysis method and device |
| CN108897773A (en) * | 2018-05-31 | 2018-11-27 | 湖南格凡安信科技有限公司 | A kind of transparent online database anonymization data parallel moving method |
| CN109284281A (en) * | 2018-09-10 | 2019-01-29 | 北京百度网讯科技有限公司 | Method and apparatus for migrating data |
| CN109408588A (en) * | 2018-09-07 | 2019-03-01 | 武汉达梦数据库有限公司 | A kind of method of breakpoint transmission when database synchronization initialization loading |
| WO2020015151A1 (en) * | 2018-07-18 | 2020-01-23 | 平安科技（深圳）有限公司 | Parameter insertion method and apparatus for database table, terminal device, and storage medium |
| CN111125085A (en) * | 2018-10-30 | 2020-05-08 | 千寻位置网络有限公司 | Python-based database index optimization method, device and system |
| CN111309790A (en) * | 2018-11-27 | 2020-06-19 | 千寻位置网络有限公司 | Automatic ETL configuration method and device based on maxcomputer database and control system |
| CN112434059A (en) * | 2021-01-26 | 2021-03-02 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN112988701A (en) * | 2019-12-13 | 2021-06-18 | 中国电信股份有限公司 | Database management method, device, system and storage medium |
| CN113377763A (en) * | 2020-03-10 | 2021-09-10 | 阿里巴巴集团控股有限公司 | Database table switching method and device, electronic equipment and computer storage medium |
| CN113704228A (en) * | 2021-08-31 | 2021-11-26 | 北京达佳互联信息技术有限公司 | Method and device for field self-increment, electronic equipment and storage medium |
| CN113918996A (en) * | 2021-11-24 | 2022-01-11 | 企查查科技有限公司 | Distributed data processing method, device, computer equipment and storage medium |
Family Cites Families (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1763745A (en) * | 2005-09-06 | 2006-04-26 | 阎海强 | Structure oriented data migration method and system |
| CN101266606B (en) * | 2007-03-15 | 2012-06-20 | 阿里巴巴集团控股有限公司 | On-line data migration method based on Oracle database |
-
2009
- 2009-11-27 CN CN2009102287965A patent/CN101706803B/en not_active Expired - Fee Related
Cited By (36)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102393860A (en) * | 2011-12-06 | 2012-03-28 | 方正国际软件有限公司 | Database data migration system |
| CN103176989A (en) * | 2011-12-21 | 2013-06-26 | 中国银联股份有限公司 | Method and system used for comparing database table levels and based on data dictionary and variable rules |
| CN103176989B (en) * | 2011-12-21 | 2016-09-28 | 中国银联股份有限公司 | Data base's table level comparative approach based on data dictionary and variable rules and system |
| CN102495906A (en) * | 2011-12-23 | 2012-06-13 | 天津神舟通用数据技术有限公司 | Incremental data migration method capable of realizing breakpoint transmission |
| CN102609337A (en) * | 2012-01-19 | 2012-07-25 | 北京神州数码思特奇信息技术股份有限公司 | Rapid data recovery method for memory database |
| CN102761614A (en) * | 2012-06-29 | 2012-10-31 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing breakpoint resume of network data transmission |
| CN103049550A (en) * | 2012-12-27 | 2013-04-17 | 北京思特奇信息技术股份有限公司 | Method and system for importing common files to database |
| CN103491091A (en) * | 2013-09-24 | 2014-01-01 | 长沙裕邦软件开发有限公司 | Method and system for monitoring user operation based on data bank |
| CN104123367B (en) * | 2014-07-24 | 2017-06-30 | 中国农业银行股份有限公司 | Data migration method and system of the non-product factory mode to product factory mode |
| CN104123367A (en) * | 2014-07-24 | 2014-10-29 | 中国农业银行股份有限公司 | Data migration method and system from non-product-factory pattern to product factory pattern |
| CN104484429A (en) * | 2014-12-18 | 2015-04-01 | 山东中创软件工程股份有限公司 | Data contrast analysis method and data contrast analysis system |
| CN105373621A (en) * | 2015-12-07 | 2016-03-02 | 高新兴科技集团股份有限公司 | Rapid database-system-across data increment migration method |
| CN105574141A (en) * | 2015-12-15 | 2016-05-11 | 杭州朗和科技有限公司 | Method and device for migrating data of database |
| CN105574141B (en) * | 2015-12-15 | 2021-04-27 | 杭州朗和科技有限公司 | Method and device for carrying out data migration on database |
| CN106776876A (en) * | 2016-11-29 | 2017-05-31 | 用友网络科技股份有限公司 | Data migration method and data mover system |
| CN106815317A (en) * | 2016-12-23 | 2017-06-09 | 中国银联股份有限公司 | A kind of data copy method and system based on disparate databases |
| CN107357885A (en) * | 2017-06-30 | 2017-11-17 | 北京奇虎科技有限公司 | Method for writing data and device, electronic equipment, computer-readable storage medium |
| CN107357885B (en) * | 2017-06-30 | 2020-11-20 | 北京奇虎科技有限公司 | Data writing method and device, electronic equipment and computer storage medium |
| CN107644122A (en) * | 2017-08-29 | 2018-01-30 | 深圳市兴森快捷电路科技股份有限公司 | A kind of ODB++ file modifications method, apparatus and readable storage medium storing program for executing |
| CN107480224A (en) * | 2017-09-11 | 2017-12-15 | 爱普（福建）科技有限公司 | The configuration data of control station realizes the device of data sharing with third party database |
| CN108255984A (en) * | 2017-12-28 | 2018-07-06 | 浪潮通用软件有限公司 | A kind of data analysis method and device |
| CN108897773B (en) * | 2018-05-31 | 2019-12-27 | 湖南格凡安信科技有限公司 | Transparent online database anonymization data parallel migration method |
| CN108897773A (en) * | 2018-05-31 | 2018-11-27 | 湖南格凡安信科技有限公司 | A kind of transparent online database anonymization data parallel moving method |
| WO2020015151A1 (en) * | 2018-07-18 | 2020-01-23 | 平安科技（深圳）有限公司 | Parameter insertion method and apparatus for database table, terminal device, and storage medium |
| CN109408588A (en) * | 2018-09-07 | 2019-03-01 | 武汉达梦数据库有限公司 | A kind of method of breakpoint transmission when database synchronization initialization loading |
| CN109284281A (en) * | 2018-09-10 | 2019-01-29 | 北京百度网讯科技有限公司 | Method and apparatus for migrating data |
| CN111125085A (en) * | 2018-10-30 | 2020-05-08 | 千寻位置网络有限公司 | Python-based database index optimization method, device and system |
| CN111309790A (en) * | 2018-11-27 | 2020-06-19 | 千寻位置网络有限公司 | Automatic ETL configuration method and device based on maxcomputer database and control system |
| CN112988701A (en) * | 2019-12-13 | 2021-06-18 | 中国电信股份有限公司 | Database management method, device, system and storage medium |
| CN113377763A (en) * | 2020-03-10 | 2021-09-10 | 阿里巴巴集团控股有限公司 | Database table switching method and device, electronic equipment and computer storage medium |
| CN112434059A (en) * | 2021-01-26 | 2021-03-02 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN112434059B (en) * | 2021-01-26 | 2021-06-22 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN113704228A (en) * | 2021-08-31 | 2021-11-26 | 北京达佳互联信息技术有限公司 | Method and device for field self-increment, electronic equipment and storage medium |
| CN113704228B (en) * | 2021-08-31 | 2024-02-06 | 北京达佳互联信息技术有限公司 | Method, device, electronic equipment and storage medium for field self-addition |
| CN113918996A (en) * | 2021-11-24 | 2022-01-11 | 企查查科技有限公司 | Distributed data processing method, device, computer equipment and storage medium |
| CN113918996B (en) * | 2021-11-24 | 2024-03-26 | 企查查科技股份有限公司 | Distributed data processing method, device, computer equipment and storage medium |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101706803B (en) | 2012-07-04 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101706803B (en) | Method for migrating production data of weaving workshop | |
| US11360950B2 (en) | System for analysing data relationships to support data query execution | |
| US20210011891A1 (en) | System for importing data into a data repository | |
| US7739547B2 (en) | Failure recovery and error correction techniques for data loading in information warehouses | |
| CN100570604C (en) | A kind of method of relation data base applications automatic upgrade | |
| CN101183387A (en) | Increment data capturing method and system | |
| US7162688B1 (en) | Method for automated generation and assembly of specifications documents in CADD environments | |
| WO2018051098A1 (en) | System for data management in a large scale data repository | |
| CN105138635B (en) | A method of it is compared using cryptographic Hash and carries out data increment duplication | |
| CN107357942A (en) | A kind of source code auxiliary Core Generator and its generation method | |
| CN101923566A (en) | Data increment extraction method based on trigger | |
| CN105279285A (en) | Synchronization system and method of relation-type database and non-relation-type database | |
| CN101499063A (en) | Tracing-based database schema evolution method and system | |
| CN105224527A (en) | Be applicable to the general ETL method of multiple object table update mode | |
| CN103927314A (en) | Data batch processing method and device | |
| CN105808653A (en) | User label system-based data processing method and device | |
| WO2019024391A1 (en) | Differential code synchronization method, storage medium, electronic device and system | |
| CN103019826A (en) | Method and device for processing events | |
| CN107766521B (en) | Power grid planning data version control method | |
| CN114169860A (en) | Enterprise organizational structure synchronization method | |
| CN112948359A (en) | Non-stop client data migration method based on database sub-base sub-table | |
| CN112328696A (en) | Method and device for synchronizing data in MySQL database to SQL Server database | |
| CN114020719A (en) | License data migration method applied to heterogeneous database | |
| US20240126747A1 (en) | Control method, and adapter | |
| CN110928892B (en) | Data information scanning synchronization system and method |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120704 Termination date: 20121127 |