CN101706803B - Method for migrating production data of weaving workshop - Google Patents
Method for migrating production data of weaving workshop Download PDFInfo
- Publication number
- CN101706803B CN101706803B CN2009102287965A CN200910228796A CN101706803B CN 101706803 B CN101706803 B CN 101706803B CN 2009102287965 A CN2009102287965 A CN 2009102287965A CN 200910228796 A CN200910228796 A CN 200910228796A CN 101706803 B CN101706803 B CN 101706803B
- Authority
- CN
- China
- Prior art keywords
- data
- field
- database
- statement
- subsidiary factory
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Expired - Fee Related
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method for migrating production data of a weaving workshop, which comprises a central database, a central data sheet structure, a branch factory database and a branch factory data sheet structure and adopts the following steps: (1) system initialization; (2) data connection configuration, namely providing an editing function for data connection; (3) data synchronization, namely finishing the migration of the production data from the branch factory database to the central database; and (4) data recovery, namely, when a database system of a certain branch factory collapses and cannot operate normally due to an accident, utilizing a branch factory identification to extract the production data of a corresponding branch factory from the central database, recovering the production data to the database of the branch factory and ensuring the completeness of the branch factory data. The migrating method of the invention has the characteristics of reconfigurability, easy expansion, support for transmission resuming at break-point, incremental migration, periodic migration, and support for the data transmission between different databases.
Description
Technical field
The present invention relates to the data processing technique of Industry Control, be specially a kind of moving method of weaving the Workshop Production data.
Background technology
In recent years; The textile industry develop rapidly; The proportion of data message management also progressively increases, and various production information data volumes also increase thereupon year by year, because the geographic position is different; Different division datas under the same company all are stored on the server separately, and interdepartmental data circulation has been caused obstacle.Effectively and accurate data can't be from another department that circulates efficiently an of department, and basic configuration and often independent existence in a plurality of systems of coded data, basic data maintenance and synchronous difficulty caused.A medium-sized textile enterprise sets up has 3-5 to weave subsidiary factory, and the loom that turns round simultaneously in each subsidiary factory has hundreds of and even thousands of.Weaving supervisory system is stored in the production data that collects on the production line in the special-purpose separately database server of subsidiary factory; The company information center then needs to pass through in addition the production data recording of the whole enterprise of independent database server stores, and this record is used for the data analysis at company information center on the other hand on the one hand as the data backup of subsidiary factory.Exist in the enterprise information system multisystem of data and reuse problem, also therefore need production data to carry out effectively and efficiently data migtation weaving between subsidiary factory and the company information center.
SQL Server 2000 databases of existing Microsoft provide one to be used for the assembly that data importing is derived, i.e. data-switching service (Data Transformaion Services is called for short DTS).DTS can realize the data dump between the SQL Server data source, and also can be the data source of other OLE DB, ODBC driver or the data importing in the text in SQL Server.Utilize the instrument DTS Import/Export Wizard under the DTS can carry out importing and exporting of data, its step is following:
(1) opens DTS Import/Export Wizard, select data source, data source type, authentication mode, database;
(2) the target database type, database-name, authentication mode etc. of importing are set;
(3) data importing is set and derives details, appointment is to duplicate one or more table/views from data source, still duplicates Query Result;
(4) select data source table or view, the script that uses when the mapping relations that are listed as between data source table and the purpose table when data-switching is set, conversion, and write transformation rule;
(5) be provided with whether preserve DTS bag, the type of bag and the scheduling mode of bag;
(6), fill in title, description, owner's password, the user cipher of bag and the server location and the verification mode of preservation are set if select to preserve bag;
(7) information that imports and exports setting is confirmed, errors excepted, returned several steps and revise;
(8) behind the validation of information, instrument is accomplished data importing and is derived.
The data importing derivation of prior art or the shortcoming of moving method are: 1. do not support data source to change during data move; 2. the data amount of movement is big, does not support the difference transmission; 3. do not support periodic data migtation; 4 pairs of concrete data list structure dependences are bigger, need human configuration literary name section.
Summary of the invention
To the deficiency of prior art, technical matters to be solved by this invention is that a kind of moving method of weaving the Workshop Production data is provided.This moving method can reduce in the data migtation dependence to the data list structure; Realize the data sync of subsidiary factory's database to central database; Solve the problem of utilizing the centre data database data to recover subsidiary factory's data simultaneously; Reinforcement improves the versatility of data migtation to the tracking of data transition process, can adapt to the data migtation between the several data storehouse.
The technical scheme that the present invention solve the technical problem is: design a kind of moving method of weaving the Workshop Production data.This moving method comprises central database, centre data list structure, subsidiary factory's database and subsidiary factory's data list structure, and adopts following steps:
(1) system initialization after system start-up, is set up a unique system configuration instance, and the reading system configuration file is given name-value pair information assignment wherein in this instance corresponding property, accomplishes the initialization of systematic parameter;
(2) data connect configuration, and the editting function that data are connected is provided, and connect through selecting corresponding data, get into edit phase, accept the user and edit; After editor submits to, editor's data are carried out verification, the legitimacy of inspection data, the reliability that test data connects and the data of configuration connect the requirement that whether meets centre data list structure and subsidiary factory's data list structure; Upgrade current system configuration parameter after the verification succeeds, comprise update system profile instance and new configuration file more;
(3) data sync is accomplished production data from the migration of subsidiary factory's database to central database;
(4) data are recovered; When certain subsidiary factory causes the Database Systems collapse to run well owing to accident; Utilize subsidiary factory to identify the production data that extracts corresponding subsidiary factory from central database, return in the database of this subsidiary factory, guarantee the integrality of subsidiary factory's data.
Wherein, said step (3) data sync comprises:
(31) synchronous initiation obtains the data link information from the system configuration instance, set up being connected of subsidiary factory's correspondence table and center correspondence table, at subsidiary factory's end creation operation record sheet, on subsidiary factory's correspondence table, creates trigger, the data manipulation on the record subsidiary factory correspondence table;
(32) cycle synchronisation; Regularly from said operation sheet, obtain by the data operation information under the said recording trigger; Binding operation type, added field are spliced into SQL statement with this operation information; On the centre data table, carry out said SQL statement, realize data variation is reacted on the centre data table, accomplish synchronization of data;
Said step (4) data are recovered to comprise:
(41) utilize subsidiary factory the partial data list structure field information and combine subsidiary factory to be identified to set up on the centre data table one to recover view; Said recovery view provides the total data information of recovering usefulness;
(42) recover thread and from said recovery view, extract data, be spliced into the insertion statement; On subsidiary factory's tables of data, carry out said insertion statement, accomplish the recovery of data;
(43) move on to next bar data, repeating step (42) recovers to accomplish up to total data.
Compared with prior art; Moving method of the present invention has designed step and implementation method and corresponding processing rules such as initialization, connection configuration, data sync, the data of database migration system are recovered, database is adaptive; Have configurable, be easy to expansion, support breakpoint transmission, increment migration, periodically migration and support between disparate databases characteristics such as data transmission, guaranteed the high efficiency and the integrality of data migtation.Specifically, the beneficial effect of moving method of the present invention is: in the process of carrying out data migtation,
1. support the difference incremental mode to carry out the periodic data migration;
2. data migration process does not rely on structure of data table and field type;
3. guarantee to connect the correctness of configuration through the matching test rule of list structure;
4. the data transition process is implemented monitoring and record, be convenient to fault diagnosis;
5. write down the migration progress when data are recovered automatically, have the breakpoint transmission ability;
6. support the data migtation between the disparate databases through the database adaptation method;
Description of drawings
Fig. 1 is the subsidiary factory of a kind of embodiment of moving method of the present invention and the graph of a relation between the central database list structure; Wherein,
Fig. 1 (a) is subsidiary factory's database production data list structure synoptic diagram;
Fig. 1 (b) is a central database production data list structure synoptic diagram;
Fig. 2 is the operation sheet visioning procedure synoptic diagram of a kind of embodiment of moving method of the present invention;
Fig. 3 is the trigger visioning procedure synoptic diagram of a kind of embodiment of moving method of the present invention;
Fig. 4 is the trigger statement splicing schematic flow sheet of a kind of embodiment of moving method of the present invention;
Fig. 5 is the cycle synchronisation schematic flow sheet of a kind of embodiment of moving method of the present invention.
Embodiment
Below in conjunction with accompanying drawing and embodiment the present invention is described in further detail:
Moving method of the present invention (be called for short moving method, referring to Fig. 1-5) structurally comprises four parts: 1. system initialization, 2. data connect configuration section, 3. data sync part and 4. data recovered part.Behind the system initialization; The user can connect configuration section through data and revise current systematic parameter; After data connection configuration finishes, can optionally partly start corresponding data sync task, start corresponding data recovery tasks through the data recovered part through data sync." disparate databases is adaptive " technology is incorporated in four parts, makes whole moving method be with good expansibility.
The production data storage environment that moving method embodiment of the present invention weaves the workshop to textile enterprise designs.For the ease of statement, below represent the company information central database with Center, represent with Factory and weave subsidiary factory's database.In this specific memory environment, production data leaves among the Factory after being gathered by " loom production data supervisory system ", is integrated among the Center through this data migtation afterwards, has accomplished the double copies of data at Factory and Center.When production data is stored in Center and Factory; The list structure of tables of data has certain relation; Concrete structural relation be (referring to Fig. 1): Center with Factory in list structure in have identical field i (i=1,2......n), these fields are used for preserving production data; All have in the list structure of Center and Factory from increasing field, but concrete numerical value independent variation separately; Tables of data among the Center has increased an added field collection than the tables of data among the Factory, is used for customizing messages such as expression " subsidiary factory's sign ", " production kind ".
1. system initialization
System initialization is meant after system start-up, sets up a unique system configuration instance, and the reading system configuration file is given name-value pair information assignment wherein in this instance corresponding property, accomplishes the initialization of systematic parameter.The purpose of system initialization is for other three parts current up-to-date system parameter message to be provided.
2. data connect configuration section
Data connect configuration section provides the editting function that data are connected.Through selecting corresponding data to connect, get into edit phase, accept the user and edit.Editor submits to the back that editor's data are carried out the school inspection; The legitimacy of inspection data; The reliability that test data connects and the data of configuration connect the list structure that whether meets Center and Factory and require (referring to Fig. 1); Be partly to have the difference except that increasing field and added field collection certainly in centre data list structure and the subsidiary factory's data list structure, title is identical in two list structures, storage class is identical or corresponding (when the disparate databases) for all the other each fields.The school is examined successfully the back and is upgraded current system configuration parameter, comprises update system profile instance and new configuration file more.
Data migtation will be used the parameter information that configuration file provides system initialization, has wherein write down the relevant information that data connect.There are two kinds of attributes configuration file inside, public attribute and privately owned attribute.Public attribute is used to control whole System Operation, and the synchronous task or the recovery tasks that are connected with concrete certain data are irrelevant; Privately owned attribute is to be subordinated to the attribute that certain particular data connects, and being connected with this of task is in close relations, and all task initialization information all obtains from privately owned attribute.In addition, also have one type of specific properties in the privately owned attribute, be used for auxiliary added field and handle, whether this generic attribute the value of other attribute occurs depending on.For example addFieldNameY and addFieldValueY occur when addFieldNum＞0.Concrete configuration file structure is listed in table 1:
Table 1 data migtation is used the configuration file structural table
| The property key title | Implication | Explanation |
| Sleep | The sync interval time | The common configuration attribute |
| connectNum | The current data linking number | The common configuration attribute |
| mssqlX.desc | Data connect to be described | Privately owned attribute |
| mssqlX.url | The database connection string | Privately owned attribute, subsidiary factory's end |
| mssqlX.DBtype | Type of database | Privately owned attribute, subsidiary factory's end |
| mssqlX.driver | The database-driven character string | Privately owned attribute, subsidiary factory's end |
| mssqlX.user | Database user name | Privately owned attribute, subsidiary factory's end |
| mssqlX.password | Database password | Privately owned attribute, subsidiary factory's end |
| mssqlX.together.url | The database connection string | Privately owned attribute, center-side |
| mssqlX.together.DBType | Type of database | Privately owned attribute, center-side |
| mssqlX.together.driver | The database-driven character string | Privately owned attribute, center-side |
| mssqlX.together.user | Database user name | Privately owned attribute, center-side |
| mssqlX.together.password | Database password | Privately owned attribute, center-side |
| mssqlX.sourceTable | Subsidiary factory's end correspondence table | Privately owned attribute, subsidiary factory's end |
| mssqlX.targetTable | The center-side correspondence table | Privately owned attribute, center-side |
| mssqlX.addFieldNum | The added field number | Privately owned attribute |
| mssqlX.addFieldNameY | The added field title | Privately owned attribute |
| mssqlX.addFieldValueY | The add-word segment value | Privately owned attribute |
In the table 1, the hyphen that the X representative data connects, Y are represented the added field numbering, and numbering is since 1.Mode with key-value pair in the configuration file is stored.
3. data sync part
The data sync part will be accomplished production data from the migration of subsidiary factory's database to central database, specifically is divided into two stages, i.e. synchronous initiation stage and cycle synchronisation stage.The synchronous initiation stage, obtain the data link information from the system configuration instance, creation operation record sheet in subsidiary factory's database is used to write down the data manipulation on subsidiary factory's table at establishment trigger on subsidiary factory's table.The cycle synchronisation stage; Regularly from subsidiary factory's end record sheet, obtain by the data operation information under the recording trigger; Binding operation type, added field are spliced into SQL statement with this operation; On the centre data table, carry out this SQL statement, data variation is reacted on the centre data table, accomplish the migration of data.
The synchronous initiation stage is wanted creation operation record sheet and operation note trigger.
The operation that operation sheet is used on the data table memory changes, and its data are provided by the operation note trigger.Each field of operation sheet specifies as follows:
Field id: major key increases field, the numbering of sign operation note certainly.
Field optype: this field has three kinds of values: 0: represent the delete operation; 1: represent the update operation; 2: represent the insert operation.
Field statement: be used to store data manipulation statement by under the recording trigger.The data manipulation device that is triggered is noted, and is written in this field with half-done SQL statement form.
Operation sheet is created by Automatic Program in the synchronous initiation process, according to the difference of type of database, adopts different database statements to create, and the operation sheet visioning procedure is (referring to Fig. 2) as follows:
(1) reading of data link information;
(2) obtain synchronisation source table name and type of database;
(3) obtain and being connected of synchronisation source tables of data;
(4) the adaptive selection of statement type;
(5) SQL statement and source table name splicing;
(6) carry out SQL statement at source database and create record sheet.
The operation note trigger is used for writing down automatically the operation to subsidiary factory's tables of data.Will inquire about the field information that obtains subsidiary factory's tables of data when creating trigger, after field title that can write field in this tables of data and type extraction, be spliced in the SQL statement of creating trigger, the trigger visioning procedure is (referring to Fig. 3) as follows:
(1) reading of data link information;
(2) obtain and being connected of synchronisation source tables of data;
(3) obtain source literary name segment information;
(4) type of database coupling;
(5) update/delete/insert trigger statement splicing;
(6) at the splicing of destination data storehouse execution synchronously statement;
(7) trigger is set up successfully prompting.
The record format of being created out by trigger is:
A. deletion action record: " the synchronous purpose table name of delete from where major key name=Major key [and major key name 2=Major key ...] ", wherein the number of major key is different because of table.
B. upgrade operation note: " update master meter name set field name=field value where major key name=Major key [and major key name 2=Major key ...] ", the field number of renewal is different because of operation.
C. insert operation note: " field value [, field value 2...] ", number is the Field Count of this table.
The statement splicing (referring to Fig. 4) as follows in (5) step in the trigger constructive process:
(1) selects adaptation rule;
(2) splice with source table table name;
(3) obtain next field information;
(4) judge whether current field is to write field, be then to carry out (5), otherwise carry out (3);
(5) with SQL statement and the splicing of current field;
(6) judged whether next field, be then to carry out (3), otherwise carry out (7);
(7) with the splicing of operation note table name.
In the whole circulation process, increase these two useless fields of field and calculated column field certainly and abandoned automatically, remaining field all is the fields of carrying effective production data.
The establishment of synchronous initiation stage complete operation record sheet and operation note trigger gets into the cycle synchronisation stage afterwards.The flow process in cycle synchronisation stage specifies as follows (referring to Fig. 5):
(1) start a synchronizing thread, this thread serves as a mark with the data hyphen;
(2) call the flip-flop states administration module, three triggers of synchronisation source table are all launched;
(3) obtain all records in the corresponding operation sheet of subsidiary factory tables of data, pointed article one record.Like no record, then change (9);
(4) according to the type of database at data connection two ends, call the function that is used for Data Format Transform the value value part of record is carried out format conversion, make new form be suitable for the destination data storehouse; Form is convertData (source format numbering, purpose form numbering, an original character string), and rreturn value is the purpose character string through format conversion;
(5) according to optype field decision operation type, if deletion and upgrade operation then directly and added field partly splice and get final product; If insert operation, the write field of inquiry subsidiary factory table is spliced with the character string among these fields and the statement, then carries out added field again and handles.
Insert statement at last and will be spliced into following form:
" insert into purpose table table name (and can write field [, added field ...]) values (the statement character string [, the add-word segment value ...]) ";
(6) obtain the connection of central database, in this connection, carry out the above-mentioned SQL statement of piecing together;
(7) run succeeded, corresponding record in the deletion record table then, and on the synchronous operation counter of corresponding types, add one;
(8) if carry out failure, then doing unusually of generation caught processing, carry out log record.If special unusual, task can't be proceeded, and changes (10), if generally unusual, then next bar record of pointed changes (4);
(9) once time synchronization carry out to finish, and log record successfully carries out the value of each synchronous synchronous operation counter, then with counter O reset, and thread dormancy a period of time, wait pending synchronously next time, return (3);
(10) synchronizing thread stops, and synchronous task finishes.
With the data sync that this mode is carried out, the data of transmission only are the production datas that changes at subsidiary factory and center, have improved transport efficiency.
4. data recovered part
Because the unexpected data system that causes collapses when can't run well, utilize subsidiary factory to identify the production data that extracts corresponding subsidiary factory from central database when certain subsidiary factory, return in the database of this subsidiary factory, guarantee subsidiary factory's data integrity.Concrete grammar is to utilize the partial data list structure field information of subsidiary factory and combine subsidiary factory to be identified to set up a recovery view on the centre data table; This view provides the total data information of recovering usefulness, recovers thread one one extraction data from this view, is spliced into the insertion statement; On subsidiary factory's tables of data, carry out and to insert statement; Accomplish the recovery of data, so carry out next bar, recover to accomplish up to total data.
Data sync is to carry out data migtation from subsidiary factory's database to central database; It is then just the opposite that data are recovered, and is to carry out data migtation from central database to subsidiary factory's database, thus data recover to carry out data can shared identical configuration item when connecting configuration, to guarantee compatibility of goals.So both saved the time of configuration, guaranteed again that same data connect the uniqueness that goes up data flow, avoided the unreasonable data collision that causes owing to the task setting.For system configuration, adopt unique profile instance to realize, be convenient to the data migtation task to the data link information obtain and to the renewal of link information.Data recovery configuring part idiographic flow is following:
(1) connect selection of configuration: from the system configuration instance, obtain all data connection names,, select to edit a certain data by the user and connect as the object of selecting editor, newly-built or delete data and connect;
(2) configuration information reads: the details that from the system configuration instance, read the data connection of choosing; Comprise driving name, connection string, username and password with the database join dependency; And information such as the source table name of data migtation, purpose table name and added field, and in editing interface, show these information;
(3) edit configuration information: the information to edit page is edited, change legacy data, perhaps additions and deletions added field etc.;
(4) submit edit file to.If the deleted data attended operation then directly forwards (6) to;
(5) validity checking: the feasibility of before configuration information is submitted to, configuration information being carried out check of data legitimacy and configuration information is checked.Accomplish the data layout inspection earlier; Comprise that whether correct the input content can not be etc. for empty, input format; Then do to connect and test, set up successfully, proceed the test of service logic feasibility if data connect according to configuration information; Promptly test this configuration and whether satisfy the basic demand of data migtation, promptly the write field in the structure of subsidiary factory's table all is included in the list structure of center table.Validity checking is not passed through, and changes (3) and updates;
(6) configuration parameter upgrades: about the information of this connection, increase, delete or revise in the update system profile instance, the parameter information in the instance after will upgrading simultaneously is rewritten in the configuration file, guarantees the consistance of configuration parameter.
Described view is meant and formally carrying out before data recover, and the write field that adopts subsidiary factory's tables of data is as the basis, sets up the recovery view of this subsidiary factory according to subsidiary factory number from the centre data table, originates as the data data recovered.Through adaptation data storehouse type, select different views to create statement and create.
Described data are provided with 3 variablees, recoverNum, totalNum and finish in recovering.RecoverNum representes the record count that current success recovers, and totalNum representes record counts all in the view, and finish representes whether this recovery tasks finishes.Execution in step is following:
(1) starts a data recovery lines journey, be connected to mark with data;
(2) data recovery tasks initialization finishes the data sync task that these data connect, and forces to take these data and connects;
(3) close the trigger that subsidiary factory shows, prevent its recording operation;
(4) query log files is recovered breakpoint if data connect to go up to exist, and then reads breakpoint data assignment and gives the recoverNum variable; If do not have the recovery breakpoint, then recoverNum be changed to 0;
(5) read all data recording in the view, and vernier is jumped to breakpoint location, promptly vernier points to recoverNum+1 bar record;
(6) statement is inserted in splicing: read the list structure of subsidiary factory's correspondence table, can write the synthetic parenthesis sentence first half of field groups, from Data View, extract data, the value of each bar record is inserted the statement latter half, accomplish splicing;
(7) spliced insertion statement is carried out on subsidiary factory's table, whenever run succeeded one, recoverNum is from increasing one.If take place unusually then go to (8), continue to carry out otherwise return (6), records all in Data View were all carried out;
(8) for doing following processing unusually: do log record earlier, the time that recording exceptional takes place, abnormal information, unusual program description etc.; Preserve current recoverNum value in the breakpoint file;
(9) trigger on the startup subsidiary factory correspondence table recovers the monitoring to subsidiary factory's table;
(10) finish is set to true, and end data is recovered thread.
For the ease of checking current recovery progress, utilize the number percent of recoverNum and totalNum to be shown to the user, if the thread normal termination as the progress of recovering; Number percent should be increased to 100%, and the finish parameter is true, and is unusual when occurring; Data are recovered to finish; Finish is true, but number percent is not 100%, can observe the situation that data are recovered equally intuitively.Utilize recoverNum at this, add 1 behind record of every completion, realized the all-the-way tracking that data are recovered effectively.
5. the adaptation method of disparate databases
In order in data migration method, to realize the support to the several data storehouse, the present invention has further designed the adaptation method of isomery or disparate databases, is used to support the data migtation between the disparate databases.This adaptation method comprises the adaptation method of driver, the adaptation method of SQL statement, three parts such as adaptation method of data type conversion.Adaptation procedure has adopted four variable: sourceTable, targetTable, sourceType and targetType.Wherein sourceTable represents subsidiary factory's tables of data; TargetTable represents the centre data table; SourceType represents subsidiary factory's type of database; TargetType represents the central database type.The value of variable is 0,1,2 and 3, and implication is respectively: 0 represents SQL Server database, and 1 represents oracle database, and 2 represent sybase database, and 3 represent the DB2 database.The value of variable can be accomplished the numbering of other data base management system (DBMS) and adaptive through expanded definition, has good adaptability.
Said driver adaptation method is meant to be held for subsidiary factory in the configuration file structure that data connect and center-side has disposed a database device string attribute respectively.Before data connected foundation, the driver character string that connects two ends according to data was carried out the search and the loading of database-driven.
Said SQL statement adaptation method is meant that this part is applicable to the functional based method that produces insertion, upgrades, deletes the splicing statement; In these functional based methods; Provide type as an extra suction parameter, the type in the destination data storehouse that is used to represent that spliced SQL statement is suitable for.Provide following the statement law that should use in the various types of data storehouse that relates in functional based method inside, judge according to the type parameter that provides then, select the rule of correspondence database type.The adaptation method of concrete SQL statement is following:
switch(type)
{ case MS-SQL type: sql=sql0; Break; // be applicable to the statement of SQL Server
Case ORACLE type: sql=sql1; Break; // be applicable to the statement of Oracle
Case SYBASE type: sql=sql2; Break; // be applicable to the statement of SYBASE
Case DB2 type: sql=sql3; Break; // be applicable to the statement of DB2
......}。
The adaptation method of said data type conversion is meant for the migration between the data of different types storehouse, need before the splicing statement is carried out, partly carry out format conversion to the character string of the presentation data value in the splicing statement, makes it the requirement of adaptation to end database.A special format conversion method is provided for this reason, and form is convertData (source database type, destination data storehouse type, an original character string), and rreturn value is the purpose character string through format conversion.
This adaptation method was called before the statement splicing of data sync, after the statement splicing that data are recovered, was called.The adaptive concrete grammar of data type conversion is following:
(1) " the character string part behind the value (", first value value of pointed in the query statement;
(2) inquire about the corresponding field title of this value;
(3) this field must be to write field, obtains the SQL types value of this field in source end and destination database respectively, and its rreturn value is used numeral;
(4) with two SQL types value preceding two suction parameters as convertData ();
(5) the value value is called convertData () method as the 3rd suction parameter, obtain rreturn value newValue;
(6) replace current value value with newValue, accomplish the type conversion of a field;
(7) pointer is skipped a comma and is pointed to next value value, returns (2), ") in running into statement " finish.
The present invention does not address part and is applicable to prior art.
Moving method of the present invention adopts cross-platform technological means; Designed a kind of effective implementation method of data migtation; Solved and weaved in the enterprise information system multisystem of data and reuse problem; Greatly reduced in the data migration process the degree of dependence of database table structure, strengthened tracking, improved the dirigibility and the security of data migtation the data transition process; Thereby solved configurable, easy expansion, be easy to integrated data migtation problem, realized that textile industry weaves production data correct, effective mobility between each subsidiary factory and enterprise center.Though what the embodiment of moving method of the present invention was directed against is to weave the information center of enterprise and effective transmission problem of subsidiary factory's database, it is equally applicable to other enterprise and establishments of identical or similar demand.
Claims (2)
1. moving method of weaving the Workshop Production data, this moving method adopts following steps:
(1) system initialization after system start-up, is set up a unique system configuration instance, and the reading system configuration file is given configuration information assignment wherein in this instance corresponding property, accomplishes the initialization of systematic parameter;
(2) data connect configuration, and the editting function that data are connected is provided, and connect through selecting corresponding data, get into edit phase, accept the user and edit; After editor submits to, editor's data are carried out verification, the legitimacy of inspection data, the reliability that test data connects and the data of configuration connect the requirement that whether meets centre data list structure and subsidiary factory's data list structure; Upgrade current system configuration parameter after the verification succeeds, comprise update system profile instance and new configuration file more;
(3) data sync is accomplished production data from the migration of subsidiary factory's database to central database;
(4) data are recovered; When certain subsidiary factory causes the Database Systems collapse to run well owing to accident; Utilize subsidiary factory to identify the production data that extracts corresponding subsidiary factory from central database, return in the database of this subsidiary factory, guarantee the integrality of subsidiary factory's data;
Wherein, said step (3) data sync comprises:
(31) synchronous initiation; From the system configuration instance, obtain the data link information, set up being connected of subsidiary factory's correspondence table and center correspondence table, at subsidiary factory's end creation operation record sheet; Creation operation record trigger on subsidiary factory's correspondence table, the data manipulation on the record subsidiary factory correspondence table; The synchronous initiation stage is wanted creation operation record sheet and operation note trigger: the operation that operation sheet is used to store on subsidiary factory's tables of data changes, and its data are provided by the operation note trigger, and each field of operation sheet specifies as follows:
Field id: major key increases field, the numbering of sign operation note certainly;
Field optype: this field has three kinds of values: 0: represent the delete operation; 1: represent the update operation; 2: represent the insert operation;
Field statement: be used to store data manipulation statement by under the operation note recording trigger; Data manipulation is got off by the operation note recording trigger, and SQL statement form complete not write as yet, that can not in database, carry out is written in this field;
Operation sheet is created by Automatic Program in the synchronous initiation process, according to the difference of type of database, adopts different database statements to create, and the operation sheet visioning procedure is following:
The operation note trigger is used for writing down automatically the operation to subsidiary factory's tables of data; To inquire about the field information that obtains subsidiary factory's tables of data during creation operation record trigger; After field title that can write field in this subsidiary factory's tables of data and type extraction; Be spliced in the SQL statement of creation operation record trigger, operation note trigger visioning procedure is following:
The splicing of
update/delete/insert operation note trigger statement;
The record format of being created out by the operation note trigger is:
A. deletion action record: " the synchronous purpose table name of delete from where major key name=Major key [and major key name 2=Major key 2 ...] ", wherein the number of major key is different because of table;
B. upgrade operation note: " update master meter name set field name=field value where major key name=Major key [and major key name 2=Major key 2 ...] ", the field number of renewal is different because of operation;
C. insert operation note: " field value [, field value 2 ...] ", number is the Field Count of this table;
The statement splicing in
step is following in the operation note trigger constructive process:
splices with synchronisation source tables of data table name;
In the whole circulation process, increase these two useless fields of field and calculated column field certainly and abandoned automatically, remaining field all is the fields of carrying effective production data;
(32) cycle synchronisation; Regularly from said operation sheet, obtain by the data operation information under the said operation note recording trigger; Binding operation type, added field are spliced into SQL statement with this operation information; On the centre data table, carry out said SQL statement, realize data variation is reacted on the centre data table, accomplish synchronization of data; The flow process in cycle synchronisation stage specifies as follows:
Insert statement at last and will be spliced into following form:
" insert into purpose table table name (and can write field [, added field ...]) values (the statement character string [, the add-word segment value ...]) ";
synchronizing thread stops, and synchronous task finishes;
Said step (4) data are recovered to comprise:
(41) utilize subsidiary factory the partial data list structure field information and combine subsidiary factory to be identified to set up on the centre data table one to recover view; Said recovery view provides the total data information of recovering usefulness;
(42) recover thread and from said recovery view, extract data, be spliced into the insertion statement; On subsidiary factory's tables of data, carry out said insertion statement, accomplish the recovery of data;
(43) move on to next bar data, repeating step (42) recovers to accomplish up to total data.
2. the moving method of weaving the Workshop Production data according to claim 1; It is characterized in that described moving method also comprises the adaptation method of disparate databases, this adaptation method comprises the adaptation method of driver, the adaptation method of SQL statement and the adaptation method of data type conversion;
Said driver adaptation method is meant to be held for subsidiary factory in the configuration file structure that data connect and center-side has disposed a database device string attribute respectively; Before data connected foundation, the driver character string that connects two ends according to data was carried out the search and the loading of database-driven;
Said SQL statement adaptation method is meant and is applicable to the functional based method that produces insertion, upgrades, deletes the splicing statement; In these functional based methods; Provide type as an extra suction parameter, the type in the destination data storehouse that is used to represent that spliced SQL statement is suitable for provides following the statement law that should use in the various types of data storehouse that relates in functional based method inside; Judge according to the type parameter that provides then, select the rule of correspondence database type;
The adaptation method of said data type conversion is meant for the migration between the data of different types storehouse; Need, the splicing statement partly carry out format conversion before carrying out to the character string of the presentation data value in the splicing statement; The form of format conversion method is a convertData (source database type; Destination data storehouse type, the original character string), rreturn value is the purpose character string through format conversion; This adaptation method was called before the statement splicing of data sync, after the statement splicing that data are recovered, was called.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102287965A CN101706803B (en) | 2009-11-27 | 2009-11-27 | Method for migrating production data of weaving workshop |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2009102287965A CN101706803B (en) | 2009-11-27 | 2009-11-27 | Method for migrating production data of weaving workshop |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN101706803A CN101706803A (en) | 2010-05-12 |
| CN101706803B true CN101706803B (en) | 2012-07-04 |
Family
ID=42377028
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2009102287965A Expired - Fee Related CN101706803B (en) | 2009-11-27 | 2009-11-27 | Method for migrating production data of weaving workshop |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN101706803B (en) |
Families Citing this family (28)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102393860A (en) * | 2011-12-06 | 2012-03-28 | 方正国际软件有限公司 | Database data migration system |
| CN103176989B (en) * | 2011-12-21 | 2016-09-28 | 中国银联股份有限公司 | Data base's table level comparative approach based on data dictionary and variable rules and system |
| CN102495906A (en) * | 2011-12-23 | 2012-06-13 | 天津神舟通用数据技术有限公司 | Incremental data migration method capable of realizing breakpoint transmission |
| CN102609337A (en) * | 2012-01-19 | 2012-07-25 | 北京神州数码思特奇信息技术股份有限公司 | Rapid data recovery method for memory database |
| CN102761614A (en) * | 2012-06-29 | 2012-10-31 | 浪潮(北京)电子信息产业有限公司 | Method and system for realizing breakpoint resume of network data transmission |
| CN103049550A (en) * | 2012-12-27 | 2013-04-17 | 北京思特奇信息技术股份有限公司 | Method and system for importing common files to database |
| CN103491091A (en) * | 2013-09-24 | 2014-01-01 | 长沙裕邦软件开发有限公司 | Method and system for monitoring user operation based on data bank |
| CN104123367B (en) * | 2014-07-24 | 2017-06-30 | 中国农业银行股份有限公司 | Data migration method and system of the non-product factory mode to product factory mode |
| CN104484429A (en) * | 2014-12-18 | 2015-04-01 | 山东中创软件工程股份有限公司 | Data contrast analysis method and data contrast analysis system |
| CN105373621A (en) * | 2015-12-07 | 2016-03-02 | 高新兴科技集团股份有限公司 | Rapid database-system-across data increment migration method |
| CN105574141B (en) * | 2015-12-15 | 2021-04-27 | 杭州朗和科技有限公司 | Method and device for carrying out data migration on database |
| CN106776876A (en) * | 2016-11-29 | 2017-05-31 | 用友网络科技股份有限公司 | Data migration method and data mover system |
| CN106815317A (en) * | 2016-12-23 | 2017-06-09 | 中国银联股份有限公司 | A kind of data copy method and system based on disparate databases |
| CN107357885B (en) * | 2017-06-30 | 2020-11-20 | 北京奇虎科技有限公司 | Data writing method and device, electronic equipment and computer storage medium |
| CN107644122A (en) * | 2017-08-29 | 2018-01-30 | 深圳市兴森快捷电路科技股份有限公司 | A kind of ODB++ file modifications method, apparatus and readable storage medium storing program for executing |
| CN107480224B (en) * | 2017-09-11 | 2021-06-22 | 爱普（福建）科技有限公司 | Device for realizing data sharing between configuration data of control station and third-party database |
| CN108255984A (en) * | 2017-12-28 | 2018-07-06 | 浪潮通用软件有限公司 | A kind of data analysis method and device |
| CN108897773B (en) * | 2018-05-31 | 2019-12-27 | 湖南格凡安信科技有限公司 | Transparent online database anonymization data parallel migration method |
| CN109063044B (en) * | 2018-07-18 | 2023-04-18 | 平安科技（深圳）有限公司 | Parameter insertion method and device for database table, terminal equipment and storage medium |
| CN109408588A (en) * | 2018-09-07 | 2019-03-01 | 武汉达梦数据库有限公司 | A kind of method of breakpoint transmission when database synchronization initialization loading |
| CN109284281A (en) * | 2018-09-10 | 2019-01-29 | 北京百度网讯科技有限公司 | Method and apparatus for migrating data |
| CN111125085A (en) * | 2018-10-30 | 2020-05-08 | 千寻位置网络有限公司 | Python-based database index optimization method, device and system |
| CN111309790A (en) * | 2018-11-27 | 2020-06-19 | 千寻位置网络有限公司 | Automatic ETL configuration method and device based on maxcomputer database and control system |
| CN112988701A (en) * | 2019-12-13 | 2021-06-18 | 中国电信股份有限公司 | Database management method, device, system and storage medium |
| CN113377763B (en) * | 2020-03-10 | 2022-10-28 | 阿里巴巴集团控股有限公司 | Database table switching method and device, electronic equipment and computer storage medium |
| CN112434059B (en) * | 2021-01-26 | 2021-06-22 | 腾讯科技（深圳）有限公司 | Data processing method, data processing device, computer equipment and storage medium |
| CN113704228B (en) * | 2021-08-31 | 2024-02-06 | 北京达佳互联信息技术有限公司 | Method, device, electronic equipment and storage medium for field self-addition |
| CN113918996B (en) * | 2021-11-24 | 2024-03-26 | 企查查科技股份有限公司 | Distributed data processing method, device, computer equipment and storage medium |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1763745A (en) * | 2005-09-06 | 2006-04-26 | 阎海强 | Structure oriented data migration method and system |
| CN101266606A (en) * | 2007-03-15 | 2008-09-17 | 阿里巴巴公司 | On-line data migration method based on Oracle database |
-
2009
- 2009-11-27 CN CN2009102287965A patent/CN101706803B/en not_active Expired - Fee Related
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1763745A (en) * | 2005-09-06 | 2006-04-26 | 阎海强 | Structure oriented data migration method and system |
| CN101266606A (en) * | 2007-03-15 | 2008-09-17 | 阿里巴巴公司 | On-line data migration method based on Oracle database |
Non-Patent Citations (2)
| Title |
|---|
| 刘天时等.信息系统数据迁移方法研究与应用.《西北大学学报（自然科学版）》.2006,第36卷(第1期),25－28. * |
| 袁莉芬.Oracle中批量数据迁移的实现方案.《湘潭师范学院学报(自然科学版)》.2005,第27卷(第1期),25-26. * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN101706803A (en) | 2010-05-12 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN101706803B (en) | Method for migrating production data of weaving workshop | |
| US11461294B2 (en) | System for importing data into a data repository | |
| US7739547B2 (en) | Failure recovery and error correction techniques for data loading in information warehouses | |
| CN100570604C (en) | A kind of method of relation data base applications automatic upgrade | |
| US7496596B2 (en) | Detecting migration differences of a customized database schema | |
| CN101183387A (en) | Increment data capturing method and system | |
| US6993708B1 (en) | System for automated generation and assembly of specifications documents in CADD environments | |
| CN109871373B (en) | Data storage method and device and computer readable storage medium | |
| US20110231358A1 (en) | Method and apparatus for loading data files into a data-warehouse system | |
| CN105279285A (en) | Synchronization system and method of relation-type database and non-relation-type database | |
| CN105224527A (en) | Be applicable to the general ETL method of multiple object table update mode | |
| CN101499063A (en) | Tracing-based database schema evolution method and system | |
| CN107479881B (en) | Method for synchronizing difference codes, storage medium, electronic device and system | |
| CN103927314A (en) | Data batch processing method and device | |
| CN105354036A (en) | Database operation code generating method and apparatus | |
| CN105808653A (en) | User label system-based data processing method and device | |
| CN103077019B (en) | A kind of graphical parameter batch maintenance method and device | |
| CN103019826A (en) | Method and device for processing events | |
| CN108984337A (en) | A kind of restorative procedure of data synchronous abnormality, prosthetic device, medium and calculate equipment | |
| CN114169860A (en) | Enterprise organizational structure synchronization method | |
| CN106155838A (en) | A kind of database back-up data restoration methods and device | |
| CN107766521B (en) | Power grid planning data version control method | |
| CN112948359A (en) | Non-stop client data migration method based on database sub-base sub-table | |
| CN112328696A (en) | Method and device for synchronizing data in MySQL database to SQL Server database | |
| CN116450637A (en) | Data management method, device, electronic equipment and storage medium |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C14 | Grant of patent or utility model | ||
| GR01 | Patent grant | ||
| C17 | Cessation of patent right | ||
| CF01 | Termination of patent right due to non-payment of annual fee |
Granted publication date: 20120704 Termination date: 20121127 |