CN102063500A - Data migration method and device - Google Patents
Data migration method and device Download PDFInfo
- Publication number
- CN102063500A CN102063500A CN 201110000617 CN201110000617A CN102063500A CN 102063500 A CN102063500 A CN 102063500A CN 201110000617 CN201110000617 CN 201110000617 CN 201110000617 A CN201110000617 A CN 201110000617A CN 102063500 A CN102063500 A CN 102063500A
- Authority
- CN
- China
- Prior art keywords
- data
- database
- source
- target
- source data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a data migration method. The method comprises the following steps of: connecting a source database to acquire source data according with a preset condition, wherein the source data have field index information, and the preset condition is an extraction condition set aiming at the field index information; and connecting a target database to store the acquired source data as target data. The invention can improve the efficiency of data migration, particularly the efficiency of mass data migration and save the migration time.
Description
Technical field
The present invention relates to the technical field of data migtation, particularly relate to a kind of device of method and a kind of data migtation of data migtation.
Background technology
In the informatization process, original infosystem is constantly replaced by new system with better function.In the handoff procedure of new-old system, must face the problem of a data migration.The old system that has replaces from enabling by new system, has often accumulated the historical data of a large amount of preciousnesses between its operating period, and wherein many historical datas all are that new system enables necessary smoothly.Data migtation filters these historical datas, change exactly, and is loaded into the process in the new system.Data migtation is applicable to that mainly the old system of a cover switches to another set of new system, or overlaps old system when switching to same set of new system more, the historical data in the old system need be transformed into the situation in the new system.
Be subjected to current technical limitation, the speed of general data migration at present is desirable not enough, and the especially migration of mass data is for example moved to database massive data in another database, because the enormous amount of data, the migration of finishing total data may need the time of a couple of days even several weeks.In the prior art, generally finish, for example,, per 10,000 parts of user data are divided into one section, be divided into into 100 sections, move in batches at 1,000,000 parts of user data by batch operation repeatedly for the migration of mass data; Yet, in the mass data transition process, if the position of peek is very high, the access efficiency of database will reduce greatly, the inventor herein finds in practice, when since the 1st part of first peek of user data (promptly taking out 10,000 piece of data since the 1st piece of data), consuming timely is about 5 minutes; And, consuming timely be about 15 minutes when when the 50th part of user data takes out 10,000 piece of data.
Therefore, need the urgent technical matters that solves of those skilled in the art to be exactly at present: how can propose a kind of data migtation mechanism, in order to improve data migtation, especially the efficient of mass data migration is saved transit time with innovating.
Summary of the invention
Technical matters to be solved by this invention provides a kind of device of method and a kind of data migtation of data migtation, and in order to improve data migtation, especially the efficient of mass data migration is saved transit time.
In order to solve the problems of the technologies described above, the embodiment of the invention discloses a kind of method of data migtation, comprising:
Connect source database and obtain and meet pre-conditioned source data, described source data has field index information, described pre-conditioned be extraction conditions at the field index information setting;
The linking objective database saves as target data with the source data of being obtained.
Preferably, described source database and target database are arranged in different server, and described method also comprises:
Before migration beginning, source database is copied to the server at target database place, and source database and target database are configured on the same database port.
Preferably, described method also comprises:
Used field index information is write buffer memory.
Preferably, described method also comprises:
Before source data is deposited target database, described source data is converted to the desired form of target data.
Preferably, the described step that source data is converted to the desired form of target data comprises:
Data structure according to target data is changed source data;
If the numerical value implication changes, then the numerical value of target data is revised;
Title according to target data is upgraded source data.
The embodiment of the invention also discloses a kind of device of data migtation, comprising:
The source data extraction module is used to connect source database and obtains and meet pre-conditioned source data, and described source data has field index information, described pre-conditioned be extraction conditions at the field index information setting;
The target data memory module is used for the linking objective database source data of being obtained is saved as target data.
Preferably, described source database and target database are arranged in different server, and described device also comprises:
The source data backup module is used for before migration beginning, and source database is copied to the server at target database place, and source database and target database are configured on the same database port.
Preferably, described device also comprises:
Cache module is used for used field index information is write buffer memory.
Preferably, described device also comprises:
Modular converter was used for before source data is deposited target database, and described source data is converted to the desired form of target data.
Preferably, described modular converter comprises:
The format conversion submodule is used for according to the data structure of target data source data being changed;
Numerical value correction submodule is used for when the numerical value implication changes the numerical value of target data being revised;
The title updating submodule is used for according to the title of target data source data being upgraded.
Compared with prior art, the present invention has the following advantages:
The present invention is by analyzing the source data feature, the data of on the field of building index, moving according to current batch of needs of pre-conditioned extraction, and extract not according to Data Position, the inefficiency problem when thereby the position of avoiding peeking is high, effectively improve efficiency of data migration, saved the time of data migtation.
Can also be before migration beginning in the preferred embodiments of the present invention, earlier with on the whole server that copies to the target database place of source database, and be configured on the same database port (example).When operation, just can utilize the data in the new and old database of the direct read operation of SQL, thereby further improve data migtation efficient, save the time of data migtation.
Used field index information can also be write buffer memory in the preferred embodiments of the present invention, can read at a high speed when use next time like this, reduce database manipulation.Thereby further improve data migtation efficient, save the data migtation time.
Description of drawings
Fig. 1 is the process flow diagram of a kind of data migration method embodiment 1 of the present invention;
Fig. 2 is the process flow diagram of a kind of data migration method embodiment 2 of the present invention;
Fig. 3 is the process flow diagram of a kind of data migration method embodiment 3 of the present invention;
Fig. 4 is the structured flowchart of a kind of data migration device embodiment of the present invention.
Embodiment
For above-mentioned purpose of the present invention, feature and advantage can be become apparent more, the present invention is further detailed explanation below in conjunction with the drawings and specific embodiments.
With reference to figure 1, show the process flow diagram of the method embodiment 1 of a kind of data migtation of the present invention, specifically can may further comprise the steps:
The embodiment of the invention is by analyzing the source data feature, the data of on the field of building index, moving according to current batch of needs of pre-conditioned extraction, and extract not according to Data Position, the inefficiency problem when thereby the position of avoiding peeking is high, effectively improve efficiency of data migration, saved the time of data migtation.
In specific implementation, can use SQL to obtain data from source database, be well known that SQL is senior deproceduring programming language, allow the user on the high level data structure, to work.It does not require that the user specifies the deposit method to data, does not need the user to understand concrete deposit data mode yet, so have the disparate databases system of complete different fabrics, can use identical sql like language as the interface of data input with management.It with set of records ends as operand, all SQL statement are accepted set as input, return set as output, this aggregating characteristic allows the input of the output of a SQL statement as another SQL statement, so SQL statement can be nested, this makes him have great dirigibility and strong functions.Particularly, SQL can carry out inquiry by data base-oriented; SQL can fetch data from database; SQL can insert new record in database; Data in the SQL updatable data storehouse; SQL can be from the database deletion record; SQL can create new database; SQL can create new table in database; SQL can create storing process in database; SQL can create view in database; SQL can be provided with the authority of table, storing process and view.SQL can with the database program collaborative work, such as MS Access, DB2, Informix, MS SQL Server, Oracle, Sybase and other Database Systems.
For example, can adopt following SQL code to obtain the current batch of data that need migration:
SELECT?table.column...FROM?table?WHERE?uid＞200000?AND?uid＜210000；
Wherein, the extraction conditions that promptly is provided with of WHERE uid＞200000 AND uid＜210000 at field index information uid.
Use the embodiment of the invention, after tested, get 5 minutes consuming time of data in 0～10000 interval, getting in 1000000～1010000 intervals data consuming time is 5 minutes, and property retention is constant.
With reference to figure 2, show the process flow diagram of the method embodiment 2 of a kind of data migtation of the present invention, specifically can may further comprise the steps:
In practice, source database and target database is not usually in same server.For migration data, need to connect the source database reading of data, and then the linking objective database deposits data in.Under the very big situation of data volume, frequently connect two databases, also will influence and carry out efficient.Be head it off, present embodiment earlier with on the whole server that copies to the target database place of source database, and is configured on the same database port (example) before migration beginning.When operation, just can utilize the data in the new and old database of the direct read operation of SQL, thereby raise the efficiency.
For example, when source database and target database are not on same server, by following SQL code execution data migration operation:
$tmpdata＝$db_old->fetchAll(‘SELECT*FROM?table1’)；
$db_new->insert(‘table2’，$tmpdata)；
When source database and target database are configured to same port, then adopt following SQL code execution data migration operation:
$db_both->query(‘INSERT?INTO?db_new.table2?SELECT*FROM?db_old.table1’)；
Adopt present embodiment will further improve efficiency of data migration, save the time of data migtation.
With reference to figure 3, show the process flow diagram of the method embodiment 3 of a kind of data migtation of the present invention, specifically can may further comprise the steps:
Step 301, source database is copied to the server at target database place, and source database and target database are configured on the same database port;
Step 302, connect source database and obtain and meet pre-conditioned source data, described source data has field index information, described pre-conditioned be extraction conditions at the field index information setting;
Step 303, described source data is converted to the desired form of target data.
Step 304, linking objective database save as target data with described source data.
In a preferred embodiment of the present invention, described source data is converted to the step 303 of the desired form of target data, specifically can comprise following substep:
Substep S1, source data is changed according to the data structure of target data;
Substep S2, if the numerical value implication change, then the numerical value of target data is revised;
Substep S3, source data is upgraded according to the title of target data.
In practice, new system and old system often exist inconsistent on data layout, need carry out migration work.Transition process reads out data for need move the data item of commentaries on classics at each from old storehouse, be treated as the form of new edition, stores into then in the database of new edition and goes.For example, data item can comprise: user's name, user's head portrait, friend information, information, multidate information, user comment, blog, photograph album etc.
The variation that new legacy data need solve when migration has aspect three: structural change, numerical value change, title change.
The first, data structure changes:
The structure of partial data storage is different with old system in the new system, must rearrange after the legacy data derivation, carries out typing according to the form of new system.
For example:
" good friend's message " module is preserved the message that sends mutually between user and the user.
Former form is: the message of transmission is kept in the transmission table as a record, the message of reception in receiving table as a record.
Analyze and find: sending and receiving is one in fact, at different objects different implications is arranged.So, merge into one in the new construction, satisfy the function application by increasing receiving flag and sending the field that indicates.
The second, numerical value change:
Data dictionary is not quite identical with old system in the new system, changes for the numerical value implication, need carry out data correction when moving.
For example:
" good friend's relation " module, the definition in old system: numerical value 6-colleague; Numerical value 7-classmate ...
In new system: numerical value 2-colleague; Numerical value 5-classmate;
So, need when migration, numerical value 6 be modified to 2, be modified to 5 with 7.
Three, title changes:
The tables of data of new system is a brand-new design, so a large amount of table names, row name are different with old system.
For example:
" user's blog " module, former data storage will be stored in the blogid field of uc_blog table when migrating to new system in the id field of blog table.
For example, carry out data migtation by following SQL code:
′blog′＝>array(
′id′＝>array(′table′＝>′uh_blog′，′field′＝>′blogid′，′table2′
＝>′uh_blogfield′，′field2′＝>′blogid′)，
Above-mentioned coding implication is:
Source data is: the numerical value of id field in the old system blog table
Destination address is: blogid field in the new uh_blog of the system table
Data to be put into simultaneously: the new uh_blogfield of system table blogid field.
In specific implementation, in order to guarantee the data consistency of new platform and legacy platform, in the process of data migtation, can use PHP program chains source database that each field is carried out individual processing, conversion, the data after will changing then write target database; And after writing target database, data do consistency check again.If find that certain bar data does not exist or imperfect (whether interrupting suddenly judging), then move the data (migration data also can once move one or several clauses and subclauses in batches) of current clauses and subclauses again by migrator in migration daily record and the transition process.The migration of incremental data promptly is that the last clauses and subclauses after the migration for the first time add one and begin reading of data, and moves.
In transition process, there are some data in a plurality of migration links, to use, for example: user's ID and user name.Go database to get during each the use, will also can have influence on efficiency of data migration.At this problem, the embodiment of the invention can also may further comprise the steps:
Used field index information is write buffer memory.
For example leave among the memcached, can at a high speed read when using next time, reduces database manipulation.Thereby further improve data migtation efficient, save the data migtation time.
For making those skilled in the art understand the present invention better, below migrate to the example that is operating as of new edition space blog photograph album with certain old edition space, website blog album data, the present invention is illustrated further.
Use the present invention, in transition process, need to ensure the integrality of migration data and transit time in tolerable time range, and reduce the human intervention number of times in the transition process, realize transition process steadily and efficient.In addition, the data of new old space product all are to be stored in the Mysql database.The operation (as: read, write) of data is all needed to carry out SQL query to be finished.
One, preliminary work:
1, DB server account number, password, IP, port, access rights, preparation setting on the line;
2, front end IP on the local binding line, the HOST binding.
3, implementing procedure is placed on the tools catalogue.Under the tools catalogue all implementing procedure and tabulations of control migrated file.
4, the WEB program that automatically performs:
{{
4.1, user data (user)--user's master data, user good friend relation
4.2, blog article data (blog)--user's blog article data.Comprise content details, TAG
And grouped data
4.3, photo migration (photo)--comprise photo essential information and photo access
The location
4.4, album data (album)--photograph album essential information
4.5, (friend)--the good friend concerns arrangement for good friend relation
4.6, blog article classification (blogcate)--the arrangement of blog article grouped data, write
4.7, statistics (stat)--corresponding statistics arrangement, upgrade
4.8, blog article top set (blogtop)--put old blog article top set data in order, new data is deposited
Store up into the serializing form
4.9, above chapter, next chapter blog article (blognear)--upgrade the value in the above chapter next chapter field of blog article automatically
4.10, the arrangement of user's head portrait migration (avatar)--converting users head portrait data, again
Store on the new server
}}
5, use browser to carry out the migration of data;
6, all migration daily records all are stored in the LOG catalogue of migration catalogue.
Two, begin migration
1, all legacy datas that need move backup to an independent place earlier, then, re-use Backup Data and carry out migration work.Incremental data is carried out after the first pass migration finishes.
2, beginning is copied to the uchome program on the server of wanting the arrangement new procedures earlier.Guarantee that uchome/yoka_upgrade exists.
3, binding outer net test I P;
4, whether configuration file was correct under the inspection configuration file must be checked before migration of programs:
The UCHOME of 5, migration acquiescence and the database structure of UCENTER
{{
5.1, enter the docs catalogue.cd?docs
5.2, import the ucenter structure of data table.
5.3, import the data structure of uchome table.
}}
6, the data list structure and the data thereof in the former blog of migration storehouse
{{
6.1, the address replaced with the IP of blog database place, former space server.
{
6.1.1, create new directory: mkdir backdata
6.1.2, enter catalogue: cd backdata
}
6.2, the SQL statement below carrying out backs up these list structures and data
6.3, put these Backup Datas cat*.sql＞allblogtable.sql in order
6.4, Backup Data is imported to the inside, new storehouse.
{
6.4.1, import the data list structure and the data mysql-h 59.151.9.117 of blog
-uroot-p-B?up_uh＜allblogtable.sql
6.4.2, the change table name claim (carrying out following statement)
7, other operation
{
" space " made into " blog "
}
8, carry out the removal indexing means
9, carry out data migtation by the WEB mode
{{
9.1, synchronizing user data
9.2, synchronous blog article
9.3, blog article classification synchronously
9.4, synchronous photograph album
9.5, synchronous photo
9.6, rearrange good friend's relation
9.7, again the statistics
9.8, revise the data of user's link module.
9.9, user's head portrait
10, the operation of conversion head figure
11, upgrade the user's of title associative operation:
11.1, the locking user's unavailable (renewal)
11.2, blocked user RSS can not upgrade
12, after program is reached the standard grade, upgrade the interface of the original JS that transfers
13, carry out the recovery indexing means
14, in database, carry out SQL statement migration comment message data
15, the data of parallel migration
16, the migration work of yk_blog_set tables of data
17, remaining operation
{{
17.1, upgrade the domain statement, domain data originally are at yk_blog_set
In the table
17.2, upgrade good friend's statistical number
17.3, increase index
17.4, in message, add new table.
17.5, revise a figure field type
Need to prove, for aforesaid each method embodiment, for simple description, so it all is expressed as a series of combination of actions, but those skilled in the art should know, the present invention is not subjected to the restriction of described sequence of movement, because according to the present invention, some step can adopt other orders or carry out simultaneously.Secondly, those skilled in the art also should know, the embodiment described in the instructions all belongs to preferred embodiment, and related action and module might not be that the present invention is necessary.
With reference to figure 4, show the structured flowchart of the device embodiment of a kind of data migtation of the present invention, specifically can comprise:
Source data extraction module 401 is used to connect source database and obtains and meet pre-conditioned source data, and described source data has field index information, described pre-conditioned be extraction conditions at the field index information setting;
Target data memory module 402 is used for the linking objective database source data of being obtained is saved as target data.
The embodiment of the invention is by analyzing the source data feature, the data of on the field of building index, moving according to current batch of needs of pre-conditioned extraction, and extract not according to Data Position, the inefficiency problem when thereby the position of avoiding peeking is high, effectively improve efficiency of data migration, saved the time of data migtation.
In a preferred embodiment of the present invention, described source database and target database are arranged in different server, and described device can also comprise:
Source data backup module 100 is used for before migration beginning, and source database is copied to the server at target database place, and source database and target database are configured on the same database port.
Use present embodiment, can utilize the data in the new and old database of the direct read operation of SQL, thereby raise the efficiency.
In a preferred embodiment of the present invention, described data migration device can also comprise:
Modular converter 103 was used for before source data is deposited target database, and described source data is converted to the desired form of target data.
As a kind of example of concrete application, described modular converter can comprise following submodule:
The format conversion submodule is used for according to the data structure of target data source data being changed;
Numerical value correction submodule is used for when the numerical value implication changes the numerical value of target data being revised;
The title updating submodule is used for according to the title of target data source data being upgraded.
In specific implementation, described device is characterized in that, also comprises:
Cache module is used for used field index information is write buffer memory.
Use present embodiment and can reduce database manipulation.Thereby further improve data migtation efficient, save the data migtation time.
Because present embodiment is substantially corresponding to aforesaid method embodiment, so not detailed part in the description of present embodiment can just not given unnecessary details at this referring to the related description in the previous embodiment.
Each embodiment in this instructions all adopts the mode of going forward one by one to describe, and what each embodiment stressed all is and the difference of other embodiment that identical similar part is mutually referring to getting final product between each embodiment.
The present invention can be used in numerous general or special purpose computingasystem environment or the configuration.For example: multicomputer system, server, network PC, small-size computer, mainframe computer, comprise distributed computing environment of above any system or equipment or the like.
The present invention can describe in the general context of the computer executable instructions of being carried out by computing machine, for example program module.Usually, program module comprises the routine carrying out particular task or realize particular abstract, program, object, assembly, data structure or the like.Also can in distributed computing environment, put into practice the present invention, in these distributed computing environment, by by communication network connected teleprocessing equipment execute the task.In distributed computing environment, program module can be arranged in the local and remote computer-readable storage medium that comprises memory device.
More than the method for a kind of data migtation provided by the present invention and a kind of device of data migtation are described in detail, used specific case herein principle of the present invention and embodiment are set forth, the explanation of above embodiment just is used for helping to understand method of the present invention and core concept thereof; Simultaneously, for one of ordinary skill in the art, according to thought of the present invention, the part that all can change in specific embodiments and applications, in sum, this description should not be construed as limitation of the present invention.
Claims (10)
1. the method for a data migtation is characterized in that, comprising:
Connect source database and obtain and meet pre-conditioned source data, described source data has field index information, described pre-conditioned be extraction conditions at the field index information setting;
The linking objective database saves as target data with the source data of being obtained.
2. the method for claim 1 is characterized in that, described source database and target database are arranged in different server, and described method also comprises:
Before migration beginning, source database is copied to the server at target database place, and source database and target database are configured on the same database port.
3. method as claimed in claim 1 or 2 is characterized in that, also comprises:
Used field index information is write buffer memory.
4. method as claimed in claim 1 or 2 is characterized in that, also comprises:
Before source data is deposited target database, described source data is converted to the desired form of target data.
5. method as claimed in claim 4 is characterized in that, the described step that source data is converted to the desired form of target data comprises:
Data structure according to target data is changed source data;
If the numerical value implication changes, then the numerical value of target data is revised;
Title according to target data is upgraded source data.
6. the device of a data migtation is characterized in that, comprising:
The source data extraction module is used to connect source database and obtains and meet pre-conditioned source data, and described source data has field index information, described pre-conditioned be extraction conditions at the field index information setting;
The target data memory module is used for the linking objective database source data of being obtained is saved as target data.
7. device as claimed in claim 6 is characterized in that described source database and target database are arranged in different server, and described device also comprises:
The source data backup module is used for before migration beginning, and source database is copied to the server at target database place, and source database and target database are configured on the same database port.
8. as claim 6 or 7 described devices, it is characterized in that, also comprise:
Cache module is used for used field index information is write buffer memory.
9. as claim 6 or 7 described devices, it is characterized in that, also comprise:
Modular converter was used for before source data is deposited target database, and described source data is converted to the desired form of target data.
10. device as claimed in claim 9 is characterized in that, described modular converter comprises:
The format conversion submodule is used for according to the data structure of target data source data being changed;
Numerical value correction submodule is used for when the numerical value implication changes the numerical value of target data being revised;
The title updating submodule is used for according to the title of target data source data being upgraded.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110000617 CN102063500A (en) | 2011-01-04 | 2011-01-04 | Data migration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN 201110000617 CN102063500A (en) | 2011-01-04 | 2011-01-04 | Data migration method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102063500A true CN102063500A (en) | 2011-05-18 |
Family
ID=43998775
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN 201110000617 Pending CN102063500A (en) | 2011-01-04 | 2011-01-04 | Data migration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102063500A (en) |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN102393860A (en) * | 2011-12-06 | 2012-03-28 | 方正国际软件有限公司 | Database data migration system |
| CN103765817A (en) * | 2011-08-30 | 2014-04-30 | 国际商业机器公司 | Replication of data objects from a source server to a target server |
| CN103914458A (en) * | 2012-12-29 | 2014-07-09 | 中国移动通信集团河北有限公司 | Mass data migration method and device |
| CN104021123A (en) * | 2013-02-28 | 2014-09-03 | 国际商业机器公司 | Method and system for data transfer |
| CN104077401A (en) * | 2014-07-04 | 2014-10-01 | 用友软件股份有限公司 | Database data migration device and method |
| CN104252502A (en) * | 2013-06-29 | 2014-12-31 | 北京新媒传信科技有限公司 | Method and device for carrying out data migration on database management platform |
| WO2015014152A1 (en) * | 2013-07-31 | 2015-02-05 | 华为技术有限公司 | Method and device for accessing shared memory |
| CN104424325A (en) * | 2013-09-10 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Data inquiry method and device |
| CN104539431A (en) * | 2014-12-31 | 2015-04-22 | 上海格尔软件股份有限公司 | Method for smoothly migrating and switching users between user systems online |
| WO2015074427A1 (en) * | 2013-11-25 | 2015-05-28 | Tencent Technology (Shenzhen) Company Limited | Systems and methods for data migration |
| CN104750729A (en) * | 2013-12-30 | 2015-07-01 | 中国移动通信集团公司 | Data management method and system based on journal file |
| CN105095518A (en) * | 2015-09-18 | 2015-11-25 | 中国农业银行股份有限公司 | Data conversion method and device |
| CN105354314A (en) * | 2015-11-10 | 2016-02-24 | 中国建设银行股份有限公司 | Data migration method and device |
| CN105389404A (en) * | 2015-12-29 | 2016-03-09 | 北京斗牛科技有限公司 | Method and device for importing data into database association table |
| CN105447172A (en) * | 2015-12-07 | 2016-03-30 | 北京先进数通信息技术股份公司 | Data processing method and system under Hadoop platform |
| CN105740411A (en) * | 2016-01-30 | 2016-07-06 | 武汉大学 | SOA (Service-Oriented Architecture) and WebService based data migration method |
| CN105808675A (en) * | 2016-03-01 | 2016-07-27 | 万达信息股份有限公司 | Method for migrating from Informix database to KingbaseES |
| CN106407321A (en) * | 2016-08-31 | 2017-02-15 | 东软集团股份有限公司 | Data synchronization method and device |
| CN106598979A (en) * | 2015-10-15 | 2017-04-26 | 北京国双科技有限公司 | Data writing method and device |
| CN106909853A (en) * | 2015-12-22 | 2017-06-30 | 北京奇虎科技有限公司 | One kind operation performs method and device |
| CN107301214A (en) * | 2017-06-09 | 2017-10-27 | 广州虎牙信息科技有限公司 | Data migration method, device and terminal device in HIVE |
| CN108153493A (en) * | 2017-12-25 | 2018-06-12 | 中铁程科技有限责任公司 | Information processing method and system, computer readable storage medium |
| CN108446315A (en) * | 2018-02-07 | 2018-08-24 | 中国平安人寿保险股份有限公司 | Big data moving method, device, equipment and storage medium |
| CN111258990A (en) * | 2020-02-17 | 2020-06-09 | 同盾控股有限公司 | Index database data migration method, device, equipment and storage medium |
| US10725708B2 (en) | 2015-07-31 | 2020-07-28 | International Business Machines Corporation | Replication of versions of an object from a source storage to a target storage |
| CN112817930A (en) * | 2019-11-15 | 2021-05-18 | 北京沃东天骏信息技术有限公司 | Data migration method and device |
Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1561496A (en) * | 2001-09-28 | 2005-01-05 | 甲骨文国际公司 | An efficient index structure to access hierarchical data in a relational database system |
-
2011
- 2011-01-04 CN CN 201110000617 patent/CN102063500A/en active Pending
Patent Citations (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN1561496A (en) * | 2001-09-28 | 2005-01-05 | 甲骨文国际公司 | An efficient index structure to access hierarchical data in a relational database system |
Non-Patent Citations (2)
| Title |
|---|
| 《SQL Server 索引结构及其使用（一）》 20100917 freedk SQL Server 索引结构及其使用（一） http://www.vckbase.com/document/viewdoc/?id=1307 全文 1-10 , 1 * |
| 《SQL 语句实现两个数据库表直接操作》 20100121 王培 SQL 语句实现两个数据库表直接操作 http://www.cnblogs.com/wangpei/archive/2010/01/21/1653395.html 全文 1-10 , 1 * |
Cited By (47)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US10664492B2 (en) | 2011-08-30 | 2020-05-26 | International Business Machines Corporation | Replication of data objects from a source server to a target server |
| CN103765817A (en) * | 2011-08-30 | 2014-04-30 | 国际商业机器公司 | Replication of data objects from a source server to a target server |
| US10664493B2 (en) | 2011-08-30 | 2020-05-26 | International Business Machines Corporation | Replication of data objects from a source server to a target server |
| CN103765817B (en) * | 2011-08-30 | 2017-01-25 | 国际商业机器公司 | System and method for replicating data objects from a source server to a target server |
| US9910904B2 (en) | 2011-08-30 | 2018-03-06 | International Business Machines Corporation | Replication of data objects from a source server to a target server |
| US9904717B2 (en) | 2011-08-30 | 2018-02-27 | International Business Machines Corporation | Replication of data objects from a source server to a target server |
| CN102393860A (en) * | 2011-12-06 | 2012-03-28 | 方正国际软件有限公司 | Database data migration system |
| CN103914458A (en) * | 2012-12-29 | 2014-07-09 | 中国移动通信集团河北有限公司 | Mass data migration method and device |
| CN104021123A (en) * | 2013-02-28 | 2014-09-03 | 国际商业机器公司 | Method and system for data transfer |
| CN104021123B (en) * | 2013-02-28 | 2017-10-31 | 国际商业机器公司 | method and system for data migration |
| US9619492B2 (en) | 2013-02-28 | 2017-04-11 | International Business Machines Corporation | Data migration |
| CN104252502A (en) * | 2013-06-29 | 2014-12-31 | 北京新媒传信科技有限公司 | Method and device for carrying out data migration on database management platform |
| CN104252502B (en) * | 2013-06-29 | 2017-12-01 | 北京新媒传信科技有限公司 | Data migration method and device in a kind of database management platform |
| CN104348862B (en) * | 2013-07-31 | 2018-03-16 | 华为技术有限公司 | Data Migration processing method, apparatus and system |
| CN104348862A (en) * | 2013-07-31 | 2015-02-11 | 华为技术有限公司 | Data migration processing method, apparatus, and system |
| WO2015014152A1 (en) * | 2013-07-31 | 2015-02-05 | 华为技术有限公司 | Method and device for accessing shared memory |
| CN104424325A (en) * | 2013-09-10 | 2015-03-18 | 阿里巴巴集团控股有限公司 | Data inquiry method and device |
| WO2015074427A1 (en) * | 2013-11-25 | 2015-05-28 | Tencent Technology (Shenzhen) Company Limited | Systems and methods for data migration |
| US9661109B2 (en) | 2013-11-25 | 2017-05-23 | Tencent Technology (Shenzhen) Company Limited | Systems and methods for data migration |
| CN104750729A (en) * | 2013-12-30 | 2015-07-01 | 中国移动通信集团公司 | Data management method and system based on journal file |
| CN104750729B (en) * | 2013-12-30 | 2018-08-28 | 中国移动通信集团公司 | A kind of data managing method and data management system based on journal file |
| CN104077401B (en) * | 2014-07-04 | 2017-11-24 | 用友网络科技股份有限公司 | Data migration device and data migration method for database |
| CN104077401A (en) * | 2014-07-04 | 2014-10-01 | 用友软件股份有限公司 | Database data migration device and method |
| CN104539431B (en) * | 2014-12-31 | 2018-08-17 | 上海格尔软件股份有限公司 | A kind of method of online user's system switching user's smooth migration |
| CN104539431A (en) * | 2014-12-31 | 2015-04-22 | 上海格尔软件股份有限公司 | Method for smoothly migrating and switching users between user systems online |
| US10725708B2 (en) | 2015-07-31 | 2020-07-28 | International Business Machines Corporation | Replication of versions of an object from a source storage to a target storage |
| CN105095518A (en) * | 2015-09-18 | 2015-11-25 | 中国农业银行股份有限公司 | Data conversion method and device |
| CN105095518B (en) * | 2015-09-18 | 2018-10-16 | 中国农业银行股份有限公司 | A kind of data transfer device and device |
| CN106598979A (en) * | 2015-10-15 | 2017-04-26 | 北京国双科技有限公司 | Data writing method and device |
| CN105354314A (en) * | 2015-11-10 | 2016-02-24 | 中国建设银行股份有限公司 | Data migration method and device |
| CN105354314B (en) * | 2015-11-10 | 2020-03-03 | 中国建设银行股份有限公司 | Data migration method and device |
| CN105447172A (en) * | 2015-12-07 | 2016-03-30 | 北京先进数通信息技术股份公司 | Data processing method and system under Hadoop platform |
| CN106909853A (en) * | 2015-12-22 | 2017-06-30 | 北京奇虎科技有限公司 | One kind operation performs method and device |
| CN105389404A (en) * | 2015-12-29 | 2016-03-09 | 北京斗牛科技有限公司 | Method and device for importing data into database association table |
| CN105389404B (en) * | 2015-12-29 | 2019-04-16 | 北京斗牛科技有限公司 | A kind of method and apparatus importing data to database association table |
| CN105740411A (en) * | 2016-01-30 | 2016-07-06 | 武汉大学 | SOA (Service-Oriented Architecture) and WebService based data migration method |
| CN105740411B (en) * | 2016-01-30 | 2018-11-13 | 武汉大学 | A kind of data migration method based on SOA and WebService |
| CN105808675A (en) * | 2016-03-01 | 2016-07-27 | 万达信息股份有限公司 | Method for migrating from Informix database to KingbaseES |
| CN105808675B (en) * | 2016-03-01 | 2020-04-28 | 万达信息股份有限公司 | Method for migrating from Informix database to Renminburgh database |
| CN106407321A (en) * | 2016-08-31 | 2017-02-15 | 东软集团股份有限公司 | Data synchronization method and device |
| CN107301214A (en) * | 2017-06-09 | 2017-10-27 | 广州虎牙信息科技有限公司 | Data migration method, device and terminal device in HIVE |
| CN107301214B (en) * | 2017-06-09 | 2020-08-28 | 广州虎牙信息科技有限公司 | Data migration method and device in HIVE and terminal equipment |
| CN108153493A (en) * | 2017-12-25 | 2018-06-12 | 中铁程科技有限责任公司 | Information processing method and system, computer readable storage medium |
| CN108446315A (en) * | 2018-02-07 | 2018-08-24 | 中国平安人寿保险股份有限公司 | Big data moving method, device, equipment and storage medium |
| CN112817930A (en) * | 2019-11-15 | 2021-05-18 | 北京沃东天骏信息技术有限公司 | Data migration method and device |
| CN111258990A (en) * | 2020-02-17 | 2020-06-09 | 同盾控股有限公司 | Index database data migration method, device, equipment and storage medium |
| CN111258990B (en) * | 2020-02-17 | 2023-04-07 | 同盾控股有限公司 | Index database data migration method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102063500A (en) | Data migration method and device | |
| US10936562B2 (en) | Type-specific compression in database systems | |
| US10509785B2 (en) | Policy-driven data manipulation in time-series database systems | |
| CN110196871B (en) | Data warehousing method and system | |
| EP3238106B1 (en) | Compaction policy | |
| CN105630864B (en) | Forced ordering of a dictionary storing row identifier values | |
| US20120150797A1 (en) | Method and system for safely transporting legacy data to an object semantic form data grid | |
| Lin et al. | Full-text indexing for optimizing selection operations in large-scale data analytics | |
| CN111324610A (en) | Data synchronization method and device | |
| CN104809237A (en) | LSM-tree (The Log-Structured Merge-Tree) index optimization method and LSM-tree index optimization system | |
| CN105144080A (en) | System for metadata management | |
| CN103617176A (en) | Method for achieving automatic synchronization of multi-source heterogeneous data resources | |
| US11314719B2 (en) | Method for implementing change data capture in database management system | |
| EP3271840A1 (en) | Mutations in a column store | |
| CN104021145A (en) | Mixed service concurrent access method and device | |
| US11741084B2 (en) | High frequency data management (HFDM) | |
| CN111651519B (en) | Data synchronization method, data synchronization device, electronic equipment and storage medium | |
| CN112214453B (en) | Large-scale industrial data compression storage method, system and medium | |
| WO2015168988A1 (en) | Data index creation method and device, and computer storage medium | |
| Kvet | Relational data index consolidation | |
| CN112965939A (en) | File merging method, device and equipment | |
| US11669538B1 (en) | Systems, devices, and methods for data analytics | |
| Belov et al. | Analysis of Big Data Storage Tools for Data Lakes based on Apache Hadoop Platform | |
| Petrov | Algorithms behind modern storage systems | |
| CN113672556A (en) | Batch file migration method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20110518 |