CN103914458A - Mass data migration method and device - Google Patents
Mass data migration method and device Download PDFInfo
- Publication number
- CN103914458A CN103914458A CN201210592673.1A CN201210592673A CN103914458A CN 103914458 A CN103914458 A CN 103914458A CN 201210592673 A CN201210592673 A CN 201210592673A CN 103914458 A CN103914458 A CN 103914458A
- Authority
- CN
- China
- Prior art keywords
- migration
- data
- allocation list
- module
- data type
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
Landscapes
- Engineering & Computer Science (AREA)
- Databases & Information Systems (AREA)
- Theoretical Computer Science (AREA)
- Data Mining & Analysis (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention discloses a mass data migration method and device. The method comprises the following steps of establishing a migration configuration table; determining the data migrating type according to whether mass data change in a preset time period; according to the data migrating type, correcting the migration configuration table; migrating the mass data according to the migration configuration table. The mass data migration method can greatly reduce data migration time, service interruption time and the requirements on the varieties of databases.
Description
Technical field
The present invention relates to database technical field, relate in particular to a kind of method and device of mass data migration.
Background technology
Along with business development is carried out change, the platform conversion of dilatation, system architecture etc. reason to the hardware of system and all can be needed the data of system to move.Wherein there are following three kinds for the technical scheme of the migration of mass data data:
One, the logic that usage data storehouse carries imports, derivation instrument, as the export of oracle, import.
Use this scheme to move, can realize cross-platform migration.In order to guarantee the consistance of migration data, business service must be interrupted, long for mass data, break period.Derive while importing and can use full storehouse, user model, or table schema.
Two, the physical backup instrument that usage data storehouse carries recovers, as the rman of oracle.
Use this scheme, Backup Data is at ordinary times recovered, in order to guarantee the consistance of migration data, business service must be interrupted, and break period is shorter.If source database is non-archive mode, this restoration methods cannot be used, because recovery is the data of full library backup time point.
Three, the bottom synchronization means that uses storage producer to provide.
Use this scheme, before migration, start real-time synchronization, during to cutover, after of short duration service disconnection, synchronously complete.
There is following defect in existing technology:
The service outage duration of scheme one is long.In order to guarantee the complete of migration data, first interrupting service, then starts data to use the logic instrument of deriving that data are derived, and then uses logic import tool to import data.Use the method, the time of Data Migration is the time of implementing backup to add the time of implementing recovery.
The service outage duration of scheme two will lack compared with scheme one, because directly recover from daily tape backup, the time of Data Migration is the time of implementing recovery.But this kind of method cannot be applied to cross-platform Data Migration, require source is archive mode simultaneously.
Scheme three service outage durations are the shortest, but the harshest to the requirement of source, destination, must be same platform, simultaneously require be that identical storage, database are identical data Data base structure.
Summary of the invention
In order to solve, mass data transit time in prior art is long, many for database type requirements, need the technical matters of long-time interrupting service when Data Migration, the present invention proposes a kind of method and device of mass data migration, can greatly reduce Data Migration time, service outage duration and the requirement for database kind.
One aspect of the present invention, a kind of method that provides mass data to move, comprises the following steps: create migration allocation list; Whether change and judge migration data type according to described mass data in preset time period; Revise described migration allocation list according to described migration data type; Move described mass data according to described migration allocation list.
Another aspect of the present invention, a kind of device of mass data migration is provided, comprises migration allocation list creation module, migration data type judging module, migration allocation list modified module and Data Migration module, wherein, described migration allocation list creation module, for creating migration allocation list; Whether described migration data type judging module, for changing and judge migration data type according to described mass data in preset time period; Described migration allocation list modified module, for revising described migration allocation list according to described migration data type; Described Data Migration module, for moving described mass data according to described migration allocation list.
Method and the device of mass data migration of the present invention, by to the data analysis of needs migrations Data classification, use migration allocation list to carry out corresponding Data Migration to various types of data, can support kinds of platform, greatly reduce service outage duration, make whole transition process controlled.
Accompanying drawing explanation
Fig. 1 is the process flow diagram of the method for embodiment of the present invention mass data migration;
Fig. 2 is the process flow diagram of the preferred mass data moving method of the embodiment of the present invention;
Fig. 3 is by the process flow diagram of dynamic data described in automatic processing scripts migration of programs in the embodiment of the present invention;
Fig. 4 is cutover Data Migration on that night schematic flow sheet in the embodiment of the present invention;
Fig. 5 is the structural representation of the device of embodiment of the present invention mass data migration.
Embodiment
Fig. 1 is the process flow diagram of the method for embodiment of the present invention mass data migration.As shown in Figure 1, comprise the following steps:
Step 101, establishment migration allocation list;
Step 102, whether change and judge migration data type according to described mass data in preset time period;
Step 103, revise described migration allocation list according to described migration data type;
Step 104, move described mass data according to described migration allocation list.
The data analysis of this embodiment to needs migration, is divided into the data of needs migration unchanged data, changes large table data, little table data frequently.Use allocation list to indicate and indicate current state to all kinds of tables, copy, still copy end.The internal table monitoring providing by large database, as the table monitoring function of ORACLE, DML number of operations that can monitoring form, analyzes and does not have vicissitudinous data nearly January, these type of data bit static migrating data.To these type of data, use export, the import instrument of oracle to import in advance, derive.
Large for data volume, to change frequently, have unique index or major key table, the mode of employing trigger, by delta data dynamic replication, these type of data are dynamic migration data, such data can be at not interruption status of business, dynamic migration.Except static data, dynamic data, remaining data are data that Data Migration cutover need to be moved that night, can shows the migration of level in order to realize cross-platform migration simultaneously, and export, the import instrument of use oracle import, derive.By query configuration table, cutover scanned again to static migrating data that night, to processing of changing.
Fig. 2 is a kind of process flow diagram of preferred mass data moving method, as shown in Figure 2, comprises the following steps:
Step 201, establishment migration allocation list;
Allocation list comprises field:
Step 202, whether change according to described mass data in preset time period;
Judge in the time cycle whether data change, the internal table monitoring that can use large database to provide, as the table monitoring function of ORACLE, DML number of operations that can monitoring form.
Step 203, in the time that in preset time period, described mass data does not change, described migration data type is static data, revises described migration allocation list, configuring static migration;
Carry out and revise migration allocation list, trans_step is made as to 0, configuring static migration.
Step 204, move described static data by default data base tool.
Carry out static data migration sub-process, distinguish by rep_flag, can play concurrent process migration (other sub-processes are also like this) by his-and-hers watches, only processing rep_flag=0 of the new process rising.
Step 205, in the time that in preset time period there is variation in described mass data, judge whether described migration allocation list has major key or unique index;
Step 206, in the time that described migration allocation list has major key or unique index, judge whether the size of described migration allocation list exceedes preset value;
Step 207, in the time that the size of described migration allocation list exceedes described preset value, described migration data type is dynamic large table data, revises described migration allocation list, configuration Dynamic date migration;
Carry out and revise migration allocation list, trans_step is made as to 1, configuration dynamic migration.
Step 208, by dynamic data described in automatic processing scripts migration of programs;
Step 209, in the time that described migration allocation list does not have the size of major key or unique index or described migration allocation list not exceed described preset value, described migration data type is dynamic little table data, revises described migration allocation list, configuration cutover Data Migration on that night;
Carry out and revise migration allocation list, trans_step is made as to 9, configuration cutover migration on that night.
Step 210, migration cutover migration data on that night.
In Dynamic date migration flow process in above-mentioned steps 208, the data of dynamic migration refer to those data volumes large, without time subregion and data variation object more frequently, record the data that change by trigger mechanism, by automatic processing scripts replicate.sh, the market demand changing is arrived to target database.The described step by dynamic data described in automatic processing scripts migration of programs as shown in Figure 3, comprising:
Step 301, set up trigger according to the described dynamically migration allocation list of large table data;
In trigger, record following information and arrive log sheet: 1) action type, 2) ROWID and 3) major key information, table based on dynamic migration is set up trigger, the data of trigger based on table increase, delete, change, record changes in log sheet, recorded information comprises action type, refer to increase, delete, change, the row ID of major key information, change.
Step 302, from source database copy data to target database;
From source database copy data to target database, logic that can usage data library tape derives, import tool.
Step 303, by automatic processing scripts program by the market demand changing to target database.
Automatically process shell program replicate.sh the market demand changing is arrived to target database.
As shown in Figure 4, be cutover Data Migration on that night schematic flow sheet, comprise the following steps:
Step 401, interrupting service;
Step 402, use logic derive, import tool migration data on that night;
Step 403, by the state of query configuration table, judge that night, whether data moved, and forwarded step 404 to if completed, otherwise forward step 402 to;
In step 404, query configuration table, be labeled as the table of static data, see whether it changes, if forward step 405 to, otherwise finish;
Step 405, use logic derive, import tool moves the static table that data variation occurs again.
The method of the magnanimity Platform-cross Data Migration that the embodiment of the present invention provides, classifies migration data, is divided into unchanged data, changes large table data, little table data frequently.And then mass data is divided into static data, dynamic data and cold migration data.By Data classification, cutover data on that night are greatly reduced, reduce service outage duration.Static data migration is not only from business level analysis, is increased, deletes, is changed by monitoring, supports from database aspect, uses in advance logic importing, derivation to carry out Data Migration.Dynamic date migration is by creating trigger, record the large table of major key increasing, delete, some other day will, then move for daily record his-and-hers watches, greatly reduced large table migration on that night required time.Move allocation list by foundation, transition process is controlled, can play concurrent process.That night static migrating data were scanned again by allocation list cutover, to processing of changing, guaranteed data integrity.In whole scheme, use general data base logic importing, derivation instrument to support cross operating system, storage platform, use the general SHELL of UNIX to be programmed into Mobile state datamation migration.
As shown in Figure 5, the embodiment of the present invention also provides a kind of device of mass data migration, comprise migration allocation list creation module 501, migration data type judging module 502, migration allocation list modified module 503 and Data Migration module 504, wherein, described migration allocation list creation module, for creating migration allocation list; Whether described migration data type judging module, for changing and judge migration data type according to described mass data in preset time period; Described migration allocation list modified module, for revising described migration allocation list according to described migration data type; Described Data Migration module, for moving described mass data according to described migration allocation list.
Preferably, described migration data type judging module is further used for: in the time that in preset time period, described mass data does not change, described migration data type is static data; In the time that variation has occurred described mass data in preset time period, judge whether described migration allocation list has major key or unique index; In the time that described migration allocation list has major key or unique index, judge whether the size of described migration allocation list exceedes preset value; In the time that the size of described migration allocation list exceedes described preset value, described migration data type is dynamically to show greatly data; In the time that described migration allocation list does not have the size of major key or unique index or described migration allocation list not exceed described preset value, described migration data type is dynamic little table data.
Described migration allocation list modified module is further used for: in the time that migration data type is static data, revise described migration allocation list, configuring static migration; Described Data Migration module is further used for: move described static data by default data base tool.
Described migration allocation list modified module is further used for: when migration data type is, while dynamically showing data greatly, to revise described migration allocation list, configuration Dynamic date migration; Described Data Migration module is further used for: by dynamic data described in automatic processing scripts migration of programs.
Described Data Migration module is further used for: set up trigger according to the described dynamically migration allocation list of large table data; From source database copy data to target database; By automatic processing scripts program, the market demand changing is arrived to target database.
Described migration allocation list modified module is further used for: in the time that migration data type is dynamic little table data, revise described migration allocation list, configuration cutover Data Migration on that night; Described Data Migration module is further used for: interrupting service; Use logic utility migration data on that night.
Described Data Migration module also for: inquire about the state of described migration allocation list, judge described that night, whether data moved; If completed, inquire about the table that is labeled as static data in described migration allocation list; If the table of described static data changes, use default data base tool to move the static table of described generation data variation.
The technical scheme of the embodiment of the present invention has versatility, supports kinds of platform, supports cross operating system, storage platform.In whole scheme, use general data base logic importing, derivation instrument, use the general SHELL of UNIX to be programmed into Mobile state datamation migration, greatly reduced service outage duration short.By migration data is classified, be divided into unchanged data, change large table data, little table data frequently.And then mass data is divided into static data, dynamic data and cold migration data.By Data classification, cutover data on that night are greatly reduced, reduce service outage duration, make whole transition process controlled, move allocation list monitoring migration progress by foundation, prevent that concurrent process from being to repeat migration.
It should be noted that: above embodiment is only unrestricted in order to the present invention to be described, the present invention is also not limited in above-mentioned giving an example, and all do not depart from technical scheme and the improvement thereof of the spirit and scope of the present invention, and it all should be encompassed in claim scope of the present invention.
Claims (14)
1. a method for mass data migration, comprises the following steps:
Create migration allocation list;
Whether change and judge migration data type according to described mass data in preset time period;
Revise described migration allocation list according to described migration data type;
Move described mass data according to described migration allocation list.
2. method according to claim 1, is characterized in that,
Describedly whether change and judge that the step of migration data type comprises according to described mass data in preset time period:
In the time that in preset time period, described mass data does not change, described migration data type is static data;
In the time that variation has occurred described mass data in preset time period, judge whether described migration allocation list has major key or unique index;
In the time that described migration allocation list has major key or unique index, judge whether the size of described migration allocation list exceedes preset value;
In the time that the size of described migration allocation list exceedes described preset value, described migration data type is dynamically to show greatly data;
In the time that described migration allocation list does not have the size of major key or unique index or described migration allocation list not exceed described preset value, described migration data type is dynamic little table data.
3. method according to claim 2, is characterized in that, the step of revising described migration allocation list according to described migration data type comprises:
In the time that migration data type is static data, revise described migration allocation list, configuring static migration;
The step of moving described mass data according to described migration allocation list comprises:
Move described static data by default data base tool.
4. method according to claim 2, is characterized in that, the step of revising described migration allocation list according to described migration data type comprises:
When migration data type is, while dynamically showing data greatly, to revise described migration allocation list, configuration Dynamic date migration;
The step of moving described mass data according to described migration allocation list comprises:
By dynamic data described in automatic processing scripts migration of programs.
5. method according to claim 4, is characterized in that, the described step by dynamic data described in automatic processing scripts migration of programs comprises:
Set up trigger according to the described dynamically migration allocation list of large table data;
From source database copy data to target database;
By automatic processing scripts program, the market demand changing is arrived to target database.
6. method according to claim 2, is characterized in that, the step of revising described migration allocation list according to described migration data type comprises:
In the time that migration data type is dynamic little table data, revise described migration allocation list, configuration cutover Data Migration on that night;
The step of moving described mass data according to described migration allocation list comprises:
Interrupting service;
Use logic utility migration data on that night.
7. method according to claim 6, is characterized in that, after the step of described use logic utility migration data on that night, also comprises:
Inquire about the state of described migration allocation list, judge described that night, whether data moved;
If completed, inquire about the table that is labeled as static data in described migration allocation list;
If the table of described static data changes, use default data base tool to move the static table of described generation data variation.
8. a device for mass data migration, comprises migration allocation list creation module, migration data type judging module, migration allocation list modified module and Data Migration module, wherein,
Described migration allocation list creation module, for creating migration allocation list;
Whether described migration data type judging module, for changing and judge migration data type according to described mass data in preset time period;
Described migration allocation list modified module, for revising described migration allocation list according to described migration data type;
Described Data Migration module, for moving described mass data according to described migration allocation list.
9. device according to claim 8, is characterized in that, described migration data type judging module is further used for:
In the time that in preset time period, described mass data does not change, described migration data type is static data;
In the time that variation has occurred described mass data in preset time period, judge whether described migration allocation list has major key or unique index;
In the time that described migration allocation list has major key or unique index, judge whether the size of described migration allocation list exceedes preset value;
In the time that the size of described migration allocation list exceedes described preset value, described migration data type is dynamically to show greatly data;
In the time that described migration allocation list does not have the size of major key or unique index or described migration allocation list not exceed described preset value, described migration data type is dynamic little table data.
10. device according to claim 9, is characterized in that, described migration allocation list modified module is further used for:
In the time that migration data type is static data, revise described migration allocation list, configuring static migration;
Described Data Migration module is further used for:
Move described static data by default data base tool.
11. devices according to claim 9, is characterized in that, described migration allocation list modified module is further used for:
When migration data type is, while dynamically showing data greatly, to revise described migration allocation list, configuration Dynamic date migration;
Described Data Migration module is further used for:
By dynamic data described in automatic processing scripts migration of programs.
12. devices according to claim 11, is characterized in that, described Data Migration module is further used for:
Set up trigger according to the described dynamically migration allocation list of large table data;
From source database copy data to target database;
By automatic processing scripts program, the market demand changing is arrived to target database.
13. devices according to claim 9, is characterized in that, described migration allocation list modified module is further used for:
In the time that migration data type is dynamic little table data, revise described migration allocation list, configuration cutover Data Migration on that night;
Described Data Migration module is further used for:
Interrupting service;
Use logic utility migration data on that night.
14. devices according to claim 13, is characterized in that, described Data Migration module also for:
Inquire about the state of described migration allocation list, judge described that night, whether data moved;
If completed, inquire about the table that is labeled as static data in described migration allocation list;
If the table of described static data changes, use default data base tool to move the static table of described generation data variation.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210592673.1A CN103914458A (en) | 2012-12-29 | 2012-12-29 | Mass data migration method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201210592673.1A CN103914458A (en) | 2012-12-29 | 2012-12-29 | Mass data migration method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN103914458A true CN103914458A (en) | 2014-07-09 |
Family
ID=51040152
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201210592673.1A Pending CN103914458A (en) | 2012-12-29 | 2012-12-29 | Mass data migration method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN103914458A (en) |
Cited By (15)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572867A (en) * | 2014-12-18 | 2015-04-29 | 上海新炬网络信息技术有限公司 | Data processing method based on Oracle |
| CN105868210A (en) * | 2015-01-21 | 2016-08-17 | 阿里巴巴集团控股有限公司 | Creating method and device of unique index in distributed database |
| CN106572172A (en) * | 2016-11-07 | 2017-04-19 | 湖北省农村信用社联合社网络信息中心 | Multi-process data migration method based on Hash algorithm |
| CN106649449A (en) * | 2016-09-22 | 2017-05-10 | 济南浪潮高新科技投资发展有限公司 | Internal and external network data exchange method based on SQLite |
| CN106909633A (en) * | 2017-01-23 | 2017-06-30 | 北京思特奇信息技术股份有限公司 | A kind of data by oracle database imported into the method and system of mongodb databases |
| CN107133228A (en) * | 2016-02-26 | 2017-09-05 | 华为技术有限公司 | A kind of method and device of fast resampling |
| WO2018036235A1 (en) * | 2016-08-22 | 2018-03-01 | 中兴通讯股份有限公司 | Solr data migration method and apparatus |
| CN108153806A (en) * | 2016-11-30 | 2018-06-12 | Ncr公司 | Data Migration |
| CN108446315A (en) * | 2018-02-07 | 2018-08-24 | 中国平安人寿保险股份有限公司 | Big data moving method, device, equipment and storage medium |
| CN109635033A (en) * | 2018-12-27 | 2019-04-16 | 贵州省广播电视信息网络股份有限公司 | A kind of processing of millions data on stock, log collection and the method for importing database |
| CN109684304A (en) * | 2018-12-13 | 2019-04-26 | 中国联合网络通信集团有限公司 | Data migration method and device |
| CN109800229A (en) * | 2018-12-29 | 2019-05-24 | 深圳云天励飞技术有限公司 | Data access method and relevant device |
| CN110851421A (en) * | 2019-11-06 | 2020-02-28 | 珠海格力电器股份有限公司 | Method and device for reducing time consumption of data migration, storage medium and electronic equipment |
| CN110914804A (en) * | 2017-07-12 | 2020-03-24 | 华为技术有限公司 | Virtual machine migration manager and method |
| CN116821100A (en) * | 2023-07-20 | 2023-09-29 | 广州云臻数据科技有限公司 | Database high-fidelity migration method based on binary collaboration |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266606A (en) * | 2007-03-15 | 2008-09-17 | 阿里巴巴公司 | On-line data migration method based on Oracle database |
| CN101908064A (en) * | 2010-07-20 | 2010-12-08 | 中兴通讯股份有限公司 | Data base backup recovery method and device |
| CN102063500A (en) * | 2011-01-04 | 2011-05-18 | 北京凯铭风尚网络技术有限公司 | Data migration method and device |
| US20110145197A1 (en) * | 2008-12-03 | 2011-06-16 | Hitachi Ltd. | Storage system and method for operating storage system |
-
2012
- 2012-12-29 CN CN201210592673.1A patent/CN103914458A/en active Pending
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101266606A (en) * | 2007-03-15 | 2008-09-17 | 阿里巴巴公司 | On-line data migration method based on Oracle database |
| US20110145197A1 (en) * | 2008-12-03 | 2011-06-16 | Hitachi Ltd. | Storage system and method for operating storage system |
| CN101908064A (en) * | 2010-07-20 | 2010-12-08 | 中兴通讯股份有限公司 | Data base backup recovery method and device |
| CN102063500A (en) * | 2011-01-04 | 2011-05-18 | 北京凯铭风尚网络技术有限公司 | Data migration method and device |
Cited By (20)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN104572867A (en) * | 2014-12-18 | 2015-04-29 | 上海新炬网络信息技术有限公司 | Data processing method based on Oracle |
| CN105868210A (en) * | 2015-01-21 | 2016-08-17 | 阿里巴巴集团控股有限公司 | Creating method and device of unique index in distributed database |
| CN107133228A (en) * | 2016-02-26 | 2017-09-05 | 华为技术有限公司 | A kind of method and device of fast resampling |
| WO2018036235A1 (en) * | 2016-08-22 | 2018-03-01 | 中兴通讯股份有限公司 | Solr data migration method and apparatus |
| CN106649449A (en) * | 2016-09-22 | 2017-05-10 | 济南浪潮高新科技投资发展有限公司 | Internal and external network data exchange method based on SQLite |
| CN106572172A (en) * | 2016-11-07 | 2017-04-19 | 湖北省农村信用社联合社网络信息中心 | Multi-process data migration method based on Hash algorithm |
| CN108153806B (en) * | 2016-11-30 | 2021-10-15 | Ncr公司 | Data migration |
| CN108153806A (en) * | 2016-11-30 | 2018-06-12 | Ncr公司 | Data Migration |
| CN106909633A (en) * | 2017-01-23 | 2017-06-30 | 北京思特奇信息技术股份有限公司 | A kind of data by oracle database imported into the method and system of mongodb databases |
| CN110914804A (en) * | 2017-07-12 | 2020-03-24 | 华为技术有限公司 | Virtual machine migration manager and method |
| CN108446315A (en) * | 2018-02-07 | 2018-08-24 | 中国平安人寿保险股份有限公司 | Big data moving method, device, equipment and storage medium |
| CN109684304A (en) * | 2018-12-13 | 2019-04-26 | 中国联合网络通信集团有限公司 | Data migration method and device |
| CN109635033A (en) * | 2018-12-27 | 2019-04-16 | 贵州省广播电视信息网络股份有限公司 | A kind of processing of millions data on stock, log collection and the method for importing database |
| CN109635033B (en) * | 2018-12-27 | 2023-03-21 | 贵州省广播电视信息网络股份有限公司 | Method for processing million-level stock data, collecting logs and importing logs into database |
| CN109800229A (en) * | 2018-12-29 | 2019-05-24 | 深圳云天励飞技术有限公司 | Data access method and relevant device |
| CN109800229B (en) * | 2018-12-29 | 2021-10-08 | 深圳云天励飞技术有限公司 | Data access method and related equipment |
| CN110851421A (en) * | 2019-11-06 | 2020-02-28 | 珠海格力电器股份有限公司 | Method and device for reducing time consumption of data migration, storage medium and electronic equipment |
| CN110851421B (en) * | 2019-11-06 | 2022-04-08 | 珠海格力电器股份有限公司 | Method and device for reducing time consumption of data migration, storage medium and electronic equipment |
| CN116821100A (en) * | 2023-07-20 | 2023-09-29 | 广州云臻数据科技有限公司 | Database high-fidelity migration method based on binary collaboration |
| CN116821100B (en) * | 2023-07-20 | 2023-12-22 | 广州云臻数据科技有限公司 | Database high-fidelity migration method based on binary collaboration |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN103914458A (en) | Mass data migration method and device | |
| CN105868343B (en) | Database migration method and system | |
| CN107958082B (en) | Off-line increment synchronization method and system from database to data warehouse | |
| CN104317843B (en) | A kind of data syn-chronization ETL system | |
| CN101266606B (en) | On-line data migration method based on Oracle database | |
| CN102541757B (en) | Write cache method, cache synchronization method and device | |
| CN102495906A (en) | Incremental data migration method capable of realizing breakpoint transmission | |
| WO2015062181A1 (en) | Method for achieving automatic synchronization of multisource heterogeneous data resources | |
| CN105843707B (en) | Database quick recovery method and equipment | |
| CN103744906A (en) | System, method and device for data synchronization | |
| JP2012511773A (en) | Method and system for efficient data synchronization | |
| US20140156603A1 (en) | Method and an apparatus for splitting and recovering data in a power system | |
| CN103514274A (en) | Method for data migration of non-relational database HBase | |
| CN102779138B (en) | The hard disk access method of real time data | |
| CN106570093B (en) | Mass data migration method and device based on independent metadata organization structure | |
| CN103886091A (en) | Database synchronization method based on record marks | |
| CN102122337A (en) | Method and system for playing back data of instrument-control simulation system of nuclear power station | |
| CN104111994A (en) | Label data screening method and device based on mixed data source | |
| CN104376073A (en) | Database recovery method and device | |
| CN104182436A (en) | Method and device for cleaning databases | |
| CN111367991B (en) | MongoDB data real-time synchronization method and system based on message queue | |
| CN105610899A (en) | Text file parallel uploading method and device | |
| WO2024041433A1 (en) | Data processing method and apparatus | |
| CN100504800C (en) | Method for snapshot of magnetic disc | |
| CN105447105A (en) | NoSQL-based single field section index query method for distributed Internet of things data |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20140709 |