CN107844605A - A kind of system of big data quick backup - Google Patents
A kind of system of big data quick backup Download PDFInfo
- Publication number
- CN107844605A CN107844605A CN201711217170.5A CN201711217170A CN107844605A CN 107844605 A CN107844605 A CN 107844605A CN 201711217170 A CN201711217170 A CN 201711217170A CN 107844605 A CN107844605 A CN 107844605A
- Authority
- CN
- China
- Prior art keywords
- data
- field
- excel
- target
- migration
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1448—Management of the data involved in backup or backup restore
- G06F11/1451—Management of the data involved in backup or backup restore by selection of backup contents
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/21—Design, administration or maintenance of databases
- G06F16/214—Database migration support
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F16/00—Information retrieval; Database structures therefor; File system structures therefor
- G06F16/20—Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
- G06F16/27—Replication, distribution or synchronisation of data between databases or within a distributed database system; Distributed database system architectures therefor
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- Databases & Information Systems (AREA)
- Physics & Mathematics (AREA)
- General Engineering & Computer Science (AREA)
- General Physics & Mathematics (AREA)
- Data Mining & Analysis (AREA)
- Computing Systems (AREA)
- Quality & Reliability (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention provides a kind of system of big data quick backup, it includes Data Migration preparation routine, data derivation program, field adjustment programme, data mover system, data and imports program and data backup module function and system, it is without the computing for needing progress complexity, also the Migration tools of complexity need not be used, only need to use conventional Microsoft Excel and common relevant database, to realize data asynchronous and caused data migration problems by cleverly operating, using data syn-chronization is simpler, cost is lower.
Description
Technical field
The invention belongs to the field of Data Migration, data processing and data syn-chronization, specifically a kind of big data quick backup
System.
Background technology
Data are the Information bases that can not lack that we will face daily, Data Migration (also known as hierarchical storage management,
Hierarchical storage management, hsm) it is a kind of technology for merging offline storage with on-line storage.It will
At a high speed, next stage equipment of the non-Online storage device of high power capacity as disk unit, then presses the data commonly used in disk
The tactful Autonomic Migration Framework specified is to tape library（Abbreviation tape pool）Etc. on two level mass-memory unit.When needing to use these data
When, hierarchical stor automatically can adjust back to these data on upper level disk from next stage storage device.Come for user
To say, above-mentioned data migration operation is entirely transparent, is simply slightly neglected in the speed for accessing disk, and in logic magnetic disc
Substantially feel to substantially increase on capacity.
Data Migration is will to be rarely employed or no file moves on to the archive of secondary storage system (such as tape or CD)
Process.These files are typically need to be in the following image document or historical information that can enter accommodating access any time.Migrate work
Work is combined with backup policy, and still requires that schedule backup.Also include computer data to migrate, migrate old computer（Old system）In
Data, application program, personal settings wait until new computer（New system）, it is necessary after system upgrade.
The asynchronous synchronization after will necessarily producing Data Migration, and migration of data.
At present, our methods of common Data Migration are relative complex, and depend on some complex migration works
Tool, carry out the computing of complexity.
The content of the invention
In order to solve the above problems, the invention provides a kind of system of big data quick backup, this method without need into
The complicated computing of row, it need not also use the Migration tools of complexity, it is only necessary to use conventional Microsoft Excel and common relationship type
Database, to realize data asynchronous and caused data migration problems by cleverly operating, using data syn-chronization it is simpler, into
This is lower.
The invention provides a kind of system of big data quick backup, and it includes Data Migration preparation routine, data export
Program, field adjustment programme, data mover system, data import program and data backup module function and system；
Wherein, Data Migration preparation routine：By the reading of relational database, it is ready to need the data migrated, and compile
Write SQL statement or SQL statement section；
Data derivation program：After getting out SQL statement or SQL statement section, executable inquiry is carried out by inquiring about editing machine
Write, and export as the file of EXCEL forms；
Field adjustment programme：Field quantity, field contents and field sequencing are adjusted according to actual conditions, meanwhile, by field
Determinant attribute carries out remarks；
Data mover system：Adjusted and prepared according to the field of field adjustment programme, contrast the word of the target Microsoft Excel of migration
Section, after confirmation, it would be desirable to the single-row duplication of derived data, then paste in field corresponding to target Microsoft Excel；
Then single-row duplication, paste operation are repeated, until all needing the data paste List of field that migrates to target EXCEL
In form, then preserve, close EXCEL file；
Data import program：The import feature carried by relational database, the EXCEL file that step S40 is obtained import
Into target database, the name of tables of data is named according to actual conditions, then according to the importing of the What You See Is What You Get of database
Prompting is completed to import step by step；
Data backup module：The backup functionality carried using relational database is backed up, so far, the operation of this Data Migration
It has been fully completed.
Wherein, in described field adjustment programme, when the Field Count in target table is more than the Field Count in former tables of data
When, default value setting is carried out according to the actual conditions of the field of target table, in data duplication, makes manually to carry out
Data are filled.
Wherein, in described field adjustment programme, when the Field Count in target table is less than the Field Count in former tables of data
When, the field having more and its content included do not replicate directly；When former table field and object table field name not
When identical and information is consistent, corresponding information is directly replicated under corresponding field.
Wherein, in described data mover system, in addition to data processing module, i.e., after data paste object table,
Before next field information duplication is not carried out, it would be desirable to " lookup ", " all replacing " that the modification of change is carried by EXCEL
Function carries out data processing.
A kind of implementation method of big data quick backup, it comprises the following steps：
Step S10, Data Migration prepare：By the reading of relational database, it is ready to need the data migrated, and write
SQL statement or SQL statement section；
Step S20, data export：After getting out SQL statement or SQL statement section, by inquire about editing machine carry out it is executable
Inquiry is write, and exports as the file of EXCEL forms；
Step S30, field adjustment：Field quantity, field contents and field sequencing are adjusted according to actual conditions, meanwhile, will
Field determinant attribute carries out remarks；
Step S40, Data Migration：Adjusted and prepared according to step S30 field, contrast the field of the target Microsoft Excel of migration,
After confirmation, it would be desirable to the single-row duplication of derived data, then paste in field corresponding to target Microsoft Excel；So
After repeat single-row duplication, paste operation, until all needing the data paste List of field that migrates to target EXCEL tables
In lattice, then preserve, close EXCEL file；
Step S50, data import：The import feature carried by relational database, the EXCEL file that step S40 is obtained
It imported into target database, the name of tables of data is named according to actual conditions, then according to the What You See Is What You Get of database
Prompting is imported to complete to import step by step；
Step S60, data backup：The backup functionality carried using relational database is backed up, so far, this Data Migration
Operation has been fully completed.
Wherein, in the adjustment of described step S30 fields, when the Field Count in target table is more than the field in former tables of data
During number, default value setting is carried out according to the actual conditions of the field of target table, in data duplication, makes manually to enter
Row data are filled.
Wherein, in the adjustment of described step S30 fields, when the Field Count in target table is less than the field in former tables of data
During number, the field having more and its content included do not replicate directly；When the field of former table and the field name of object table
Differ and when information is consistent, directly replicate corresponding information under corresponding field.
Wherein, in described step S40 data migration operation, in addition to step S401 data processings, i.e., glued in data
After being attached to object table, before next field information duplication is not carried out, it would be desirable to which the modification of change " is looked into by what EXCEL was carried
Look for ", " all replace " functions carry out data processing.
Embodiment
A kind of system of big data quick backup, it includes Data Migration preparation routine, data derivation program, field adjustment
Program, data mover system, data import program and data backup module function and system；
Wherein, Data Migration preparation routine：By the reading of relational database, it is ready to need the data migrated, and compile
Write SQL statement or SQL statement section；
Data derivation program：After getting out SQL statement or SQL statement section, executable inquiry is carried out by inquiring about editing machine
Write, and export as the file of EXCEL forms；
Field adjustment programme：Field quantity, field contents and field sequencing are adjusted according to actual conditions, meanwhile, by field
Determinant attribute carries out remarks；
Data mover system：Adjusted and prepared according to the field of field adjustment programme, contrast the word of the target Microsoft Excel of migration
Section, after confirmation, it would be desirable to the single-row duplication of derived data, then paste in field corresponding to target Microsoft Excel；
Then single-row duplication, paste operation are repeated, until all needing the data paste List of field that migrates to target EXCEL
In form, then preserve, close EXCEL file；
Data import program：The import feature carried by relational database, the EXCEL file that step S40 is obtained import
Into target database, the name of tables of data is named according to actual conditions, then according to the importing of the What You See Is What You Get of database
Prompting is completed to import step by step；
Data backup module：The backup functionality carried using relational database is backed up, so far, the operation of this Data Migration
It has been fully completed.
Wherein, in described field adjustment programme, when the Field Count in target table is more than the Field Count in former tables of data
When, default value setting is carried out according to the actual conditions of the field of target table, in data duplication, makes manually to carry out
Data are filled.
Wherein, in described field adjustment programme, when the Field Count in target table is less than the Field Count in former tables of data
When, the field having more and its content included do not replicate directly；When former table field and object table field name not
When identical and information is consistent, corresponding information is directly replicated under corresponding field.
Wherein, in described data mover system, in addition to data processing module, i.e., after data paste object table,
Before next field information duplication is not carried out, it would be desirable to " lookup ", " all replacing " that the modification of change is carried by EXCEL
Function carries out data processing.
A kind of implementation method of big data quick backup, it comprises the following steps：
Step S10, Data Migration prepare：By the reading of relational database, it is ready to need the data migrated, and write
SQL statement or SQL statement section；
Step S20, data export：After getting out SQL statement or SQL statement section, by inquire about editing machine carry out it is executable
Inquiry is write, and exports as the file of EXCEL forms；
Step S30, field adjustment：Field quantity, field contents and field sequencing are adjusted according to actual conditions, meanwhile, will
Field determinant attribute carries out remarks；
Step S40, Data Migration：Adjusted and prepared according to step S30 field, contrast the field of the target Microsoft Excel of migration,
After confirmation, it would be desirable to the single-row duplication of derived data, then paste in field corresponding to target Microsoft Excel；So
After repeat single-row duplication, paste operation, until all needing the data paste List of field that migrates to target EXCEL tables
In lattice, then preserve, close EXCEL file；
Step S50, data import：The import feature carried by relational database, the EXCEL file that step S40 is obtained
It imported into target database, the name of tables of data is named according to actual conditions, then according to the What You See Is What You Get of database
Prompting is imported to complete to import step by step；
Step S60, data backup：The backup functionality carried using relational database is backed up, so far, this Data Migration
Operation has been fully completed.
Wherein, in the adjustment of described step S30 fields, when the Field Count in target table is more than the field in former tables of data
During number, default value setting is carried out according to the actual conditions of the field of target table, in data duplication, makes manually to enter
Row data are filled.
Wherein, in the adjustment of described step S30 fields, when the Field Count in target table is less than the field in former tables of data
During number, the field having more and its content included do not replicate directly；When the field of former table and the field name of object table
Differ and when information is consistent, directly replicate corresponding information under corresponding field.
Wherein, in described step S40 data migration operation, in addition to step S401 data processings, i.e., glued in data
After being attached to object table, before next field information duplication is not carried out, it would be desirable to which the modification of change " is looked into by what EXCEL was carried
Look for ", " all replace " functions carry out data processing.
The above embodiments are intended merely to illustrate the purpose of the present invention and feature, not limit its protection domain.Therefore, other are not
Depart from the simplification carried out under Spirit Essence disclosed in this invention or principle, replacement or modification, completely should all include the present invention's
In covering scope.
Claims (8)
1. a kind of system of big data quick backup, it is characterised in that it includes Data Migration preparation routine, data export journey
Sequence, field adjustment programme, data mover system, data import program and data backup module function and system；
Wherein, Data Migration preparation routine：By the reading of relational database, it is ready to need the data migrated, and compile
Write SQL statement or SQL statement section；
Data derivation program：After getting out SQL statement or SQL statement section, executable inquiry is carried out by inquiring about editing machine
Write, and export as the file of EXCEL forms；
Field adjustment programme：Field quantity, field contents and field sequencing are adjusted according to actual conditions, meanwhile, by field
Determinant attribute carries out remarks；
Data mover system：Adjusted and prepared according to the field of field adjustment programme, contrast the word of the target Microsoft Excel of migration
Section, after confirmation, it would be desirable to the single-row duplication of derived data, then paste in field corresponding to target Microsoft Excel；
Then single-row duplication, paste operation are repeated, until all needing the data paste List of field that migrates to target EXCEL
In form, then preserve, close EXCEL file；
Data import program：The import feature carried by relational database, the EXCEL file that step S40 is obtained import
Into target database, the name of tables of data is named according to actual conditions, then according to the importing of the What You See Is What You Get of database
Prompting is completed to import step by step；
Data backup module：The backup functionality carried using relational database is backed up, so far, the operation of this Data Migration
It has been fully completed.
A kind of 2. system of big data quick backup according to claim 1, it is characterised in that described field adjustment journey
In sequence, when the Field Count in target table is more than the Field Count in former tables of data, according to the actual feelings of the field of target table
Condition carries out default value setting, in data duplication, makes manually to carry out data filling.
A kind of 3. system of big data quick backup according to claim 1, it is characterised in that described field adjustment journey
In sequence, when the Field Count in target table is less than the Field Count in former tables of data, the field having more and its content included
Do not replicate directly；When the field of former table with the field name of object table differs and information is consistent when, directly replicate corresponding
Information under corresponding field.
A kind of 4. system of big data quick backup according to claim 1, it is characterised in that described Data Migration system
In system, in addition to data processing module, i.e., after data paste object table, before next field information duplication is not carried out,
" lookup ", the progress data processing of " all replacing " function that the modification for needing to change is carried by EXCEL.
5. a kind of implementation method of big data quick backup, it is characterised in that it comprises the following steps：
Step S10, Data Migration prepare：By the reading of relational database, it is ready to need the data migrated, and write
SQL statement or SQL statement section；
Step S20, data export：After getting out SQL statement or SQL statement section, by inquire about editing machine carry out it is executable
Inquiry is write, and exports as the file of EXCEL forms；
Step S30, field adjustment：Field quantity, field contents and field sequencing are adjusted according to actual conditions, meanwhile, will
Field determinant attribute carries out remarks；
Step S40, Data Migration：Adjusted and prepared according to step S30 field, contrast the field of the target Microsoft Excel of migration,
After confirmation, it would be desirable to the single-row duplication of derived data, then paste in field corresponding to target Microsoft Excel；So
After repeat single-row duplication, paste operation, until all needing the data paste List of field that migrates to target EXCEL tables
In lattice, then preserve, close EXCEL file；
Step S50, data import：The import feature carried by relational database, the EXCEL file that step S40 is obtained
It imported into target database, the name of tables of data is named according to actual conditions, then according to the What You See Is What You Get of database
Prompting is imported to complete to import step by step；
Step S60, data backup：The backup functionality carried using relational database is backed up, so far, this Data Migration
Operation has been fully completed.
A kind of 6. implementation method of big data quick backup according to claim 5, it is characterised in that described step
In the adjustment of S30 fields, when the Field Count in target table is more than the Field Count in former tables of data, according to the field of target table
Actual conditions carry out default value setting, in data duplication, make manually to carry out data filling.
A kind of 7. implementation method of big data quick backup according to claim 5, it is characterised in that described step
In the adjustment of S30 fields, when the Field Count in target table is less than the Field Count in former tables of data, the field that has more and its wrapped
The content contained does not replicate directly；When the field of former table with the field name of object table differs and information is consistent when, directly
Corresponding information is replicated under corresponding field.
A kind of 8. implementation method of big data quick backup according to claim 5, it is characterised in that described step
In S40 data migration operation, in addition to step S401 data processings, i.e., after data paste object table, in the case where not carrying out
Before one field information replicates, it would be desirable to which " lookup " that the modification of change carries by EXCEL, " whole replacement " function enter line number
According to processing.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711217170.5A CN107844605A (en) | 2017-11-28 | 2017-11-28 | A kind of system of big data quick backup |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201711217170.5A CN107844605A (en) | 2017-11-28 | 2017-11-28 | A kind of system of big data quick backup |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN107844605A true CN107844605A (en) | 2018-03-27 |
Family
ID=61680437
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201711217170.5A Pending CN107844605A (en) | 2017-11-28 | 2017-11-28 | A kind of system of big data quick backup |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN107844605A (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783779A (en) * | 2018-12-27 | 2019-05-21 | 北京航天福道高技术股份有限公司 | A kind of data EXCEL imports and exports the tool of conversion |
-
2017
- 2017-11-28 CN CN201711217170.5A patent/CN107844605A/en active Pending
Cited By (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN109783779A (en) * | 2018-12-27 | 2019-05-21 | 北京航天福道高技术股份有限公司 | A kind of data EXCEL imports and exports the tool of conversion |
| CN109783779B (en) * | 2018-12-27 | 2023-12-22 | 北京航天福道高技术股份有限公司 | Tool for importing, exporting and converting data EXCEL |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20190317944A1 (en) | Methods and apparatus for integrated management of structured data from various sources and having various formats | |
| JP5918243B2 (en) | System and method for managing integrity in a distributed database | |
| KR101556435B1 (en) | Database backing-up and recovering method and device | |
| WO2018133578A1 (en) | Method and device for migrating virtual machine, computer storage medium, and electronic apparatus | |
| CN102541694A (en) | Method and device for database backup | |
| CN105205053A (en) | Method and system for analyzing database incremental logs | |
| CN106055670A (en) | Inter-system data migration method and device | |
| US10783040B2 (en) | Apparatus, system and method for data collection, import and modeling | |
| US11494271B2 (en) | Dynamically updating database archive log dependency and backup copy recoverability | |
| CN104376073A (en) | Database recovery method and device | |
| CN102393860A (en) | Database data migration system | |
| CN110019128A (en) | A kind of system of the synchronization MSSQL data of EXCEL application | |
| CN104820625A (en) | A data record, backup, and recovery method oriented to an information management system | |
| CN109343863B (en) | Interface configuration method and system for HDFS (Hadoop distributed File System) permission | |
| CN107844605A (en) | A kind of system of big data quick backup | |
| US20220121523A1 (en) | Identifying database backup copy chaining | |
| CN107832465A (en) | A kind of method for realizing data fast transferring | |
| CN117194385A (en) | Data migration method between different types of relational databases | |
| CN105630783A (en) | Database data migration system | |
| CN109254960B (en) | Method and device for migrating mass data of database | |
| CN110019127A (en) | It is a kind of to fast implement the asynchronous information synchronization method of MYSQL database | |
| CN106708906A (en) | Database data migration system | |
| CN107943971A (en) | A kind of AC system of the quick mutual photos and sending messages of internal user | |
| Bradberry et al. | Practical Cassandra: a developer's approach | |
| Osuna et al. | Data Protection Strategies in IBM System Storage N Series |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| WD01 | Invention patent application deemed withdrawn after publication | ||
| WD01 | Invention patent application deemed withdrawn after publication |
Application publication date: 20180327 |