CN102609337A - Rapid data recovery method for memory database - Google Patents
Rapid data recovery method for memory database Download PDFInfo
- Publication number
- CN102609337A CN102609337A CN2012100175327A CN201210017532A CN102609337A CN 102609337 A CN102609337 A CN 102609337A CN 2012100175327 A CN2012100175327 A CN 2012100175327A CN 201210017532 A CN201210017532 A CN 201210017532A CN 102609337 A CN102609337 A CN 102609337A
- Authority
- CN
- China
- Prior art keywords
- memory
- memory database
- data
- file
- recovery method
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Images
Landscapes
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a rapid data recovery method for a memory database. All memory data are lost if the memory database is abnormally interrupted for various reasons. Firstly, when the memory database normally runs, all the memory data are written into a specified file (full file) at regular intervals (such as every one day or one hour); secondly, when an external application program adds or deletes the data for the memory database, operated flow logs are immediately written into an incremental file including a memory coordinate corresponding to each change record; thirdly, when in restart once the memory database abnormally stops, the most recently full file is loaded into a memory; and fourthly, all incremental log records after a time point of the full file are sequentially and additionally loaded into the memory one by one, and finally the data are restored into a state before abnormity. As the flow log operated by each record contains coordinate information, the record can be rapidly positioned in the memory, and rapid recovery is realized.
Description
Technical field
The present invention relates to a kind of memory database fast data recovery method, be used for solving after memory database takes place unusually, internal storage data loses, can pass through this method, restore data from disk file fast guarantees that data do not lose.
Background technology
Along with telecommunications industry, financial circles constantly develop, professional expansion is more and more wider, and various new business emerge in an endless stream, and data volume is also increasing, and is also increasingly high to the response speed requirement of data visit.A large amount of subscriber data data, customer consumption data all need leave in the memory database, but because the perishability of internal memory abends in case the memory database process perhaps belongs to main frame, the data in the internal memory just will all be lost.After memory database restarts, how to guarantee that data can return to unusual state before, become the gordian technique of memory database safety.
At present, general memory database adopts the mode of timed backup internal storage data, such as per hour backing up once; Like this, unusual when the internal storage data generation, can recover from the Backup Data of the last time; But; Backup to system from the last time unusual this section period takes place, the retouching operation that memory database takes place all can be lost, and only depends on the mechanism of memory database self to recover; Can only come restore data through the reform business of that a period of time of concrete business procedure, not only time-consuming but also cause mistake easily.
In addition, the SQL statement of the each operation of memory database meeting real time record that has if system takes place unusually like this, can backup to the SQL statement of unusual this section period of system's generation from the last time through reforming and come restore data.Because SQL statement itself need be resolved, and searches the record that needs operation again in internal memory, in the huge telecommunication service system of data volume, be difficult to recover at short notice total data.
Summary of the invention
Technical matters to be solved by this invention provides a kind of new memory database fast data recovery method; After memory database system abends; Can be in the relatively short time; Memory database is restarted, and make each memory table data can return to the state before unusual, guarantee that data do not lose fully.
The technical scheme that the present invention solves the problems of the technologies described above is following: memory database fast data recovery method comprises the steps:
Step 1: memory database regularly all writes the full dose file with internal storage data when normal operation;
Step 2: external application carries out additions and deletions to memory database at every turn and changes when operation, and the flowing water daily record with operation immediately writes delta file;
Step 3:, when memory database restarts, at first load the last full dose file in internal memory in case abending appears in memory database;
Step 4: and then with all the increment log records after the full dose document time point, be appended to one by one in order in the internal memory, finally make data return to unusual preceding state.
The invention has the beneficial effects as follows: regularly (such as every at a distance from 1 day; Perhaps per hour) internal storage data is all write specified file (full dose file); Can let memory database begin to recover from the last BACKUP TIME point, avoid when recovering need to before all delta files reform.Change when operation memory database being carried out additions and deletions at every turn, all immediately the flowing water daily record of operation is write delta file, can guarantee like this being modified in of internal memory all to be left record on the disk at every turn, can effectively prevent losing suddenly of internal storage data.
Further, in delta file, the modification of each bar record all contains the internal memory coordinate of corresponding record, saves the time loss of resolving SQL like this, can directly navigate to corresponding internal memory record fast according to coordinate, realizes fast quick-recovery.
Further, when reforming delta file, strictness is appended in the internal memory in order one by one, can guarantee finally to make data to return to unusual preceding state.
Description of drawings
Fig. 1 is internal storage data storage mode of the present invention and delta file form synoptic diagram;
Fig. 2 is a concrete processing flow chart of the present invention.
Embodiment
Below in conjunction with accompanying drawing principle of the present invention and characteristic are described, institute gives an actual example and only is used to explain the present invention, is not to be used to limit scope of the present invention.
Memory database fast data recovery method of the present invention comprises the steps:
Step 1: memory database regularly all writes the full dose file with internal storage data when normal operation;
Step 2: external application carries out additions and deletions to memory database at every turn and changes when operation, and the flowing water daily record with operation immediately writes delta file;
Step 3:, when memory database restarts, at first load the last full dose file in internal memory in case abending appears in memory database;
Step 4: and then with all the increment log records after the full dose document time point, be appended to one by one in order in the internal memory, finally make data return to unusual preceding state.
At first, the data organization form in the memory database is as shown in Figure 1, and the internal storage data of each table is a unit with 100000, and every is write down a corresponding unique memory block sequence number and the record number in the specified memory piece.Delta file whenever generated a new file at a distance from 10 minutes, and file designation is ALL.yyyymmddhhmi00, for example ALL.20101021080000, ALL.20101021081000 etc.; Every record format of delta file is as shown in Figure 1, has write down the internal memory coordinate and the action type of every amendment record.
Fig. 2 has provided memory database after abending; Operational flowchart when restarting at first adopts multithreading that the full dose file load of each memory table is arrived internal memory, and the time point from last backup begins then; Read the internal memory coordinate in the delta file one by one; Navigate to after the designated recorder, go to revise corresponding internal memory record according to the value of operation flag and each field again, make internal storage data return to unusual preceding state.
Below for adopting the profile information of the inventive method, the presents configuration be the process operation time essential information used, adopt the ini file form.
The above is merely preferred embodiment of the present invention, and is in order to restriction the present invention, not all within spirit of the present invention and principle, any modification of being done, is equal to replacement, improvement etc., all should be included within protection scope of the present invention.
Claims (5)
1. memory database fast data recovery method comprises:
Step 1: memory database regularly all writes the full dose file with internal storage data when normal operation;
Step 2: external application carries out additions and deletions to memory database at every turn and changes when operation, and the flowing water daily record with operation immediately writes delta file;
Step 3:, when memory database restarts, at first load the last full dose file in internal memory in case abending appears in memory database;
Step 4: and then with all the increment log records after the full dose document time point, be appended to one by one in order in the internal memory, finally make data return to unusual preceding state.
2. memory database fast data recovery method according to claim 1 is characterized in that: internal storage data need regularly write the full dose file, and the time regularly is every at a distance from 1 day or 1 hour.
3. memory database fast data recovery method according to claim 1 is characterized in that: delta file is the REDO daily record, need write in real time, and the operation that any additions and deletions change is in case success just must be write delta file immediately.
4. memory database fast data recovery method according to claim 1; It is characterized in that: in the delta file; In every operation Flow Record; All contain the corresponding internal memory coordinate of change record every day, represent that this modification is which bar record of having operated which memory block, so that when data are recovered, can locate fast.
5. memory database fast data recovery method according to claim 1; It is characterized in that: when data are recovered; With all the increment log records after the full dose document time point, strictness is operated one time in order again, guarantees that data can be in full accord with the state before taking place unusually.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100175327A CN102609337A (en) | 2012-01-19 | 2012-01-19 | Rapid data recovery method for memory database |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN2012100175327A CN102609337A (en) | 2012-01-19 | 2012-01-19 | Rapid data recovery method for memory database |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN102609337A true CN102609337A (en) | 2012-07-25 |
Family
ID=46526729
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN2012100175327A Pending CN102609337A (en) | 2012-01-19 | 2012-01-19 | Rapid data recovery method for memory database |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN102609337A (en) |
Cited By (23)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103823634A (en) * | 2012-11-16 | 2014-05-28 | 腾讯科技（深圳）有限公司 | Data processing method and system supporting non-random write mode |
| CN104216802A (en) * | 2014-09-25 | 2014-12-17 | 北京金山安全软件有限公司 | Memory database recovery method and device |
| CN104850556A (en) * | 2014-02-17 | 2015-08-19 | 阿里巴巴集团控股有限公司 | Method and device for data processing |
| CN105022676A (en) * | 2014-04-22 | 2015-11-04 | 大唐软件技术股份有限公司 | Recovery method and device of main memory database redo log files |
| CN105071966A (en) * | 2015-08-10 | 2015-11-18 | 浪潮(北京)电子信息产业有限公司 | Log information management method and log extraction server |
| CN105573859A (en) * | 2014-10-15 | 2016-05-11 | 中兴通讯股份有限公司 | Data recovery method and device of database |
| CN105955845A (en) * | 2016-04-26 | 2016-09-21 | 浪潮电子信息产业股份有限公司 | Data recovery method and device |
| CN105975579A (en) * | 2016-05-05 | 2016-09-28 | 北京思特奇信息技术股份有限公司 | Primary and backup copying method of memory database and memory database system |
| CN105989160A (en) * | 2015-03-03 | 2016-10-05 | 大唐软件技术股份有限公司 | Memory data persistence method and apparatus for Redis database |
| CN106570024A (en) * | 2015-10-10 | 2017-04-19 | 北京国双科技有限公司 | Data increment processing method and apparatus |
| WO2017067397A1 (en) * | 2015-10-20 | 2017-04-27 | 阿里巴巴集团控股有限公司 | Data recovery method and device |
| CN106776120A (en) * | 2016-11-24 | 2017-05-31 | 珠海市魅族科技有限公司 | The method and device of data recovery |
| CN107102940A (en) * | 2017-03-16 | 2017-08-29 | 浙江大学 | A kind of combination full dose type checking point inversely performs implementation method with the target program that incremental is reformed |
| WO2018058633A1 (en) * | 2016-09-30 | 2018-04-05 | 深圳市华傲数据技术有限公司 | Data processing method and apparatus based on increment |
| CN108614876A (en) * | 2018-04-26 | 2018-10-02 | 武汉斗鱼网络科技有限公司 | A kind of system and data processing method based on Redis databases |
| CN109165194A (en) * | 2018-08-13 | 2019-01-08 | 腾讯科技（深圳）有限公司 | A kind of data conversion storage method, apparatus, electronic equipment and storage medium |
| CN110134550A (en) * | 2019-05-15 | 2019-08-16 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and computer readable storage medium |
| CN110908838A (en) * | 2019-11-19 | 2020-03-24 | 杭州安恒信息技术股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN110928890A (en) * | 2019-11-08 | 2020-03-27 | 广州华多网络科技有限公司 | Data storage method and device, electronic equipment and computer readable medium |
| WO2021051444A1 (en) * | 2019-09-19 | 2021-03-25 | 苏州浪潮智能科技有限公司 | Method and system for recovering memory data after program fault restart |
| CN115269274A (en) * | 2022-08-04 | 2022-11-01 | 广州鼎甲计算机科技有限公司 | Data recovery method, apparatus, computer device, storage medium, and program product |
| CN115328704A (en) * | 2022-09-06 | 2022-11-11 | 安徽鼎甲计算机科技有限公司 | File backup method, file recovery method, device, equipment and storage medium |
| WO2024040502A1 (en) * | 2022-08-25 | 2024-02-29 | Intel Corporation | Apparatus, device, method, and computer program for persisting memory recovery actions |
Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101625655A (en) * | 2009-08-20 | 2010-01-13 | 华中科技大学 | Parallel recovery method of memory database |
| CN101706803A (en) * | 2009-11-27 | 2010-05-12 | 天津工业大学 | Method for migrating production data of weaving workshop |
-
2012
- 2012-01-19 CN CN2012100175327A patent/CN102609337A/en active Pending
Patent Citations (2)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101625655A (en) * | 2009-08-20 | 2010-01-13 | 华中科技大学 | Parallel recovery method of memory database |
| CN101706803A (en) * | 2009-11-27 | 2010-05-12 | 天津工业大学 | Method for migrating production data of weaving workshop |
Non-Patent Citations (1)
| Title |
|---|
| 冯宇傲: "《嵌入式实时内存数据库ARTs-EDB的故障恢复策略研究》", 《中国优秀硕士学位论文全文数据库-信息科技辑》, 22 December 2009 (2009-12-22) * |
Cited By (33)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN103823634B (en) * | 2012-11-16 | 2017-12-12 | 腾讯科技（深圳）有限公司 | A kind of data processing method and system supported without random WriteMode |
| CN103823634A (en) * | 2012-11-16 | 2014-05-28 | 腾讯科技（深圳）有限公司 | Data processing method and system supporting non-random write mode |
| CN104850556A (en) * | 2014-02-17 | 2015-08-19 | 阿里巴巴集团控股有限公司 | Method and device for data processing |
| CN104850556B (en) * | 2014-02-17 | 2018-06-29 | 阿里巴巴集团控股有限公司 | A kind of method and device of data processing |
| CN105022676A (en) * | 2014-04-22 | 2015-11-04 | 大唐软件技术股份有限公司 | Recovery method and device of main memory database redo log files |
| CN105022676B (en) * | 2014-04-22 | 2018-08-17 | 大唐软件技术股份有限公司 | A kind of memory database redoes log the restoration methods and device of file |
| CN104216802B (en) * | 2014-09-25 | 2017-05-10 | 北京金山安全软件有限公司 | Memory database recovery method and device |
| CN104216802A (en) * | 2014-09-25 | 2014-12-17 | 北京金山安全软件有限公司 | Memory database recovery method and device |
| CN105573859A (en) * | 2014-10-15 | 2016-05-11 | 中兴通讯股份有限公司 | Data recovery method and device of database |
| CN105989160A (en) * | 2015-03-03 | 2016-10-05 | 大唐软件技术股份有限公司 | Memory data persistence method and apparatus for Redis database |
| CN105071966A (en) * | 2015-08-10 | 2015-11-18 | 浪潮(北京)电子信息产业有限公司 | Log information management method and log extraction server |
| CN106570024A (en) * | 2015-10-10 | 2017-04-19 | 北京国双科技有限公司 | Data increment processing method and apparatus |
| CN106570024B (en) * | 2015-10-10 | 2020-03-06 | 北京国双科技有限公司 | Data increment processing method and device |
| WO2017067397A1 (en) * | 2015-10-20 | 2017-04-27 | 阿里巴巴集团控股有限公司 | Data recovery method and device |
| CN105955845A (en) * | 2016-04-26 | 2016-09-21 | 浪潮电子信息产业股份有限公司 | Data recovery method and device |
| CN105975579A (en) * | 2016-05-05 | 2016-09-28 | 北京思特奇信息技术股份有限公司 | Primary and backup copying method of memory database and memory database system |
| CN105975579B (en) * | 2016-05-05 | 2019-09-17 | 北京思特奇信息技术股份有限公司 | A kind of active and standby clone method and memory database system of memory database |
| WO2018058633A1 (en) * | 2016-09-30 | 2018-04-05 | 深圳市华傲数据技术有限公司 | Data processing method and apparatus based on increment |
| CN106776120A (en) * | 2016-11-24 | 2017-05-31 | 珠海市魅族科技有限公司 | The method and device of data recovery |
| CN107102940A (en) * | 2017-03-16 | 2017-08-29 | 浙江大学 | A kind of combination full dose type checking point inversely performs implementation method with the target program that incremental is reformed |
| CN108614876A (en) * | 2018-04-26 | 2018-10-02 | 武汉斗鱼网络科技有限公司 | A kind of system and data processing method based on Redis databases |
| CN109165194A (en) * | 2018-08-13 | 2019-01-08 | 腾讯科技（深圳）有限公司 | A kind of data conversion storage method, apparatus, electronic equipment and storage medium |
| CN109165194B (en) * | 2018-08-13 | 2021-10-29 | 腾讯科技（深圳）有限公司 | Data unloading method and device, electronic equipment and storage medium |
| CN110134550A (en) * | 2019-05-15 | 2019-08-16 | 腾讯科技（深圳）有限公司 | A kind of data processing method, device and computer readable storage medium |
| WO2021051444A1 (en) * | 2019-09-19 | 2021-03-25 | 苏州浪潮智能科技有限公司 | Method and system for recovering memory data after program fault restart |
| CN110928890A (en) * | 2019-11-08 | 2020-03-27 | 广州华多网络科技有限公司 | Data storage method and device, electronic equipment and computer readable medium |
| CN110928890B (en) * | 2019-11-08 | 2023-01-24 | 广州华多网络科技有限公司 | Data storage method and device, electronic equipment and computer readable storage medium |
| CN110908838A (en) * | 2019-11-19 | 2020-03-24 | 杭州安恒信息技术股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN110908838B (en) * | 2019-11-19 | 2022-09-02 | 杭州安恒信息技术股份有限公司 | Data processing method and device, electronic equipment and storage medium |
| CN115269274A (en) * | 2022-08-04 | 2022-11-01 | 广州鼎甲计算机科技有限公司 | Data recovery method, apparatus, computer device, storage medium, and program product |
| CN115269274B (en) * | 2022-08-04 | 2023-09-29 | 广州鼎甲计算机科技有限公司 | Data recovery method, device, computer equipment and storage medium |
| WO2024040502A1 (en) * | 2022-08-25 | 2024-02-29 | Intel Corporation | Apparatus, device, method, and computer program for persisting memory recovery actions |
| CN115328704A (en) * | 2022-09-06 | 2022-11-11 | 安徽鼎甲计算机科技有限公司 | File backup method, file recovery method, device, equipment and storage medium |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN102609337A (en) | Rapid data recovery method for memory database | |
| US8635187B2 (en) | Method and system of performing incremental SQL server database backups | |
| US20130132346A1 (en) | Method of and system for merging, storing and retrieving incremental backup data | |
| US9542279B2 (en) | Shadow paging based log segment directory | |
| US20080162599A1 (en) | Optimizing backup and recovery utilizing change tracking | |
| CN103197988A (en) | Data backup and recovery method, device and database system | |
| WO2014169142A1 (en) | Partition level backup and restore of a massively parallel processing database | |
| CN102541691B (en) | Log check point recovery method applied to memory data base OLTP (online transaction processing) | |
| US10459804B2 (en) | Database rollback using WAL | |
| CN103617277A (en) | Method for restoring data table content deleted mistakenly | |
| CN103019890A (en) | Block-level disk data protection system and method thereof | |
| US8538925B2 (en) | System and method for backing up test data | |
| US9798761B2 (en) | Apparatus and method for fsync system call processing using ordered mode journaling with file unit | |
| CN103810060A (en) | Data backup method and system thereof based on distributed database | |
| CN113360322A (en) | Method and equipment for recovering data based on backup system | |
| CN111949445A (en) | Incremental backup data storage method, device, equipment and product | |
| CN106502830B (en) | A kind of method for restoring system backup based on Btrfs file system | |
| CN103176867A (en) | Fast file differential backup method | |
| US8595271B1 (en) | Systems and methods for performing file system checks | |
| CN102096613A (en) | Method and device for generating snapshot | |
| CN105022676A (en) | Recovery method and device of main memory database redo log files | |
| CN102073554A (en) | Method and device for recovering files closed abnormally | |
| CN112612775A (en) | Data storage method and device, computer equipment and storage medium | |
| CN104615948A (en) | Method for automatically recognizing file completeness and restoring | |
| WO2017067397A1 (en) | Data recovery method and device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| C06 | Publication | ||
| PB01 | Publication | ||
| C10 | Entry into substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| C53 | Correction of patent of invention or patent application | ||
| CB02 | Change of applicant information |
Address after: 100085 Haidian District, Zhongguancun, South Street, No. 6,, building information, floor, No. 16 Applicant after: SI-TECH Information Technology Ltd. Address before: 100085 Haidian District, Zhongguancun, South Street, No. 6,, building information, floor, No. 16 Applicant before: Beijing Digital China SI-TECH Information Technology Co., Ltd. |
|
| COR | Change of bibliographic data |
Free format text: CORRECT: APPLICANT; FROM: BEIJING DIGITAL CHINA SI-TECH INFORMATION TECHNOLOGY LTD. TO: BEIJING SI-TECH INFORMATION TECHNOLOGY LTD. |
|
| C12 | Rejection of a patent application after its publication | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20120725 |