CN110874285B - Method for realizing reducible write operation of EXT file system - Google Patents
Method for realizing reducible write operation of EXT file system Download PDFInfo
- Publication number
- CN110874285B CN110874285B CN201911134404.9A CN201911134404A CN110874285B CN 110874285 B CN110874285 B CN 110874285B CN 201911134404 A CN201911134404 A CN 201911134404A CN 110874285 B CN110874285 B CN 110874285B
- Authority
- CN
- China
- Prior art keywords
- write operation
- content
- file
- information
- node
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Active
Links
Images
Classifications
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/08—Error detection or correction by redundancy in data representation, e.g. by using checking codes
- G06F11/10—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
- G06F11/1004—Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's to protect a block of data words, e.g. CRC or checksum
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1402—Saving, restoring, recovering or retrying
- G06F11/1446—Point-in-time backing up or restoration of persistent data
- G06F11/1458—Management of the backup or restore process
-
- G—PHYSICS
- G06—COMPUTING; CALCULATING OR COUNTING
- G06F—ELECTRIC DIGITAL DATA PROCESSING
- G06F11/00—Error detection; Error correction; Monitoring
- G06F11/07—Responding to the occurrence of a fault, e.g. fault tolerance
- G06F11/14—Error detection or correction of the data by redundancy in operation
- G06F11/1479—Generic software techniques for error detection or fault masking
- G06F11/1489—Generic software techniques for error detection or fault masking through recovery blocks
Landscapes
- Engineering & Computer Science (AREA)
- Theoretical Computer Science (AREA)
- General Engineering & Computer Science (AREA)
- Quality & Reliability (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Computer Security & Cryptography (AREA)
- Information Retrieval, Db Structures And Fs Structures Therefor (AREA)
Abstract
The invention relates to a method for realizing restorable write operation of an EXT file system, which comprises the following steps: s1, acquiring the EXT file system to be accessed; s2, reading the node information and the content of the file before the write operation; s3, judging the type of the write operation, if the write operation is a simple write operation, turning to S4, and if the write operation is a complex write operation, turning to S5; s4, recording relevant parameters and check values of the write operation; s5, recording the detailed steps of the writing operation and the parameters and the check value involved in each step. The method does not distinguish off-line and on-line, and the content before modification can be accurately restored by the method.
Description
Technical Field
The invention relates to the field of an Ext file system, in particular to a method for realizing reducible write operation of an Ext file system.
Background
In the Ext file system writing scheme of the existing physical storage device, the directory file name, the attribute content and the like in the file system can be modified quickly. In some application scenarios of special access, the requirement of final access to the physical storage device can be met by modifying the non-user data file. Fig. 1 and 2 show the values of the file attribute before and after modification of a file by the existing scheme, respectively.
The writing scheme has both offline and online. In the off-line writing scheme, the purpose of some application scenes with special access can be realized by mirroring the physical storage equipment, modifying the attribute content of some system files and the like, and rewriting the modified mirror image files into the physical storage equipment. The online writing scheme is to directly write to the storage device.
For some physical storage devices, the data of the storage unit needs to be checked at startup. When some file attribute contents of the device are modified, the original data and the verification data of the storage unit cannot be matched, and the device cannot be normally started.
In the current Ext file system writing scheme of the physical storage device, the following problems exist:
for the offline writing scheme, if the state before the writing operation is to be restored, the image file before the modification needs to be backed up first, and then the image file is rewritten in the mobile terminal, which wastes a certain amount of time and space resources.
For online write schemes, it is not possible to return to the state before the write operation.
Disclosure of Invention
The present invention is directed to a method for implementing a recoverable write operation of an EXT file system to solve the above-mentioned problems. Therefore, the invention adopts the following specific technical scheme:
a method of implementing a restorable write operation of an EXT file system, comprising the steps of:
s1, acquiring an EXT file system needing to be accessed;
s2, reading the node information and the content of the file before the write operation;
s3, judging the type of the write operation, if the write operation is a simple write operation, turning to S4, and if the write operation is a complex write operation, turning to S5;
s4, recording relevant parameters and check values of the write operation;
s5, recording the detailed steps of the writing operation and the parameters and the check value involved in each step.
Further, the relevant parameters in step S4 include the start position and length of the file.
Further, the specific process of step S4 is:
s41, extracting the content and the position before the write operation;
s42, calculating a check value of the content before the write operation;
and S43, executing the write operation and modifying the content.
Still further, the method further includes S6, restoring the file content, specifically, filling 0 in the whole area where the content is located; carrying out reduction operation according to the parameters recorded in the S41; and calculating a check value of the restored content, and comparing the check value with the check value before modification.
Further, the specific process of step S5 is:
s51, reading the superblock information before the write operation: backing up super block information before write operation and calculating a check value;
s52, reading the grouping information of the file system before modification: storing descriptors, block bitmaps, node bitmaps and node table information of corresponding related groups according to the distribution of all node information blocks and content blocks of the file, and calculating check values of related information;
s53, obtaining node information and content of the file according to S1 and S2, recording block addresses related to all the node information and content, storing the node information and content, and calculating corresponding check values;
s54, calculating the change after the write operation: the method comprises the steps of increasing the number of node information blocks and the number of content blocks, and changing grouping information after space distribution;
s55, reading the node information, the file content and the grouping information of the newly added changes after the pre-modification (no substantial modification at present): according to the added information blocks and content blocks calculated by S54, storing the data of the original information blocks and content blocks, storing the descriptors, block bitmaps, node bitmaps and node table information of corresponding related groups, and calculating the check values of the related information;
s56, executing write operation, modifying the file content, specifically, modifying the file node information, including the information of the newly added node block; modify all packet information involved: including the packet descriptors, block bitmaps, node bitmaps, and node table information; the superblock information is modified.
Further, the method further comprises the step of S6, restoring the file content, and specifically, writing the node information and the file content of the newly added change of S55 in the original position; writing the node information and the content of the original file of the S53 into the original position; writing the grouping information of the file system before modification of S52 and the grouping information newly changed in step S55 into the corresponding grouping position; and restoring the super block backed up by the S51.
Further, the simple write operation includes modifying the file permission value and modifying the file time.
Further, the complex write operation includes modifying a file name and modifying a file content.
By adopting the technical scheme, the invention has the beneficial effects that: the method does not distinguish off-line and on-line, and the content before modification can be accurately restored by the method.
Drawings
To further illustrate the various embodiments, the present invention provides the accompanying figures. The accompanying drawings, which are incorporated in and constitute a part of this disclosure, illustrate embodiments of the invention and, together with the description, serve to explain the principles of the embodiments. Those skilled in the art will appreciate still other possible embodiments and advantages of the present invention with reference to these figures. Elements in the figures are not drawn to scale and like reference numerals are generally used to indicate like elements.
FIG. 1 illustrates a screenshot of a file attribute value before a file modification;
FIG. 2 illustrates a screenshot of a file attribute value after a modification of a file;
FIG. 3 is an overall flow diagram of the present invention;
FIG. 4 is a diagram of a file node information structure of EXT 4;
FIG. 5 is a schematic diagram of a mapping relationship between a logical structure and a physical storage structure of an EXT sparse file;
FIG. 6 is a flow chart of the content change calculation of the EXT sparse file;
FIG. 7 shows a screenshot of the authority value of a certain file before modification;
FIG. 8 illustrates a screenshot of the authority value of a certain file after modification;
FIG. 9 illustrates a screenshot of zeroing out content prior to restore;
FIG. 10 shows a screenshot after the authority value of a file that has not previously been cleared to content has been restored.
Detailed Description
The invention will now be further described with reference to the accompanying drawings and detailed description.
As shown in FIG. 3, a method of implementing a restorable write operation of an EXT file system may include the steps of:
100. acquiring an EXT file system needing to be accessed: whether offline or online access, it is necessary to locate the EXT file system that needs to be accessed. The scope of a single-partition image file is the entire file, while a multi-partition image and physical storage device typically need to be located through a partition table.
102. Reading the node information and the content of the file before the write operation: reference may be made to the relevant standard documents of the EXT file system: initializing the file system, including information of superblocks, group descriptors, block bitmaps, node tables, etc. And traversing from the root directory through the accessed file path, and finally reading the node information and the content of the file. The node information mainly stores file attributes and content addresses.
104. The type of write operation is determined and if it is a simple write operation, a transition is made to 106, and if it is a complex write operation, a transition is made to 108. Wherein, the simple writing operation (small modification) refers to the modification which only involves one content and does not affect the modification of the file system area. Common simple operations include "modify file authority value", "modify file time", and the like. Whereas a complex write operation (a large modification) refers to a modification that involves multiple modifications, or affects the file system area, or meets both conditions. Common complex operations are "modify file name", "modify file content", etc.
Through the storage structure of the file system, it can be known in advance whether each modification operation involves multiple modifications or affects the system area of the file system. For example, the authority value of a certain file can be known to be stored at the tail position of the node area of the file through the structure of the EXT4, and only at this position, modifying the content only affects the file authority itself and does not affect the system area of the file system. The file content, EXT, adopts a dynamic storage mode to modify the content, and may involve multiple areas, even affect the redistribution of the used space, and further affect the system area.
106. Recording relevant parameters and check values of the writing operation; specifically, extracting the content and the position before the write operation; calculating a check value of the content before the write operation; and executing the write operation and modifying the content. For some small modifications, parameters such as the initial position, the length and the like of the modified content are recorded, and a check value is added, so that the aim of simple writing and restoring is fulfilled. The following description will take the example of a restorable write operation for modifying the file authority value of EXT 4.
FIG. 4 is a diagram illustrating the file node information structure (256 bytes) of EXT4, and from the structure diagram, it can be seen that the file node information of EXT4 is divided into 5 blocks, respectively denoted as block 1 to block 5.
Note that in the EXT4 File node information structure, the offset address of Block 3 is G3. The method comprises the following specific steps:
1. extracting the position and the content of the file authority value before modification: fig. 7 shows a screenshot of the authority value of a certain file before modification. According to the file node information read in the preparation steps (100 and 102), the offset position and the length of the authority attribute can be obtained by combining the node information structure chart and are respectively marked as P1 and L1. While the binary string of the read authority values is denoted as S1. Where P1 is the offset from offset address G3 of Block 3.
2. The check value of block 5 is calculated, denoted as M5, for example using MD5 and a hash algorithm.
3. And modifying the file authority value: the length of the binary string S2, denoted L2, for the new weight value is calculated. Then P2 is 0x100(256 bytes) -G3-L2. Write the values of P2 and L2 to Block 3, respectively, while writing the new weight value at the P2 offset of G3, as shown in FIG. 8;
4. fill block 5 with 0, as shown in FIG. 9; preventing the presence of intermediate modified residual data after restoration. FIG. 10 shows a screenshot after the authority value of a file that has not previously been cleared to content has been restored. It can be seen from fig. 10 that if the content is not cleared first, residual data 3a 7330 appears after restoration.
5. Restoring the file authority value: according to the parameters recorded in step 1, P1 and L1 are written to block 3, while S1 is written at the P1 offset of G3.
6. The check value for block 5 is recalculated and compared to M5. If the two are consistent, the reduction is successful; if not, a code check is required to find the error. This step is to ensure the reduction accuracy.
It should be noted that 4 to 6 are reduction operations, which are carried out only when necessary.
108. The detailed steps of the writing operation and the parameters and the check values involved in each step are recorded, and for some complex writing operations (large modification), the purpose of complex writing restoration is achieved by recording the detailed steps of the modification process and some parameters and check values involved in each step.
The following describes an example of a recoverable write operation for adding the content of the EXT sparse file. FIG. 5 is a mapping relationship between a logical structure and a physical storage structure of the EXT sparse file, where the contents of the logical block M and the logical block N are both 0, storage is not required, and all physical blocks do not have corresponding. The method comprises the following specific steps:
1. reading super block information before write operation: and backing up the superblock information before the write operation, and calculating a check value which is recorded as M1.
2. According to the preparation steps (100 and 102), node information (including attribute information and content block address of the file) and content (i.e. all physical blocks of the sparse file) of the file are obtained: and recording all node information blocks and content block addresses, storing the node information and the content, and calculating a corresponding check value which is recorded as M2.
3. Before reading and writing operation, grouping information related to the file: according to the step 2, storing corresponding related grouping information including grouping descriptor, block bitmap, node bitmap and node table information at the file system positions of all node information blocks and content blocks of the file, and calculating a check value of the related information, and recording the check value as M3.
4. Calculate the change after write operation: including the increased number of node information blocks and content blocks, and the change of grouping information after space allocation. There is a special place in the sparse file, except that the number of the content blocks added is calculated according to the change of the file size, the content change of the sparse block needs to be judged, and the flow is shown in fig. 6. Wherein, O1 is the number of content blocks directly increased after the file size is changed, O2 is the change caused by whether the contents of the logic block M and the logic block N are not 0, and the total number of newly added content blocks is marked as Q, which is O1+ O2.
5. Reading newly added and changed node information, file content and grouping information after pre-modification (no substantial modification at present): according to the added node information blocks and content blocks calculated in the step 4, storing the data of the original information blocks and content blocks, and calculating a check value which is recorded as M4; and storing the descriptors, the block bitmaps, the node bitmaps and the node table information of the corresponding related groups, and calculating the check value of the related information, and recording the check value as M5.
6. Performing write operations to modify file content
a. And modifying the original node information of the file, and newly adding node block information and content.
b. Modify all packet information involved: including packet descriptors, block bitmaps, node bitmaps, and node table information.
c. And modifying the super block information.
7. Restoring file content
The step needs to recalculate the check value, and if the check value is consistent with the previous check value, the restoration is successful; if not, a code check is required to find the error. The check values are compared to ensure the accuracy of the reduction.
a. And writing the newly changed node information and the file content in the step 5 into the original position, and calculating a check value, namely M6, and comparing the check value with M4.
b. Writing the node information and the content of the original file in the step 2 into the original position, and calculating a check value, namely M7, and comparing with M2.
c. And writing the grouping information of the file system before modification in the step 3 and the grouping information newly changed in the step 5 into corresponding grouping positions, and respectively calculating check values which are recorded as M8 and M9. Where M8 was compared to M3 and M9 was compared to M5.
d. And restoring the superblock backed up in the step 1, and calculating a check value, namely M10, and comparing the check value with M1.
It should be noted that the step of restoring the content of the file is only performed when needed.
It should be noted that the check value calculation referred to above may employ, for example, MD5, a hash algorithm, and the like.
The invention provides a method for realizing restorable write operation of an Ext file system on the basis of the existing writing scheme of the EXT file system of a physical storage device, and the method is successfully applied to restorable write operation of an Android mobile terminal. The scheme does not distinguish online and offline access modes, has two ideas of a simple parameter recording method and a detailed process recording method, and is the perfection and the extension of the idea of the original writing scheme.
While the invention has been particularly shown and described with reference to a preferred embodiment, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims.
Claims (4)
1. A method of implementing a restorable write operation of an EXT file system, comprising the steps of:
s1, acquiring an EXT file system needing to be accessed;
s2, reading the node information and the content of the file before the write operation;
s3, judging the type of the write operation, if the write operation is a simple write operation, turning to S4, and if the write operation is a complex write operation, turning to S5, wherein the simple write operation comprises file right value modification and file time modification, and the complex write operation comprises file name modification and file content modification;
s4, recording relevant parameters and check values of the write operation, and the specific process is as follows:
s41, extracting the content and the position before the write operation;
s42, calculating a check value of the content before the write operation;
s43, executing write operation and modifying the content;
s5, recording the detailed steps of the write operation and the parameters and the check values related to each step, and the specific process is as follows:
s51, reading superblock information before writing: backing up super block information before write operation and calculating a check value;
s52, reading grouping information of the file system before the write operation: storing descriptors, block bitmaps, node bitmaps and node table information of corresponding related groups according to the distribution of all node information blocks and content blocks of the file, and calculating check values of related information;
s53, obtaining node information and content of the file according to S1 and S2, recording block addresses related to all the node information and content, storing the node information and content, and calculating corresponding check values;
s54, calculating the change after the write operation: the method comprises the steps of increasing the number of node information blocks and the number of content blocks, and changing grouping information after space distribution;
s55, reading the node information, the file content and the grouping information which are newly changed after the pre-modification: according to the added information blocks and content blocks calculated in the S54, storing the data of the original information blocks and content blocks, storing the descriptors, block bitmaps, node bitmaps and node table information of corresponding related groups, and calculating the check values of related information, wherein the pre-modification means that the substantial modification is not performed currently;
s56, executing write operation, modifying the file content, specifically, modifying the file node information, including the information of the newly added node block; modify all packet information involved: including the packet descriptors, block bitmaps, node bitmaps, and node table information; and modifying the super block information.
2. The method of claim 1, wherein the related parameters in step S4 include a start position and a length of the file.
3. The method according to claim 1, characterized in that the method further comprises S6, restoring the file content, specifically, filling 0 in the whole area where the content is located; carrying out reduction operation according to the parameters recorded in the S41; and calculating a check value of the restored content, and comparing the check value with the check value before modification.
4. The method according to claim 1, wherein the method further comprises S6, restoring the file content, specifically, writing the node information of the S55 new change and the file content to the original location; writing the node information and the content of the original file of S53 into the original position; writing the grouping information of the file system before modification of S52 and the grouping information newly changed in step S55 into the corresponding grouping position; and restoring the super block backed up by the S51.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911134404.9A CN110874285B (en) | 2019-11-19 | 2019-11-19 | Method for realizing reducible write operation of EXT file system |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201911134404.9A CN110874285B (en) | 2019-11-19 | 2019-11-19 | Method for realizing reducible write operation of EXT file system |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN110874285A CN110874285A (en) | 2020-03-10 |
| CN110874285B true CN110874285B (en) | 2022-06-21 |
Family
ID=69718291
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201911134404.9A Active CN110874285B (en) | 2019-11-19 | 2019-11-19 | Method for realizing reducible write operation of EXT file system |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN110874285B (en) |
Families Citing this family (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN110569218B (en) * | 2019-08-01 | 2021-11-23 | 厦门市美亚柏科信息股份有限公司 | Offline modification method and device for EXT file system and storage medium |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011198357A (en) * | 2010-02-26 | 2011-10-06 | Jvc Kenwood Corp | Processing device and write method |
| CN102955720A (en) * | 2011-08-25 | 2013-03-06 | 北京中科智网科技有限公司 | Method for improving stability of EXT (extended) file system |
| CN104035830A (en) * | 2014-06-24 | 2014-09-10 | 浙江宇视科技有限公司 | Method and device for recovering data |
| CN107807792A (en) * | 2017-10-27 | 2018-03-16 | 郑州云海信息技术有限公司 | A kind of data processing method and relevant apparatus based on copy storage system |
| CN108090168A (en) * | 2017-12-14 | 2018-05-29 | 厦门市美亚柏科信息股份有限公司 | A kind of general F2FS file system analytic method, terminal device and storage medium |
| CN110278222A (en) * | 2018-03-15 | 2019-09-24 | 华为技术有限公司 | The method, system and relevant device of data management in distributed file storage system |
-
2019
- 2019-11-19 CN CN201911134404.9A patent/CN110874285B/en active Active
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2011198357A (en) * | 2010-02-26 | 2011-10-06 | Jvc Kenwood Corp | Processing device and write method |
| CN102955720A (en) * | 2011-08-25 | 2013-03-06 | 北京中科智网科技有限公司 | Method for improving stability of EXT (extended) file system |
| CN104035830A (en) * | 2014-06-24 | 2014-09-10 | 浙江宇视科技有限公司 | Method and device for recovering data |
| CN107807792A (en) * | 2017-10-27 | 2018-03-16 | 郑州云海信息技术有限公司 | A kind of data processing method and relevant apparatus based on copy storage system |
| CN108090168A (en) * | 2017-12-14 | 2018-05-29 | 厦门市美亚柏科信息股份有限公司 | A kind of general F2FS file system analytic method, terminal device and storage medium |
| CN110278222A (en) * | 2018-03-15 | 2019-09-24 | 华为技术有限公司 | The method, system and relevant device of data management in distributed file storage system |
Non-Patent Citations (1)
| Title |
|---|
| Android智能终端中基于日志的文件恢复方法;丁云冰等;《长春工程学院学报(自然科学版)》;20190915(第3期);第81-84页 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN110874285A (en) | 2020-03-10 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN109343790B (en) | Data storage method based on NAND FLASH, terminal equipment and storage medium | |
| CN110058873B (en) | Application page updating method, device, equipment and storage medium | |
| US8812816B2 (en) | Garbage collection schemes for index block | |
| US9104329B2 (en) | Mount-time reconciliation of data availability | |
| US7103811B2 (en) | Mechanisms for detecting silent errors in streaming media devices | |
| US20030204699A1 (en) | Method for embedding integrity metadata | |
| CN107479823B (en) | Data verification method and device in random read-write file test | |
| CN110502881B (en) | Method, system and storage medium for dynamically modifying read-only attribute of android system | |
| CN109086078B (en) | Android system upgrading method and device, server and mobile terminal | |
| US8347052B2 (en) | Initializing of a memory area | |
| CN107203331B (en) | Method and device for writing data | |
| CN110888851B (en) | Method and device for creating and decompressing compressed file, and electronic and storage device | |
| CN112214346A (en) | Method and apparatus for error detection during data modification in a memory subsystem | |
| CN110874285B (en) | Method for realizing reducible write operation of EXT file system | |
| JP2000305857A (en) | Data medium with recoverable basic data basis state and its formation | |
| CN107908500B (en) | Bad block identification method and device | |
| CN107257281B (en) | Method, apparatus and computer readable storage medium for NOR F L ASH storage key record | |
| CN114756523A (en) | Distributed storage method, device and medium | |
| US7536507B1 (en) | Methods, systems, and computer program products for preserving atomic writes using leaves and bitmaps | |
| CN112394873B (en) | Data management method, system, electronic equipment and storage medium | |
| CN114443353A (en) | Mirror image file simulation method and device, electronic equipment and storage medium | |
| CN115698975A (en) | Device, method for executing file transaction and method for executing access operation | |
| CN112148220A (en) | Method and device for realizing data processing, computer storage medium and terminal | |
| CN111625393A (en) | L2P table verification method, device, equipment and readable storage medium | |
| US12032860B2 (en) | Data protection method for memory and storage device thereof |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |