CN113791932A - File operation processing method and device - Google Patents

File operation processing method and device Download PDF

Info

Publication number
CN113791932A
CN113791932A CN202111148977.4A CN202111148977A CN113791932A CN 113791932 A CN113791932 A CN 113791932A CN 202111148977 A CN202111148977 A CN 202111148977A CN 113791932 A CN113791932 A CN 113791932A
Authority
CN
China
Prior art keywords
crc
block
user data
metadata
check
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202111148977.4A
Other languages
Chinese (zh)
Inventor
章灵芝
白杰
王志刚
张春玲
王丹
李金安
王文华
吴庆
方爽
方梦然
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
State Grid Information and Telecommunication Co Ltd
Original Assignee
State Grid Information and Telecommunication Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by State Grid Information and Telecommunication Co Ltd filed Critical State Grid Information and Telecommunication Co Ltd
Priority to CN202111148977.4A priority Critical patent/CN113791932A/en
Publication of CN113791932A publication Critical patent/CN113791932A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1004Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's to protect a block of data words, e.g. CRC or checksum
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1008Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
    • G06F11/1044Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices with specific ECC/EDC distribution
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F11/00Error detection; Error correction; Monitoring
    • G06F11/07Responding to the occurrence of a fault, e.g. fault tolerance
    • G06F11/08Error detection or correction by redundancy in data representation, e.g. by using checking codes
    • G06F11/10Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's
    • G06F11/1008Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices
    • G06F11/1048Adding special bits or symbols to the coded information, e.g. parity check, casting out 9's or 11's in individual solid state devices using arrangements adapted for a specific error detection or correction feature
    • G06F11/1056Updating check bits on partial write, i.e. read/modify/write

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Quality & Reliability (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Computer Security & Cryptography (AREA)
  • Techniques For Improving Reliability Of Storages (AREA)

Abstract

The application discloses a file operation processing method and device, the scheme of the application simultaneously ensures the reliability of user data and metadata thereof by providing a dual mechanism based on CRC error detection and ECC error correction, the ECC error correction is matched with two-stage CRC error detection, so that the error correction capability is provided to a certain extent for a file system, and the ECC error correction and CRC error detection functions are simultaneously provided for the user data and the metadata thereof, thereby not only improving the stability of the file system, but also improving the safety of the user data.

Description

File operation processing method and device
Technical Field
The application belongs to the technical field of computer file systems, and particularly relates to a file operation processing method and device.
Background
NAND (computer flash memory device) is continuously entering the era of 30nm, 20nm and 10nm from 50nm, increasing transistor density, increasing NAND capacity, and reducing cost, but the increase in NAND process means that a silicon dioxide layer for blocking electrons is also made thinner and thinner, resulting in poor reliability. Therefore, the performance and reliability are degraded to be disadvantages of NAND devices such as TLC/QLC (three-layer memory cell/four-bit cell) flash memories.
In order to meet the requirements of users on cost and capacity, a certain checking and error correcting mechanism needs to be introduced into the file system, so that the reliability of data storage is improved, and an embedded use scene is better served.
Disclosure of Invention
Therefore, the application discloses a file operation processing method and device, which improve the reliability of data storage by introducing a certain checking and error correcting mechanism into a file system, and solve the problem that the storage reliability of equipment based on TLC/QLC and other storage media is insufficient in the industrial embedded field.
The specific technical scheme is as follows:
a method for processing operations on a file, the method comprising:
in a write operation of a file:
writing user data into the data block determined based on the write operation indication, and synchronously updating metadata;
respectively carrying out ECC (error correction code) check on the written user data and the data block where the metadata are located to obtain and record ECC block check values respectively corresponding to the user data and the data block where the metadata are located; performing CRC on the written user data and the data block where the metadata are located respectively to obtain CRC block check values corresponding to the user data and the data block where the metadata are located respectively and recording the CRC block check values; respectively performing CRC (cyclic redundancy check) processing on the rows where the written user data and the metadata thereof are respectively located to obtain CRC row check values corresponding to the rows where the user data and the metadata thereof are respectively located and record the CRC row check values;
in a read operation of a file:
according to the corresponding CRC row check value and/or CRC block check value, checking the metadata indicated by the reading operation indication;
if the metadata verification is successful, determining a data block of the user data to be read according to the metadata data block indicated by the reading operation indication, performing verification processing on the user data to be read according to a corresponding CRC row verification value and/or a CRC block verification value, and reading the user data and returning the user data under the condition of successful verification; and if the metadata check fails or the user data check fails, carrying out corresponding ECC correction processing according to the corresponding ECC block check value.
Optionally, in the above method:
the recording process of the ECC block check value corresponding to the user data and the metadata thereof respectively comprises the following steps: writing the ECC block check value corresponding to the user data into the data block where the user data is located; writing the ECC block check value corresponding to the metadata into the data block where the metadata is located;
the recording process of CRC block check values corresponding to user data and metadata thereof respectively comprises the following steps: writing a CRC block check value corresponding to the user data into a data block where the user data is located; writing a CRC block check value corresponding to the metadata into a data block where the metadata is located;
the recording process of CRC row check values corresponding to the rows where the user data and the metadata are respectively located comprises the following steps: writing a CRC row check value corresponding to the row where the user data is in the last data block in the row where the user data is in; and writing the CRC row check value corresponding to the row in which the metadata is positioned into the last data block in the row in which the metadata is positioned.
Optionally, the performing, according to the corresponding CRC row check value and/or CRC block check value, check processing on the metadata indicated by the read operation indication includes:
performing CRC on the metadata indicated by the reading operation indication according to the corresponding CRC row check value;
if the row CRC of the metadata fails, performing block CRC on the metadata indicated by the reading operation indication according to the corresponding CRC block check value;
if the row CRC check of the metadata is successful or the block CRC check of the metadata is successful, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction;
if the block CRC check of the metadata fails, ECC correction processing is performed on the data block of the metadata indicated by the reading operation indication according to the corresponding ECC block check value.
Optionally, the performing, according to the corresponding CRC row check value and/or CRC block check value, check processing on the user data to be read includes:
performing CRC on the user data to be read according to the corresponding CRC row check value;
if the row CRC of the user data fails, carrying out block CRC on the user data to be read according to a corresponding CRC block check value;
if the row CRC check of the user data is successful or the block CRC check is successful, reading and returning the user data;
and if the block CRC of the user data fails, executing ECC (error correction code) error correction processing on the data block of the user data to be read according to the corresponding ECC block check value.
Optionally, the ECC error correction processing includes:
executing ECC error correction on the data block of the metadata indicated by the reading operation indication or the data block of the user data to be read according to the corresponding ECC block check value;
performing CRC check on the error correction result, and judging whether the error correction result is correct or not;
if the error correction result is correct, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data;
and if the error correction result is wrong, returning prompt information representing the error state.
Optionally, after determining that the error correction result is incorrect, before returning the prompt information representing the error state, the method further includes:
performing speculative error correction processing within a set upper limit of times;
performing CRC (cyclic redundancy check) on an error correction result of the speculative error correction processing to judge whether the error correction result is correct or not;
if the error correction result is correct, triggering to inquire the data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data;
and if the error correction result is wrong, triggering the processing of returning prompt information representing the error state.
An operation processing apparatus of a file, the apparatus comprising:
a write processing module, configured to, in a write operation of a file:
writing user data into the data block determined based on the write operation indication, and synchronously updating metadata;
respectively carrying out ECC (error correction code) check on the written user data and the metadata thereof to obtain and record ECC block check values respectively corresponding to the user data and the metadata thereof; performing CRC on the written user data and the written metadata thereof respectively to obtain CRC block check values corresponding to the user data and the metadata thereof respectively and recording the CRC block check values; respectively performing CRC (cyclic redundancy check) processing on the rows where the written user data and the metadata thereof are respectively located to obtain CRC row check values corresponding to the rows where the user data and the metadata thereof are respectively located and record the CRC row check values;
a read processing module, configured to, in a read operation of a file:
according to the corresponding CRC row check value and/or CRC block check value, checking the metadata indicated by the reading operation indication;
if the metadata verification is successful, determining a data block of the user data to be read according to the metadata data block indicated by the reading operation indication, performing verification processing on the user data to be read according to a corresponding CRC row verification value and/or a CRC block verification value, and reading the user data and returning the user data under the condition of successful verification; and if the metadata check fails or the user data check fails, carrying out corresponding ECC correction processing according to the corresponding ECC block check value.
Optionally, in the above apparatus:
when the write processing module records the ECC block check values corresponding to the user data and the metadata thereof, the write processing module is specifically configured to: writing the ECC block check value corresponding to the user data into the data block where the user data is located; writing the ECC block check value corresponding to the metadata into the data block where the metadata is located;
when the write processing module records CRC block check values respectively corresponding to the user data and the metadata thereof, the write processing module is specifically configured to: writing a CRC block check value corresponding to the user data into a data block where the user data is located; writing a CRC block check value corresponding to the metadata into a data block where the metadata is located;
the write processing module, when recording CRC row check values corresponding to respective rows in which the user data and the metadata thereof are located, is specifically configured to: writing a CRC row check value corresponding to the row where the user data is in the last data block in the row where the user data is in; and writing the CRC row check value corresponding to the row in which the metadata is positioned into the last data block in the row in which the metadata is positioned.
Optionally, in the above apparatus:
when the read processing module performs check processing on the metadata indicated by the read operation indication according to the corresponding CRC row check value and/or CRC block check value, the read processing module is specifically configured to:
performing CRC on the metadata indicated by the reading operation indication according to the corresponding CRC row check value;
if the row CRC of the metadata fails, performing block CRC on the metadata indicated by the reading operation indication according to the corresponding CRC block check value;
if the row CRC check of the metadata is successful or the block CRC check of the metadata is successful, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction;
if the block CRC check of the metadata fails, executing ECC correction processing on the data block of the metadata indicated by the reading operation indication according to the corresponding ECC block check value;
when the read processing module performs check processing on the user data to be read according to the corresponding CRC row check value and/or CRC block check value, the read processing module is specifically configured to:
performing CRC on the user data to be read according to the corresponding CRC row check value;
if the row CRC of the user data fails, carrying out block CRC on the user data to be read according to a corresponding CRC block check value;
if the row CRC check of the user data is successful or the block CRC check is successful, reading and returning the user data;
and if the block CRC of the user data fails, executing ECC (error correction code) error correction processing on the data block of the user data to be read according to the corresponding ECC block check value.
Optionally, when the read processing module executes ECC error correction processing, the read processing module is specifically configured to:
executing ECC error correction on the data block of the metadata indicated by the reading operation indication or the data block of the user data to be read according to the corresponding ECC block check value;
performing CRC check on the error correction result, and judging whether the error correction result is correct or not;
and if the error correction result is correct, inquiring the data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data.
And if the error correction result is wrong, returning prompt information representing the error state.
According to the scheme, the operation processing method and device for the file, disclosed by the application, can simultaneously ensure the reliability of the user data and the metadata thereof by providing a dual mechanism based on CRC error detection and ECC error correction, the ECC error correction is matched with two-stage CRC error detection, so that the error correction capability is provided for the file system to a certain extent, and the ECC error correction and CRC error detection functions are simultaneously provided for the user data and the metadata thereof, so that the stability of the file system can be improved, and the safety of the user data can also be improved.
Drawings
In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings needed to be used in the description of the embodiments or the prior art will be briefly introduced below, it is obvious that the drawings in the following description are only embodiments of the present application, and for those skilled in the art, other drawings can be obtained according to the provided drawings without creative efforts.
FIG. 1 is a process flow diagram of a method of handling a document operation provided herein;
FIG. 2 is a flow chart of a detailed implementation of a write operation provided herein;
FIG. 3 is a flow chart of a detailed implementation of a read operation provided herein;
FIG. 4 is a CRC check flow provided herein;
fig. 5 is a schematic structural diagram of an operation processing device for documents provided in the present application.
Detailed Description
The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application, and it is obvious that the described embodiments are only a part of the embodiments of the present application, and not all of the embodiments. All other embodiments, which can be derived by a person skilled in the art from the embodiments given herein without making any creative effort, shall fall within the protection scope of the present application.
The application discloses a file operation processing method and device, which improve the reliability of data storage and better serve embedded use scenes by introducing a certain checking and error correcting mechanism into a file system.
First, technical terms, basic data structures, and applicable operating environments related to the operation processing method and apparatus of the present application will be described below.
1) Technical terms:
ECC: the abbreviation of Error Correcting Code, the Chinese name is Error checking and Correcting, and the method is an algorithm for Correcting errors based on parity check codes;
CRC: the abbreviation of cyclic Redundancy Check, the name of Chinese is "cyclic Redundancy Check", it uses the principle of division and remainder to make Error detection (Error detection), calculates CRC value and sends to the Check end with the data together, the Check end recalculates CRC value to the received data and compares with received CRC value, if two CRC values are different, it shows that the data is wrong;
BLOCK: the minimum unit of file system access, a Block accessed by one file system, may correspond to N physical blocks or sectors.
2) The basic data structure:
in the present application, the main format of the file system data organization based on ECC error correction and CRC check is specifically shown in table 1 below:
TABLE 1
Figure BDA0003286461000000071
In table 1 above:
the ECC result in the block stores the ECC result value of the content (user data or metadata) of the block (the CRC result in the block does not participate in the ECC check);
an intra-block CRC result that stores a CRC check result value of the contents (USER DATA or metadata) of the block to which it belongs, for example, a check value in which the check result of each block is included in each of META DATA (META DATA) and USER DATA (USER DATA);
an intra-row ECC result indicating the result of ECC checking all contents (user data or metadata) in n blocks included in a certain row of the table;
the intra-row CRC result indicates the result of CRC checking all the contents (user data or metadata) in n blocks included in a certain row of the above table.
In table 1, the value of n can be set by a technician according to requirements, and preferably, the value of n is not more than 4. In this embodiment, each Row in the data table of table 1 is denoted as Row.
3) Operating environment
Including software operating environments and hardware operating environments, see tables 2 and 3, below, respectively.
TABLE 2
Software operating environment Detailed Description
Operating system UBUNTU 12(x86/x64)
TABLE 3
Figure BDA0003286461000000072
Figure BDA0003286461000000081
Based on the above-mentioned related explanations of technical terms, basic data structures, and the like referred to in the present application, the following continues to describe the processing procedure of the operation processing method of the document disclosed in the present application.
The file operation processing method comprises a file write operation flow and a file read operation flow.
Firstly, writing operation flow:
as shown in fig. 1, the write operation of the file at least corresponds to the following write operation flow:
and step 111, writing the user data into the data block determined based on the write operation instruction, and synchronously updating the metadata.
The data blocks described in the embodiments of the present application refer to physical blocks, and the english description in the embodiments of the present application is not distinguished by case.
When an OS (Operation System) transfers user data and writing position information based on a write action of an upper layer (for example, a user inputs data to a file), the file System correspondingly receives an Operation instruction of the OS, which carries the user data and the writing position information.
Optionally, the writing position information is an offset address.
On this basis, based on the basic data structure of table 1, in combination with the detailed implementation flow of the write operation provided with reference to fig. 2, this step 111 can be further implemented as steps a-B shown in fig. 2, specifically as follows:
step A: and determining the logic address of the data to be written according to the offset address written by the operating system, and inquiring and selecting the physical BLOCK to be written according to the logic address.
At this stage, the file system will preferentially select the physical BLOCK to be written among the free BLOCKs in each Row.
And B: user DATA is written to the selected BLOCK while META DATA is updated.
The META DATA is metadata of user DATA, and mainly stores a mapping relationship between a write address and a real physical address issued by an operating system, and other configuration information issued by the operating system, such as read-write permission, sharing permission, directory/file description information, and the like of a file.
Step 112, performing ECC check on the written user data and the data block where the metadata is located, respectively, to obtain and record ECC block check values corresponding to the user data and the data block where the metadata is located, respectively; performing CRC on the written user data and the data block where the metadata are located respectively to obtain CRC block check values corresponding to the user data and the data block where the metadata are located respectively and recording the CRC block check values; and respectively carrying out CRC check processing on the lines where the written user data and the metadata thereof are respectively located to obtain CRC line check values corresponding to the lines where the user data and the metadata thereof are respectively located and recording the CRC line check values.
The recording process of the ECC block check values corresponding to the user data and the metadata thereof includes:
writing the ECC block check value corresponding to the user data into the data block where the user data is located; and writing the ECC block check value corresponding to the metadata into the data block where the metadata is located.
The recording process of CRC block check values corresponding to user data and metadata thereof respectively comprises the following steps:
writing a CRC block check value corresponding to the user data into a data block where the user data is located; writing a CRC block check value corresponding to the metadata into a data block where the metadata is located;
the recording process of CRC row check values corresponding to the rows where the user data and the metadata are respectively located comprises the following steps: writing a CRC row check value corresponding to the row where the user data is in the last data block in the row where the user data is in; and writing the CRC row check value corresponding to the row in which the metadata is positioned into the last data block in the row in which the metadata is positioned.
With further reference to fig. 2, this step 112 can be specifically realized by steps C-E in fig. 2:
and C: and respectively carrying out ECC (error correction code) check on the BLOCKs where the META DATA and the USER DATA are located, and writing check values into the corresponding BLOCKs, namely writing the ECC check value of the META DATA into the BLOCKs where the META DATA is located, and writing the ECC check value of the USER DATA into the BLOCKs where the USER DATA is located.
Here, the ECC check value of META DATA refers to an ECC block check value corresponding to a DATA block in which metadata of USER DATA is located, and the ECC check value of USER DATA refers to an ECC block check value corresponding to a DATA block in which USER DATA is located.
Step D: and respectively performing CRC check on the BLOCK where the META DATA and the USER DATA are located, and writing check values into the corresponding BLOCKs, namely writing the CRC check value of the META DATA into the BLOCK where the META DATA is located, and writing the CRC check value of the USER DATA into the BLOCK where the USER DATA is located.
Here, the CRC value of META DATA refers to a CRC block check value corresponding to a DATA block in which metadata of USER DATA is located, and the CRC value of USER DATA refers to a CRC block check value corresponding to a DATA block in which USER DATA is located.
Step E: CRC checks are performed on the rows (rows) where META DATA and USER DATA are located, respectively, and check values are written.
The check value here refers to a CRC row check value corresponding to a row in which META DATA or USER DATA is located.
Through the processing, the META DATA DATA block and the USER DATA block can respectively have ECC check DATA and CRC check DATA. Meanwhile, the META DATA DATA block and the USER DATA DATA block are grouped according to rows and have CRC DATA at a group level.
Secondly, in the reading operation of the file:
referring further to fig. 1, in the read operation of the file, at least the following read operation flows correspond to:
and step 121, checking the data block of the metadata indicated by the reading operation instruction according to the corresponding CRC row check value and/or CRC block check value.
This step 121 is a process of reading and verifying metadata. The method specifically comprises the following steps: performing CRC on the metadata indicated by the reading operation indication according to the corresponding CRC row check value; if the row CRC of the metadata fails, performing block CRC on the metadata indicated by the reading operation indication according to the corresponding CRC block check value; if the row CRC check of the metadata is successful or the block CRC check of the metadata is successful, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction; if the block CRC check of the metadata fails, ECC correction processing is performed on the data block of the metadata indicated by the reading operation indication according to the corresponding ECC block check value.
Based on the basic data structure of table 1, in conjunction with the detailed implementation flow of the read operation provided with reference to fig. 3, this step 121 can be further implemented as steps a.1-a.6 shown in fig. 3.
Wherein, steps a.1-a.3 are META DATA reading processes in normal state, and steps a.4-a.6 are META DATA reading processes in abnormal state, specifically as follows:
step A.1: and selecting the META BLOCK to be read according to the offset address issued by the operating system.
And step A.2, performing ROW CRC check on the META BLOCK according to the CRC ROW check value of the META BLOCK.
Step A.3: if the ROW CRC check is successful, the step of inquiring USER DATA according to META DATA is entered.
Step A.4: and if the ROW CRC check fails, performing BLOCK CRC check on the META BLOCK according to the CRC BLOCK check value of the META BLOCK.
Step A.5, if the BLOCK CRC check is successful, entering the step of inquiring USER DATA USER DATA according to META DATA.
Step A.6: and if the BLOCK CRC check fails, entering an ECC correction processing flow.
Step 122, if the data block of the metadata is successfully checked, determining a data block of the user data to be read according to the data block of the metadata indicated by the reading operation indication, checking the data block of the user data to be read according to a corresponding CRC row check value and/or a corresponding CRC block check value, and reading the user data and returning the user data under the condition of successful checking; and if the data block of the metadata fails to be checked or the data block of the user data fails to be checked, carrying out corresponding ECC correction processing according to the corresponding ECC block check value.
Step 122 specifically includes reading and verifying the user data when the metadata verification succeeds, and ECC error correction processing when the metadata verification fails or the user data verification fails.
The reading and verifying processing process of the user data comprises the following steps: performing CRC on the user data to be read according to the corresponding CRC row check value; if the row CRC of the user data fails, carrying out block CRC on the user data to be read according to a corresponding CRC block check value; if the row CRC check of the user data is successful or the block CRC check is successful, reading and returning the user data; and if the block CRC of the user data fails, executing ECC (error correction code) error correction processing on the data block of the user data to be read according to the corresponding ECC block check value.
The ECC correction processing process comprises the following steps: executing ECC error correction on the data block of the metadata indicated by the reading operation indication or the data block of the user data to be read according to the corresponding ECC block check value; performing CRC check on the error correction result, and judging whether the error correction result is correct or not; if the error correction result is correct, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data; and if the error correction result is wrong, returning prompt information representing the error state.
Preferably, in another embodiment, after determining that the error correction result is an error, before returning the hint information representing the error state, the flow of the ECC error correction process may further include:
performing speculative error correction processing within a set upper limit of times; performing CRC (cyclic redundancy check) on an error correction result of the speculative error correction processing to judge whether the error correction result is correct or not; if the error correction result is correct, triggering to inquire the data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data; and if the error correction result is wrong, triggering the processing of returning prompt information representing the error state.
In this embodiment, when it is determined that the error correction result is erroneous, the security of the user data is further improved by the speculative error correction processing within the upper limit of the set number of times.
Based on the basic data structure of table 1, in conjunction with the example of a refined implementation flow of the read operation provided with reference to fig. 3, this step 122 can be further implemented as steps b.1-b.5 and steps s.1-s.5 shown in fig. 3.
Steps b.1 to b.5 are read and check processing of USER DATA in case of successful metadata check, steps b.1 to b.2 are specifically USER DATA read flows in normal state (check successful), and steps b.3 to b.5 are USER DATA read flows in abnormal state (check failed). Respectively as follows:
step B.1: if the ROW CRC check of META BLOCK succeeds or the BLOCK CRC check succeeds, the USER DATA BLOCK to be read is selected according to META DATA BLOCK.
And B.2, performing ROW CRC check on the USER DATA ROW according to the CRC ROW check value of the ROW where the USER DATA BLOCK is positioned, and if the ROW CRC check is successful, returning a success state to the operating system and returning the USER DATA.
Step B.3: if the ROW CRC check fails, a BLOCK CRC check is performed on the USER DATA according to the BLOCK check value of the USER DATA BLOCK.
And B.4, if the CRC of the USER DATA BLOCK succeeds, returning the read USER DATA and returning a success state to the operating system.
And step B.5, if the CRC check of the USER DATA BLOCK fails, entering an ECC correction processing flow.
Referring to fig. 3, steps s.1-s.5 are an exemplary ECC error correction processing flow of this embodiment, and specifically are as follows:
step S.1: and performing ECC correction on the META DATA BLOCK or the USER DATA BLOCK, performing CRC check after the ECC correction, and judging whether the error correction result is correct.
And S.2, if the error correction result is correct, selecting the USER DATA BLOCK to be read according to the META DATA BLOCK, or returning the read USER DATA and returning to the correct state of the OS.
And S.3, if the error correction result is wrong, attempting to correct the error speculatively.
And S.4, if the speculative error correction within N times can save the data, returning the read user data and returning the user data to the correct state of the OS.
Step S.5, if the speculative error correction mechanism can not save the data, an error state is returned to the OS.
Further, as shown in fig. 4, when performing CRC check after ECC error correction, the following CRC check flow may be specifically adopted:
removing the last 4 bytes of the META DATA BLOCK or the USER DATA BLOCK, performing CRC check on the META DATA BLOCK or the USER DATA BLOCK, comparing the check value with the last 4 bytes of the BLOCK, judging whether the check value and the last 4 bytes of the BLOCK are consistent, if so, indicating that the DATA content is correct, namely, the ECC correction result is correct, otherwise, indicating that the DATA content is damaged, namely, the ECC correction result is wrong.
According to the scheme, the operation processing method of the file disclosed by the application can simultaneously ensure the reliability of the user data and the metadata thereof by providing a dual mechanism based on CRC error detection and ECC error correction, the ECC error correction is matched with two-stage CRC error detection, so that the error correction capability of a certain degree is provided for the file system, and the ECC error correction and CRC error detection functions are simultaneously provided for the user data and the metadata thereof, so that the stability of the file system can be improved, and the safety of the user data can be improved.
Corresponding to the above method for processing the file, an embodiment of the present application further discloses a device for processing the file, where a composition structure of the device is shown in fig. 5, and the device specifically includes:
a write processing module 501, configured to, in a write operation of a file:
writing user data into the data block determined based on the write operation indication, and synchronously updating metadata;
respectively carrying out ECC (error correction code) check on the written user data and the metadata thereof to obtain and record ECC block check values respectively corresponding to the user data and the metadata thereof; performing CRC on the written user data and the written metadata thereof respectively to obtain CRC block check values corresponding to the user data and the metadata thereof respectively and recording the CRC block check values; respectively performing CRC (cyclic redundancy check) processing on the rows where the written user data and the metadata thereof are respectively located to obtain CRC row check values corresponding to the rows where the user data and the metadata thereof are respectively located and record the CRC row check values;
a reading processing module 502, configured to, in a reading operation of a file:
according to the corresponding CRC row check value and/or CRC block check value, checking the metadata indicated by the reading operation indication;
if the metadata verification is successful, determining a data block of the user data to be read according to the metadata data block indicated by the reading operation indication, performing verification processing on the user data to be read according to a corresponding CRC row verification value and/or a CRC block verification value, and reading the user data and returning the user data under the condition of successful verification; and if the metadata check fails or the user data check fails, carrying out corresponding ECC correction processing according to the corresponding ECC block check value.
In an embodiment, when recording the ECC block check values corresponding to the user data and the metadata thereof, the write processing module 501 is specifically configured to: writing the ECC block check value corresponding to the user data into the data block where the user data is located; writing the ECC block check value corresponding to the metadata into the data block where the metadata is located;
when recording CRC block check values respectively corresponding to user data and metadata thereof, the write processing module 501 is specifically configured to: writing a CRC block check value corresponding to the user data into a data block where the user data is located; writing a CRC block check value corresponding to the metadata into a data block where the metadata is located;
when recording CRC row check values corresponding to respective rows in which user data and metadata thereof are located, the write processing module 501 is specifically configured to: writing a CRC row check value corresponding to the row where the user data is in the last data block in the row where the user data is in; and writing the CRC row check value corresponding to the row in which the metadata is positioned into the last data block in the row in which the metadata is positioned.
In an embodiment, when the read processing module 502 performs check processing on the metadata indicated by the read operation indication according to the corresponding CRC row check value and/or CRC block check value, specifically:
performing CRC on the metadata indicated by the reading operation indication according to the corresponding CRC row check value; if the row CRC of the metadata fails, performing block CRC on the metadata indicated by the reading operation indication according to the corresponding CRC block check value; if the row CRC check of the metadata is successful or the block CRC check of the metadata is successful, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction; if the block CRC check of the metadata fails, executing ECC correction processing on the data block of the metadata indicated by the reading operation indication according to the corresponding ECC block check value;
when the read processing module 502 performs check processing on the user data to be read according to the corresponding CRC row check value and/or CRC block check value, it is specifically configured to:
performing CRC on the user data to be read according to the corresponding CRC row check value; if the row CRC of the user data fails, carrying out block CRC on the user data to be read according to a corresponding CRC block check value; if the row CRC check of the user data is successful or the block CRC check is successful, reading and returning the user data; and if the block CRC of the user data fails, executing ECC (error correction code) error correction processing on the data block of the user data to be read according to the corresponding ECC block check value.
In an embodiment, when the read processing module 502 executes the ECC error correction process, it is specifically configured to:
executing ECC error correction on the data block of the metadata indicated by the reading operation indication or the data block of the user data to be read according to the corresponding ECC block check value; performing CRC check on the error correction result, and judging whether the error correction result is correct or not; and if the error correction result is correct, inquiring the data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data. And if the error correction result is wrong, returning prompt information representing the error state.
In an embodiment, after determining that the error correction result is incorrect, before returning the prompt information representing the error status, the read processing module 502 is further configured to:
performing speculative error correction processing within a set upper limit of times; performing CRC (cyclic redundancy check) on an error correction result of the speculative error correction processing to judge whether the error correction result is correct or not; if the error correction result is correct, triggering to inquire the data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data; and if the error correction result is wrong, triggering the processing of returning prompt information representing the error state.
The operation processing device for the document disclosed in the embodiment of the present application is relatively simple in description because it corresponds to the operation processing method for the document disclosed in each of the above method embodiments, and for the relevant similarities, please refer to the description of the corresponding method embodiments above, and the details are not described here.
To sum up, the method and the device for processing the file operation disclosed by the embodiment of the application have the following technical advantages:
1) both ECC error correction and CRC error detection functions are provided for the metadata and user data. Not only can the stability of the file system be improved, but also the safety of user data can be improved;
2) the ECC can provide 1/8 data fault tolerance through parity check algorithm, if data jump occurs inside the BLOCK, the ECC can directly correct the data to a certain extent, rather than directly report the error;
3) since CRC can provide an all-around error detection capability, during error correction, if the strength of error correction provided by ECC is not enough to provide enough support, the file system can try to recover data in a limited trial and error manner, which can further expand the error tolerance and error correction capability of ECC to a certain extent.
It should be noted that, in the present specification, the embodiments are all described in a progressive manner, each embodiment focuses on differences from other embodiments, and the same and similar parts among the embodiments may be referred to each other.
For convenience of description, the above system or apparatus is described as being divided into various modules or units by function, respectively. Of course, the functionality of the units may be implemented in one or more software and/or hardware when implementing the present application.
From the above description of the embodiments, it is clear to those skilled in the art that the present application can be implemented by software plus necessary general hardware platform. Based on such understanding, the technical solutions of the present application may be essentially or partially implemented in the form of a software product, which may be stored in a storage medium, such as a ROM/RAM, a magnetic disk, an optical disk, etc., and includes several instructions for enabling a computer device (which may be a personal computer, a server, or a network device, etc.) to execute the method according to the embodiments or some parts of the embodiments of the present application.
Finally, it is further noted that, herein, relational terms such as first, second, third, fourth, and the like may be used solely to distinguish one entity or action from another entity or action without necessarily requiring or implying any actual such relationship or order between such entities or actions. Also, the terms "comprises," "comprising," or any other variation thereof, are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but may include other elements not expressly listed or inherent to such process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising an … …" does not exclude the presence of other identical elements in a process, method, article, or apparatus that comprises the element.
The foregoing is only a preferred embodiment of the present application and it should be noted that those skilled in the art can make several improvements and modifications without departing from the principle of the present application, and these improvements and modifications should also be considered as the protection scope of the present application.

Claims (10)

1. A file operation processing method is characterized by comprising the following steps:
in a write operation of a file:
writing user data into the data block determined based on the write operation indication, and synchronously updating metadata;
respectively carrying out ECC (error correction code) check on the written user data and the data block where the metadata are located to obtain and record ECC block check values respectively corresponding to the user data and the data block where the metadata are located; performing CRC on the written user data and the data block where the metadata are located respectively to obtain CRC block check values corresponding to the user data and the data block where the metadata are located respectively and recording the CRC block check values; respectively performing CRC (cyclic redundancy check) processing on the rows where the written user data and the metadata thereof are respectively located to obtain CRC row check values corresponding to the rows where the user data and the metadata thereof are respectively located and record the CRC row check values;
in a read operation of a file:
according to the corresponding CRC row check value and/or CRC block check value, checking the metadata indicated by the reading operation indication;
if the metadata verification is successful, determining a data block of the user data to be read according to the metadata data block indicated by the reading operation indication, performing verification processing on the user data to be read according to a corresponding CRC row verification value and/or a CRC block verification value, and reading the user data and returning the user data under the condition of successful verification; and if the metadata check fails or the user data check fails, carrying out corresponding ECC correction processing according to the corresponding ECC block check value.
2. The method of claim 1, wherein:
the recording process of the ECC block check value corresponding to the user data and the metadata thereof respectively comprises the following steps: writing the ECC block check value corresponding to the user data into the data block where the user data is located; writing the ECC block check value corresponding to the metadata into the data block where the metadata is located;
the recording process of CRC block check values corresponding to user data and metadata thereof respectively comprises the following steps: writing a CRC block check value corresponding to the user data into a data block where the user data is located; writing a CRC block check value corresponding to the metadata into a data block where the metadata is located;
the recording process of CRC row check values corresponding to the rows where the user data and the metadata are respectively located comprises the following steps: writing a CRC row check value corresponding to the row where the user data is in the last data block in the row where the user data is in; and writing the CRC row check value corresponding to the row in which the metadata is positioned into the last data block in the row in which the metadata is positioned.
3. The method according to claim 1, wherein the performing check processing on the metadata indicated by the read operation indication according to the corresponding CRC row check value and/or CRC block check value comprises:
performing CRC on the metadata indicated by the reading operation indication according to the corresponding CRC row check value;
if the row CRC of the metadata fails, performing block CRC on the metadata indicated by the reading operation indication according to the corresponding CRC block check value;
if the row CRC check of the metadata is successful or the block CRC check of the metadata is successful, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction;
if the block CRC check of the metadata fails, ECC correction processing is performed on the data block of the metadata indicated by the reading operation indication according to the corresponding ECC block check value.
4. The method according to claim 3, wherein the checking the user data to be read according to the corresponding CRC row check value and/or CRC block check value includes:
performing CRC on the user data to be read according to the corresponding CRC row check value;
if the row CRC of the user data fails, carrying out block CRC on the user data to be read according to a corresponding CRC block check value;
if the row CRC check of the user data is successful or the block CRC check is successful, reading and returning the user data;
and if the block CRC of the user data fails, executing ECC (error correction code) error correction processing on the data block of the user data to be read according to the corresponding ECC block check value.
5. The method of claim 4, wherein the ECC correction process comprises:
executing ECC error correction on the data block of the metadata indicated by the reading operation indication or the data block of the user data to be read according to the corresponding ECC block check value;
performing CRC check on the error correction result, and judging whether the error correction result is correct or not;
if the error correction result is correct, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data;
and if the error correction result is wrong, returning prompt information representing the error state.
6. The method of claim 5, further comprising, after determining that the error correction result is incorrect, before returning prompt information indicating an error status, the method further comprising:
performing speculative error correction processing within a set upper limit of times;
performing CRC (cyclic redundancy check) on an error correction result of the speculative error correction processing to judge whether the error correction result is correct or not;
if the error correction result is correct, triggering to inquire the data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data;
and if the error correction result is wrong, triggering the processing of returning prompt information representing the error state.
7. An apparatus for processing a file operation, the apparatus comprising:
a write processing module, configured to, in a write operation of a file:
writing user data into the data block determined based on the write operation indication, and synchronously updating metadata;
respectively carrying out ECC (error correction code) check on the written user data and the metadata thereof to obtain and record ECC block check values respectively corresponding to the user data and the metadata thereof; performing CRC on the written user data and the written metadata thereof respectively to obtain CRC block check values corresponding to the user data and the metadata thereof respectively and recording the CRC block check values; respectively performing CRC (cyclic redundancy check) processing on the rows where the written user data and the metadata thereof are respectively located to obtain CRC row check values corresponding to the rows where the user data and the metadata thereof are respectively located and record the CRC row check values;
a read processing module, configured to, in a read operation of a file:
according to the corresponding CRC row check value and/or CRC block check value, checking the metadata indicated by the reading operation indication;
if the metadata verification is successful, determining a data block of the user data to be read according to the metadata data block indicated by the reading operation indication, performing verification processing on the user data to be read according to a corresponding CRC row verification value and/or a CRC block verification value, and reading the user data and returning the user data under the condition of successful verification; and if the metadata check fails or the user data check fails, carrying out corresponding ECC correction processing according to the corresponding ECC block check value.
8. The apparatus of claim 7, wherein:
when the write processing module records the ECC block check values corresponding to the user data and the metadata thereof, the write processing module is specifically configured to: writing the ECC block check value corresponding to the user data into the data block where the user data is located; writing the ECC block check value corresponding to the metadata into the data block where the metadata is located;
when the write processing module records CRC block check values respectively corresponding to the user data and the metadata thereof, the write processing module is specifically configured to: writing a CRC block check value corresponding to the user data into a data block where the user data is located; writing a CRC block check value corresponding to the metadata into a data block where the metadata is located;
the write processing module, when recording CRC row check values corresponding to respective rows in which the user data and the metadata thereof are located, is specifically configured to: writing a CRC row check value corresponding to the row where the user data is in the last data block in the row where the user data is in; and writing the CRC row check value corresponding to the row in which the metadata is positioned into the last data block in the row in which the metadata is positioned.
9. The apparatus of claim 7, wherein:
when the read processing module performs check processing on the metadata indicated by the read operation indication according to the corresponding CRC row check value and/or CRC block check value, the read processing module is specifically configured to:
performing CRC on the metadata indicated by the reading operation indication according to the corresponding CRC row check value;
if the row CRC of the metadata fails, performing block CRC on the metadata indicated by the reading operation indication according to the corresponding CRC block check value;
if the row CRC check of the metadata is successful or the block CRC check of the metadata is successful, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction;
if the block CRC check of the metadata fails, executing ECC correction processing on the data block of the metadata indicated by the reading operation indication according to the corresponding ECC block check value;
when the read processing module performs check processing on the user data to be read according to the corresponding CRC row check value and/or CRC block check value, the read processing module is specifically configured to:
performing CRC on the user data to be read according to the corresponding CRC row check value;
if the row CRC of the user data fails, carrying out block CRC on the user data to be read according to a corresponding CRC block check value;
if the row CRC check of the user data is successful or the block CRC check is successful, reading and returning the user data;
and if the block CRC of the user data fails, executing ECC (error correction code) error correction processing on the data block of the user data to be read according to the corresponding ECC block check value.
10. The apparatus according to claim 9, wherein the read processing module, when performing the ECC error correction processing, is specifically configured to:
executing ECC error correction on the data block of the metadata indicated by the reading operation indication or the data block of the user data to be read according to the corresponding ECC block check value;
performing CRC check on the error correction result, and judging whether the error correction result is correct or not;
if the error correction result is correct, inquiring a data block of the user data to be read according to the data block of the metadata indicated by the reading operation instruction, or reading and returning the user data;
and if the error correction result is wrong, returning prompt information representing the error state.
CN202111148977.4A 2021-09-29 2021-09-29 File operation processing method and device Pending CN113791932A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202111148977.4A CN113791932A (en) 2021-09-29 2021-09-29 File operation processing method and device

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202111148977.4A CN113791932A (en) 2021-09-29 2021-09-29 File operation processing method and device

Publications (1)

Publication Number Publication Date
CN113791932A true CN113791932A (en) 2021-12-14

Family

ID=78877496

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202111148977.4A Pending CN113791932A (en) 2021-09-29 2021-09-29 File operation processing method and device

Country Status (1)

Country Link
CN (1) CN113791932A (en)

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070113120A1 (en) * 2005-07-01 2007-05-17 Don Dodge System employing data verification operations of differing computational costs
US20140013182A1 (en) * 2011-05-27 2014-01-09 Huawei Technologies Co., Ltd. Data processing method, apparatus and system
KR20150073717A (en) * 2013-12-23 2015-07-01 삼성전자주식회사 Storage device and data encoding and decoding methods thereof
CN107710325A (en) * 2015-12-31 2018-02-16 京微雅格(北京)科技有限公司 A kind of FPGA circuitry and its configuration file processing method
US20190004903A1 (en) * 2015-04-29 2019-01-03 Huawei Technologies Co., Ltd. Data Restoration Method and Apparatus, and Terminal

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070113120A1 (en) * 2005-07-01 2007-05-17 Don Dodge System employing data verification operations of differing computational costs
US20140013182A1 (en) * 2011-05-27 2014-01-09 Huawei Technologies Co., Ltd. Data processing method, apparatus and system
KR20150073717A (en) * 2013-12-23 2015-07-01 삼성전자주식회사 Storage device and data encoding and decoding methods thereof
US20190004903A1 (en) * 2015-04-29 2019-01-03 Huawei Technologies Co., Ltd. Data Restoration Method and Apparatus, and Terminal
CN107710325A (en) * 2015-12-31 2018-02-16 京微雅格(北京)科技有限公司 A kind of FPGA circuitry and its configuration file processing method

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘梦影;蔡阳阳;: "一种ECC校验算法的设计与实现", 电子与封装, no. 02 *

Similar Documents

Publication Publication Date Title
US8725944B2 (en) Implementing raid in solid state memory
US7103811B2 (en) Mechanisms for detecting silent errors in streaming media devices
US8321757B2 (en) Method and apparatus for error correction
US20060143507A1 (en) Data-duplication control apparatus
EP3244315B1 (en) Method and apparatus for performing data recovery in redundant storage system
US20100235678A1 (en) System and Method for Data Protection Against Power Failure During Sector Remapping
US9262284B2 (en) Single channel memory mirror
CN109390027B (en) Decoding method and related flash memory controller and electronic device
CN112612637B (en) Memory data storage method, memory controller, processor chip and electronic device
US20130166991A1 (en) Non-Volatile Semiconductor Memory Device Using Mats with Error Detection and Correction and Methods of Managing the Same
US11314594B2 (en) Method, device and computer program product for recovering data
US20100205508A1 (en) Redundant Data in Storage Medium
CN113220221B (en) Memory controller and data processing method
US7577804B2 (en) Detecting data integrity
CN107329850B (en) Memory satisfying mass read-write frequency operation and data storage method
CN109857708B (en) Data access method of embedded system and computer readable storage medium
CN113791932A (en) File operation processing method and device
US20100169572A1 (en) Data storage method, apparatus and system for interrupted write recovery
CN111625199B (en) Method, device, computer equipment and storage medium for improving reliability of solid state disk data path
CN112230855A (en) Solid state disk and read-write method thereof
CN111897676A (en) File backup method and device based on database index
US10250279B2 (en) Circuits and methods for writing and reading data
CN113704028B (en) Memory, data storage method and door lock system
JP2008158724A (en) Staging method for disk array device
US11550710B2 (en) Data processing method and memory controller utilizing the same

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination