WO2021088403A1 - 一种数据纠错方法、装置、设备及可读存储介质 - Google Patents

一种数据纠错方法、装置、设备及可读存储介质 Download PDF

Info

Publication number
WO2021088403A1
WO2021088403A1 PCT/CN2020/102018 CN2020102018W WO2021088403A1 WO 2021088403 A1 WO2021088403 A1 WO 2021088403A1 CN 2020102018 W CN2020102018 W CN 2020102018W WO 2021088403 A1 WO2021088403 A1 WO 2021088403A1
Authority
WO
WIPO (PCT)
Prior art keywords
data
error correction
target
neural network
correct
Prior art date
Application number
PCT/CN2020/102018
Other languages
English (en)
French (fr)
Inventor
王岩
李卫军
Original Assignee
深圳大普微电子科技有限公司
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by 深圳大普微电子科技有限公司 filed Critical 深圳大普微电子科技有限公司
Publication of WO2021088403A1 publication Critical patent/WO2021088403A1/zh
Priority to US17/733,165 priority Critical patent/US12107601B2/en

Links

Images

Classifications

    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/033—Theoretical methods to calculate these checking codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
    • H03M13/13—Linear codes
    • H03M13/15—Cyclic codes, i.e. cyclic shifts of codewords produce other codewords, e.g. codes defined by a generator polynomial, Bose-Chaudhuri-Hocquenghem [BCH] codes
    • H03M13/151—Cyclic codes, i.e. cyclic shifts of codewords produce other codewords, e.g. codes defined by a generator polynomial, Bose-Chaudhuri-Hocquenghem [BCH] codes using error location or error correction polynomials
    • H03M13/1515—Reed-Solomon codes
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/044—Recurrent networks, e.g. Hopfield networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
    • H03M13/3707—Adaptive decoding and hybrid decoding, e.g. decoding methods or techniques providing more than one decoding algorithm for one code
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/37—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35
    • H03M13/3746—Decoding methods or techniques, not specific to the particular type of coding provided for in groups H03M13/03 - H03M13/35 with iterative decoding
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/65—Purpose and implementation aspects
    • H03M13/6597—Implementations using analogue techniques for coding or decoding, e.g. analogue Viterbi decoder
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
    • H03M13/11—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits using multiple parity bits
    • H03M13/1102—Codes on graphs and decoding on graphs, e.g. low-density parity check [LDPC] codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
    • H03M13/13—Linear codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
    • H03M13/13—Linear codes
    • H03M13/15—Cyclic codes, i.e. cyclic shifts of codewords produce other codewords, e.g. codes defined by a generator polynomial, Bose-Chaudhuri-Hocquenghem [BCH] codes
    • H03M13/151—Cyclic codes, i.e. cyclic shifts of codewords produce other codewords, e.g. codes defined by a generator polynomial, Bose-Chaudhuri-Hocquenghem [BCH] codes using error location or error correction polynomials
    • H03M13/152—Bose-Chaudhuri-Hocquenghem [BCH] codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/05—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using block codes, i.e. a predetermined number of check bits joined to a predetermined number of information bits
    • H03M13/13—Linear codes
    • H03M13/19—Single error correction without using particular properties of the cyclic codes, e.g. Hamming codes, extended or generalised Hamming codes
    • H—ELECTRICITY
    • H03—ELECTRONIC CIRCUITRY
    • H03M—CODING; DECODING; CODE CONVERSION IN GENERAL
    • H03M13/00—Coding, decoding or code conversion, for error detection or error correction; Coding theory basic assumptions; Coding bounds; Error probability evaluation methods; Channel models; Simulation or testing of codes
    • H03M13/03—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words
    • H03M13/23—Error detection or forward error correction by redundancy in data representation, i.e. code words containing more digits than the source words using convolutional codes, e.g. unit memory codes

Definitions

  • This application relates to the field of computer coding technology, and in particular to a data error correction method, device, equipment, and readable storage medium.
  • error correction capabilities of various error correction codes are limited.
  • Hamming Code can correct at most one bit error
  • LDPC code Low-Density Parity-Check code
  • LDPC code Low-Density Parity-Check code
  • the purpose of this application is to provide a data error correction method, device, device, and readable storage medium to avoid error correction failure and data loss in the data error correction process.
  • the specific plan is as follows:
  • this application provides a data error correction method, including:
  • using the target neural network to correct the target data to obtain the second data includes:
  • using the target neural network to correct the data part to obtain the second data includes:
  • the data part is input into the target neural network in reverse order to obtain the second data.
  • using an error correction code to perform error correction on the target data to obtain the first data includes:
  • the error correction code is used to correct the erroneous data bits of the data part to obtain the first data.
  • using the target neural network to correct the target data to obtain the second data includes:
  • the generation process of the target neural network includes:
  • the initial neural network includes: encoding subnet and decoding subnet;
  • Input training data containing different types of data to train the initial neural network
  • the current initial neural network is determined as the target neural network.
  • the method further includes:
  • Each data set is used to train the target neural network separately, and multiple target neural networks corresponding to each data set are obtained.
  • using the target neural network to correct the target data to obtain the second data includes:
  • the data characteristics of the target data are determined, and the target neural network with the highest matching degree with the data characteristics is selected from the multiple target neural networks to correct the target data to obtain the second data.
  • this application provides a data error correction device, including:
  • the acquisition module is used to acquire the target data to be corrected
  • the error correction module is used to perform error correction on the target data by using the error correction code to obtain the first data;
  • the judgment module is used to judge whether the error correction of the first data is successful
  • the correction module is used to correct the target data using the target neural network if the error correction of the first data is not successful, obtain the second data, determine the second data as the target data, and transmit the target data to the error correction module;
  • the determining module is configured to determine the first data as target data for completing the error correction if the error correction of the first data is successful.
  • this application provides a data error correction device, including:
  • Memory used to store computer programs
  • the processor is used to execute a computer program to implement the data error correction method disclosed above.
  • the present application provides a readable storage medium for storing a computer program, where the computer program is executed by a processor to implement the data error correction method disclosed above.
  • this application provides a data error correction method, including: S11, obtaining the target data to be corrected; S12, using the error correction code to correct the target data to obtain the first data; S13, judging the first data 1. Whether the error correction of the data is successful; if yes, execute S15; if not, execute S14; S14, use the target neural network to correct the target data to obtain the second data, determine the second data as the target data, and execute S12; S15. Determine the first data as target data for completing error correction.
  • this method is using the error correction code to correct the target data. If the error correction is not successful, then the target neural network is used to correct the target data to obtain the second data, and the second data is determined as the target data. In order to use the error correction code to perform error correction on the data again, so as to obtain the first data with a successful error correction, then the first data is determined as the target data for completing the error correction, and the correct target data is obtained. It can be seen that the present application can use the error correction code to cooperate with the target neural network to correct the error data, thereby avoiding the error correction failure of the error correction code data and the data loss during the error correction process.
  • the data error correction device, equipment, and readable storage medium provided in this application also have the above technical effects.
  • Figure 1 is a flow chart of the first data error correction method disclosed in this application.
  • FIG. 2 is a flow chart of the second data error correction method disclosed in this application.
  • FIG. 3 is a flow chart of the third data error correction method disclosed in this application.
  • FIG. 4 is a schematic diagram of the first type of error data distribution disclosed in this application.
  • FIG. 5 is a schematic diagram of the second type of error data distribution disclosed in this application.
  • Fig. 6 is a schematic diagram of LSTM error correction data disclosed in this application.
  • Fig. 7 is a schematic diagram of an LSTM Cell disclosed in this application.
  • FIG. 8 is a schematic diagram of LSTM reverse order error correction data disclosed in this application.
  • FIG. 9 is a schematic diagram of a two-layer structure LSTM network disclosed in this application.
  • FIG. 10 is a flowchart of the fourth data error correction method disclosed in this application.
  • FIG. 11 is a schematic diagram of a data error correction device disclosed in this application.
  • FIG. 12 is a schematic diagram of a data error correction device disclosed in this application.
  • the present application provides a data error correction solution, which can avoid error correction failure and data loss in the data error correction process.
  • an embodiment of the present application discloses a first data error correction method, including:
  • the error correction code can be any type of error correction code, such as Hamming code, BCH (Bose Ray-Chaudhuri Hocquenghem), Polar Code (Polar Code), and so on.
  • the target neural network can also be any kind of neural network, such as: RNN (Recurrent Neural Networks), GRU (Gated Recurrent Unit), FCN (Fully-Connected Networks), CNN (Convolutional Neural Networks), Random Forest (Random Forest Algorithm), SVM (Support Vector Machine), Logical Regression, etc.
  • the error correction of the first data is successful, it indicates that the first data is the target data for completing the error correction; otherwise, the target neural network is used to correct the target data to reduce the number of erroneous data in the target data.
  • the data can be corrected with an error correction code.
  • the current coded decoder generally adopts the design scheme of iterative decoding. If the calculation of the iterative decoding can successfully converge and the calculation is completed, the decoding is successful; otherwise, the decoding fails. Therefore, the key to judging whether the error correction is successful or not lies in whether the iterative decoding calculation can complete convergence. For details, please refer to the prior art, which will not be repeated in this specification.
  • using the target neural network to correct the target data to obtain the second data includes: separating the data part and the verification part of the target data; using the target neural network to correct the data part to obtain the second data .
  • the error correction code is used to correct the target data to obtain the first data, including: separating the data part and the check part of the target data; using the error correction code to correct the erroneous data bits of the data part , Get the first data.
  • each piece of data includes a data part and a check part.
  • the data of the data part and the error mode are more related, and the proportion in a data is larger (generally more than 90%), and the check part is the same or similar for each data in the same system of. Therefore, the target neural network or error correction code can be used to process only the data part of the data. After the processing is completed, the processed result and the original check part can be combined again to obtain the processed data.
  • correcting the target data by using the target neural network to obtain the second data includes: inputting the target data into the target neural network in reverse order to obtain the second data.
  • Inputting the target data into the target neural network in reverse order can make the position of the data to be processed closer to the position where the data is encoded, which is convenient for information recording and encoding.
  • the target neural network is used to correct the target data to obtain the second data.
  • the second data is determined as the target data, so that the error correction code is used to correct the data again, so as to obtain the first data with a successful error correction, then the first data is determined as the target data for completing the error correction, and the result is obtained Correct target data.
  • the present application can use the error correction code to cooperate with the target neural network to correct the error data, thereby avoiding the error correction failure of the error correction code data and the data loss during the error correction process.
  • an embodiment of the present application discloses a second data error correction method, including:
  • S202 Perform error correction on the target data by using the error correction code to obtain first data
  • S205 Input the data part into the target neural network in reverse order to obtain second data, determine the second data as target data, and execute S202;
  • the data part is input into the target neural network in reverse order, and the obtained output data is recombined with the check part to obtain the second data.
  • the second data when the second data is obtained, it can be further judged whether the second data has been successfully corrected; if so, the second data can be directly determined as the target data for completing the error correction; if not, the second data Determine the target data, and execute S202.
  • inputting the data part of the target data into the target neural network in reverse order can make the position of the data to be processed closer to the position where the data is encoded, which is convenient for information recording and encoding.
  • the data part of the target data to be processed is: "ABSD”, after reversing the order, you can get “DSBA”, then input "DSBA” into the target neural network, and the target neural network encodes the data in the order of "DSBA”, But the data is decoded in the order of "ABSD”, so for "A”, its encoding position and decoding position are closer.
  • the encoding positions of "B, S, D” and their respective decoding positions are also relatively close.
  • the data is encoded and decoded in the order of "ABSD".
  • ABSSD atomic layer decoding
  • 3 characters between the encoding position and the decoding position so it is not convenient to record the character information.
  • the above examples are only to illustrate the technical effects of this method, and the encoding and decoding of data in the actual application process should not be so simple to understand.
  • the error correction code is used to correct the target data to obtain the first data, including: separating the data part and the check part of the target data; using the error correction code to correct the erroneous data bits of the data part , Get the first data.
  • each piece of data includes a data part and a check part.
  • the data of the data part and the error mode are more related, and the proportion in a data is larger (generally more than 90%), and the check part is the same or similar for each data in the same system of. Therefore, the error correction code can be used to process only the data part of the data. After the processing is completed, the processed result and the original check part are combined again to obtain the corrected data.
  • the target neural network is used to correct the target data to obtain the second data.
  • the second data is determined as the target data, so that the error correction code is used to correct the data again, so as to obtain the first data with a successful error correction, then the first data is determined as the target data for completing the error correction, and the result is obtained Correct target data.
  • the present application can use the error correction code to cooperate with the target neural network to correct the error data, thereby avoiding the error correction failure of the error correction code data and the data loss during the error correction process.
  • the generation process of the target neural network includes: building an initial neural network, which includes an encoding subnet and a decoding subnet; training the initial neural network with training data containing different types of data; If the error loss function of the initial neural network (such as the cross-entropy loss function) converges to a stable state during the training process, the current initial neural network is determined as the target neural network.
  • the error loss function of the initial neural network such as the cross-entropy loss function
  • the error loss function is also called the loss function.
  • the cross-entropy loss function can be used as a loss function in a neural network. It uses the sigmoid function to avoid the problem of the decrease of the learning rate of the mean square error loss function when the gradient is descent, because the learning rate can be controlled by the output error.
  • the specific error loss function can be referred to the introduction of the prior art, which will not be repeated in this specification.
  • the target neural network obtained by training with training data containing different types of data has good versatility. If you want to obtain a more targeted neural network, you can determine the current initial neural network as the target neural network. After that, it also includes: dividing the training data into multiple data sets, and the data in each data set has an association relationship; using each data set to train the target neural network separately to obtain multiple target neural networks corresponding to each data set .
  • Each target neural network thus obtained will have unique characteristics. For example, if the data in the data set has spatially adjacent association relationships, then the target neural network trained on the data set will be able to compare spatially adjacent data It has a better correction effect, so the characteristics of the target neural network can be determined to be spatially adjacent.
  • the data to be processed is spatially adjacent data
  • the data feature of the data has the highest matching degree with the target neural network
  • the target neural network has a better processing effect on the data.
  • the target neural network can reduce the amount of erroneous data in the data, thereby providing a guarantee for the success of the error correction of the error correction code.
  • association relationship includes: adjacent in space and/or adjacent in time, and may also include: association in business applications, correlation in data format, and the like.
  • using the target neural network to modify the target data to obtain the second data includes: determining the data characteristics of the target data, and selecting the target nerve with the highest matching degree with the data characteristics from the multiple target neural networks The network corrects the target data to obtain the second data.
  • the degree of matching between the data feature of the data and the target neural network can be calculated according to the data feature of the data and the network feature of the target neural network.
  • the encoding subnet and the decoding subnet exist independently, and the encoding subnet can be shared. That is, multiple target neural networks share a coding subnet, but each has a specific decoding subnet, so when multiple neural networks are used to correct the same data at the same time, only one coding calculation is required to avoid repeated coding.
  • the error correction code is LDPC
  • the neural network is LSTM (Long Short-Term Memory). The process of combining the two error correction data is shown in FIG. 3.
  • the error correction process shown in FIG. 3 is implemented by the main control chip of the solid state hard disk.
  • the data error bit in the output data is 90errors/4KB.
  • Figure 4 for details.
  • the error correction process shown in Figure 3 there are no error data bits in the output data, and the number of error data bits in the LSTM output data is reduced to 42.
  • Figure 5 for details. It can be seen that errors that LDPC cannot correct can be resolved in this embodiment.
  • the small black dots in Figure 4 and Figure 5 indicate error data bits.
  • FIG. 6 please refer to FIG. 6 for the process of LSTM correction data.
  • LSTM corrects "ABSD” to "ABCD".
  • a file saves the four letters "ABCD” in ASCII format, which is 32 bits in total. Because it has been stored in the SSD for too long, it will become “ABSD” when it is read out again.
  • LSTM recognizes that this is a four-letter string through encoding, but "ABSD” rarely appears continuously in the file, which may be caused by a reading error; then, in the decoding process, LSTM changed "ABSD” to "ABCD”.
  • the LSTM coding in Fig. 6 can realize code multiplexing, adapting to the pairing of different types of error correction codes.
  • x t and h t-1 are the input data of the LSTM Cell
  • x t is the data to be processed by the current LSTM Cell
  • h t-1 is the output result of the previous LSTM Cell.
  • the input data also includes the previous LSTM Cell.
  • "Input Gate”, “Forget Gate”, “Cell Gate” and “Output Gate” in Fig. 7 are the four processing gates of the current LSTM Cell, and these four processing gates process x t and h t-1 respectively .
  • Figure 7 Represents the vector matrix multiplication operation, Means addition, Represents Hadamard Product (Hadamard Product), Represents the activation function.
  • W x is the weight value of x t in the current processing gate
  • W h is the weight value of h t-1 in the current processing gate.
  • W xi is the weight value of x t in the “Input Gate”
  • W hi is the weight value of h t-1 “Input Gate”
  • each frame in Figure 6, Figure 8 and Figure 9 is an LSTM Cell set. The structure of each LSTM Cell in Figure 6, Figure 8 and Figure 9 can be seen in Figure 7.
  • the network parameters of LSTM can be selected according to the following examples. For example, if the file uses UTF-32 (32-bit Unicode Transformation Format) encoding format, the following network parameters may be more effective:
  • Input sequence length 128 (if calculated according to 2KB/32, it should be 512, but it should not be too long; therefore, 128 is selected here); output sequence length: 128 (same as input sequence length); each step of the input vector during encoding: 32 rows ⁇ 1 column of binary; input vector for each step of decoding: 32 rows ⁇ 1 column of binary (need to use indicator function for post-processing); Basic LSTM Cells: 32.
  • a multi-layer structure can also be set up, so that the network error correction capability is enhanced.
  • Each box in Figure 9 is an LSTM Cell set. You can also set up more LSTM Cells accordingly to form a three-layer, four-layer, or even more layer network structure, thereby improving the accuracy of LSTM correction.
  • LSTM training collect as comprehensive as possible, conform to the error distribution, and multiple types of data to train the constructed initial LSTM, so as to obtain a general LSTM. Further, the training data is divided according to spatial neighbors or time before and after to obtain different training sets, and these two training sets are used to train the above-mentioned general LSTM respectively, so as to obtain two more targeted LSTMs. Such LSTMs are targeted Stronger, higher probability of successful error correction, also has its value in scenarios where data is extremely important.
  • this embodiment improves the error correction capability of the codec without increasing the redundant bits of the error correction code.
  • the LSTM can still assist its correct error correction and improve the error correction. Success rate.
  • the following describes a data error correction device provided in an embodiment of the present application.
  • the data error correction device described below and the data error correction method described above can be cross-referenced.
  • an embodiment of the present application discloses a data error correction device, including:
  • the obtaining module 1101 is used to obtain target data to be corrected
  • the error correction module 1102 is configured to perform error correction on the target data by using an error correction code to obtain the first data;
  • the judging module 1103 is used to judge whether the error correction of the first data is successful
  • the correction module 1104 is configured to correct the target data by using the target neural network to obtain the second data, determine the second data as the target data, and transmit the target data to the error correction module if the error correction of the first data is unsuccessful;
  • the determining module 1105 is configured to determine the first data as target data for completing the error correction if the error correction of the first data is successful.
  • the correction module includes:
  • the first separation unit is used to separate the data part and the check part of the target data
  • the correction unit is used to correct the data part by using the target neural network to obtain the second data.
  • the correction unit is specifically used for:
  • the data part of the target data is input into the target neural network in reverse order to obtain the second data.
  • the error correction module includes:
  • the second separation unit is used to separate the data part and the check part of the target data
  • the correction module is used to correct the erroneous data bits of the data part by using the error correction code to obtain the first data.
  • the correction module is specifically used for:
  • it further includes a generation module for generating the target neural network, and the generation module includes:
  • the construction unit is used to construct the initial neural network, the initial neural network includes: encoding subnet and decoding subnet;
  • the first training unit is used to input training data containing different types of data to train the initial neural network
  • the third determining unit is used to determine the current initial neural network as the target neural network if the error loss function of the current initial neural network converges to a stable state during the training process.
  • the generating module further includes:
  • the dividing unit is used to divide the training data into multiple data sets, and the data in each data set has an association relationship;
  • the second training unit is used for separately training the target neural network using each data set to obtain multiple target neural networks corresponding to each data set.
  • the correction module is specifically used for:
  • the data characteristics of the target data are determined, and the target neural network with the highest matching degree with the data characteristics is selected from the multiple target neural networks to correct the target data to obtain the second data.
  • this embodiment provides a data error correction device. After the device uses the error correction code to correct the target data to obtain the first data, if the first data error correction is not successful, then the target neural network is used to correct the error. The target data is corrected to obtain the second data, and the second data is determined as the target data, so that the error correction code is used to correct the data again, so as to obtain the first data with successful error correction, then the first data is determined to be completed The correct target data is obtained by correcting the target data. It can be seen that the present application can use the error correction code to cooperate with the target neural network to correct the error data, thereby avoiding the error correction failure of the error correction code data and the data loss during the error correction process.
  • the following describes a data error correction device provided in an embodiment of the present application.
  • the data error correction device described below and the data error correction method and device described above can be cross-referenced.
  • an embodiment of the present application discloses a data error correction device, including:
  • the memory 1201 is used to store computer programs
  • the processor 1202 is configured to execute the computer program to implement the method disclosed in any of the foregoing embodiments.
  • the following introduces a readable storage medium provided by an embodiment of the present application.
  • the readable storage medium described below and the method, device, and device for data error correction described above can be cross-referenced.
  • a readable storage medium for storing a computer program, where the computer program implements the data error correction method disclosed in the foregoing embodiment when the computer program is executed by a processor.
  • the computer program implements the data error correction method disclosed in the foregoing embodiment when the computer program is executed by a processor.
  • the steps of the method or algorithm described in combination with the embodiments disclosed in this document can be directly implemented by hardware, a software module executed by a processor, or a combination of the two.
  • the software module can be placed in random access memory (RAM), internal memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disks, removable disks, CD-ROMs, or all areas in the technical field. Any other form of well-known readable storage medium.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Probability & Statistics with Applications (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • Software Systems (AREA)
  • Algebra (AREA)
  • Pure & Applied Mathematics (AREA)
  • Detection And Correction Of Errors (AREA)

Abstract

一种数据纠错方法、装置、设备及可读存储介质。本申请公开的方法包括：S11、获取待纠错的目标数据；S12、利用纠错码对目标数据进行纠错，获得第一数据；S13、判断第一数据是否纠错成功；若是，则执行S15；若否，则执行S14；S14、利用目标神经网络对目标数据进行修正，获得第二数据，将第二数据确定为目标数据，并执行S12；S15、将第一数据确定为完成纠错的目标数据。本申请可利用纠错码和目标神经网络相配合纠错数据，从而可避免纠错码数据纠错失败和纠错过程中的数据丢失。

Description

一种数据纠错方法、装置、设备及可读存储介质
本申请要求于2019年11月06日提交至中国专利局、申请号为201911077492.3、发明名称为“一种数据纠错方法、装置、设备及可读存储介质”的中国专利申请的优先权，其全部内容通过引用结合在本申请中。
技术领域
本申请涉及计算机编码技术领域，特别涉及一种数据纠错方法、装置、设备及可读存储介质。
背景技术
在数据传输和存储的过程中，因为环境、介质、距离、时间等因素的影响，数据难免会发生错误。为了识别和纠正数据中的错误数据，现有技术常使用纠错码(Error-Correcting Code)来纠正数据。
但是，各种纠错码的纠错能力是有限的。如：海明码(Hamming Code)最多可纠正一位错误；LDPC编码(Low-Density Parity-Check code)一般具有50～60bits/4KB的纠错能力。因此当数据中的错误数据的位数超出纠错码的纠错能力时，数据中的错误数据将不能被定位并纠正，会导致纠错失败，进而发生数据丢失。
因此，如何避免数据纠错过程中的纠错失败和数据丢失，是本领域技术人员需要解决的问题。
发明内容
有鉴于此，本申请的目的在于提供一种数据纠错方法、装置、设备及可读存储介质，以避免数据纠错过程中的纠错失败和数据丢失。其具体方案如下：
第一方面，本申请提供了一种数据纠错方法，包括：
S11、获取待纠错的目标数据；
S12、利用纠错码对目标数据进行纠错，获得第一数据；
S13、判断第一数据是否纠错成功；若是，则执行S15；若否，则执行S14；
S14、利用目标神经网络对目标数据进行修正，获得第二数据，将第二数据确定为目标数据，并执行S12；
S15、将第一数据确定为完成纠错的目标数据。
优选地，利用目标神经网络对目标数据进行修正，获得第二数据，包括：
分离目标数据的数据部分和校验部分；
利用目标神经网络对数据部分进行修正，获得第二数据。
优选地，利用目标神经网络对数据部分进行修正，获得第二数据，包括：
将数据部分倒序输入目标神经网络，获得第二数据。
优选地，利用纠错码对目标数据进行纠错，获得第一数据，包括：
分离目标数据的数据部分和校验部分；
利用纠错码对数据部分的出错数据位进行校正，获得第一数据。
优选地，利用目标神经网络对目标数据进行修正，获得第二数据，包括：
将目标数据倒序输入目标神经网络，获得第二数据。
优选地，目标神经网络的生成过程包括：
构建初始神经网络，初始神经网络包括：编码子网和解码子网；
输入包含不同类型数据的训练数据训练初始神经网络；
若训练过程中，当前初始神经网络的错误损失函数收敛到稳定状态，则将当前初始神经网络确定为目标神经网络。
优选地，将当前初始神经网络确定为目标神经网络之后，还包括：
将训练数据划分为多个数据集合，每个数据集合中的数据具有关联关系；
利用每个数据集合分别训练目标神经网络，获得与每个数据集合对应的多个目标神经网络。
优选地，利用目标神经网络对目标数据进行修正，获得第二数据，包括：
确定目标数据的数据特征，并在多个目标神经网络中选择与数据特征匹配度最高的目标神经网络对目标数据进行修正，获得第二数据。
第二方面，本申请提供了一种数据纠错装置，包括：
获取模块，用于获取待纠错的目标数据；
纠错模块，用于利用纠错码对目标数据进行纠错，获得第一数据；
判断模块，用于判断第一数据是否纠错成功；
修正模块，用于若第一数据纠错未成功，则利用目标神经网络对目标数据进行修正，获得第二数据，将第二数据确定为目标数据，并将目标数据传输至纠错模块；
确定模块，用于若第一数据纠错成功，则将第一数据确定为完成纠错的目标数据。
第三方面，本申请提供了一种数据纠错设备，包括：
存储器，用于存储计算机程序；
处理器，用于执行计算机程序，以实现前述公开的数据纠错方法。
第四方面，本申请提供了一种可读存储介质，用于保存计算机程序，其中，计算机程序被处理器执行时实现前述公开的数据纠错方法。
通过以上方案可知，本申请提供了一种数据纠错方法，包括：S11、获取待纠错的目标数据；S12、利用纠错码对目标数据进行纠错，获得第一数据；S13、判断第一数据是否纠错成功；若是，则执行S15；若否，则执行S14；S14、利用目标神经网络对目标数据进行修正，获得第二数据，将第二数据确定为目标数据，并执行S12；S15、将第一数据确定为完成纠错的目标数据。
可见，该方法在利用纠错码对目标数据进行纠错，若纠错未成功，那么转而利用目标神经网络对目标数据进行修正，从而获得第二数据，将第二数据确定为目标数据，以便再次利用纠错码对数据进行纠错，从而获得纠错成功的第一数据，那么将第一数据确定为完成纠错的目标数据，也就获得了正确的目标数据。由此可见，本申请可利用纠错码和目标神经网络相配合纠错数据，从而可避免纠错码数据纠错失败和纠错过程中的数据丢失。
相应地，本申请提供的一种数据纠错装置、设备及可读存储介质，也同样具有上述技术效果。
附图说明
为了更清楚地说明本申请实施例或现有技术中的技术方案，下面将对实施例或现有技术描述中所需要使用的附图作简单地介绍，显而易见地，下面描述中的附图仅仅是本申请的实施例，对于本领域普通技术人员来讲，在不付出创造性劳动的前提下，还可以根据提供的附图获得其他的附图。
图1为本申请公开的第一种数据纠错方法流程图；
图2为本申请公开的第二种数据纠错方法流程图；
图3为本申请公开的第三种数据纠错方法流程图；
图4为本申请公开的第一种错误数据分布示意图；
图5为本申请公开的第二种错误数据分布示意图；
图6为本申请公开的一种LSTM纠错数据示意图；
图7为本申请公开的一种LSTM Cell示意图；
图8为本申请公开的一种LSTM倒序纠错数据示意图；
图9为本申请公开的一种两层结构的LSTM网络示意图；
图10为本申请公开的第四种数据纠错方法流程图；
图11为本申请公开的一种数据纠错装置示意图；
图12为本申请公开的一种数据纠错设备示意图。
具体实施方式
下面将结合本申请实施例中的附图，对本申请实施例中的技术方案进行清楚、完整地描述，显然，所描述的实施例仅仅是本申请一部分实施例，而不是全部的实施例。基于本申请中的实施例，本领域普通技术人员在没有做出创造性劳动前提下所获得的所有其他实施例，都属于本申请保护的范围。
目前，仅利用纠错码纠正数据，数据中的错误数据不能被全部定位并纠正，会导致纠错失败，进而发生数据丢失。为此，本申请提供了一种数据纠错方案，能够避免数据纠错过程中的纠错失败和数据丢失。
参见图1所示，本申请实施例公开了第一种数据纠错方法，包括：
S11、获取待纠错的目标数据；
S12、利用纠错码对目标数据进行纠错，获得第一数据；
S13、判断第一数据是否纠错成功；若是，则执行S15；若否，则执行S14；
S14、利用目标神经网络对目标数据进行修正，获得第二数据，将第二数据确定为目标数据，并执行S12；
S15、将第一数据确定为完成纠错的目标数据。
需要说明的是，纠错码可以为任意种类的纠错码，如：海明码、BCH(Bose Ray-Chaudhuri Hocquenghem)、极化码(Polar Code)等。目标神经网络的也可以为任意种类的神经网络，如：RNN(循环神经网络，Recurrent Neural Networks)、GRU(门控循环单元，Gated Recurrent Unit)、FCN(全连接网络，Fully-Connected Networks)、CNN(卷积神经网络，Convolutional Neural Networks)、Random Forest(随机森林算法)、SVM(支持向量机，Support Vector Machine)、Logical Regression(逻辑回归)等。
具体的，若第一数据纠错成功，则表明第一数据是完成纠错的目标数据，否则，转而利用目标神经网络对目标数据进行修正，以降低目标数据中的错误数据的位数，从而使数据可用纠错码进行纠错。目前编码的译码器一般采用迭代译码的设计方案，如果迭代译码的计算可以顺利地收敛计算完成，即译码成功；反之，译码失败。因此判断纠错成功与否的关键在于：迭代译码的计算能否完成收敛，具体可参见现有技术，本说明书在此不再赘述。
在一种具体实施方式中，利用目标神经网络对目标数据进行修正，获得第二数据，包括：分离目标数据的数据部分和校验部分；利用目标神经网络对数据部分进行修正，获得第二数据。
在一种具体实施方式中，利用纠错码对目标数据进行纠错，获得第一数据，包括：分离目标数据的数据部分和校验部分；利用纠错码对数据部分的出错数据位进行校正，获得第一数据。
需要说明的是，在数据传输或存储时，每个数据包括数据部分与校验部分。其中，数据部分的数据及错误模式相关性更大，并且在一个数据中的比重较大(一般超过90％)，而校验部分对于同一系统中的每个数据而言，都是相同或类似的。因此可利用目标神经网络或纠错码仅对数据的数据部分进行处理，在处理完成后，将处理后的结果与原来的校验部分再合 并，从而获得处理后的数据。
在一种具体实施方式中，利用目标神经网络对目标数据进行修正，获得第二数据，包括：将目标数据倒序输入目标神经网络，获得第二数据。将目标数据倒序输入目标神经网络能够使待处理数据的位置与该数据被编码的位置更接近，如此便于信息记录和编码，具体请参见下述实施例的介绍。
可见，本申请实施例在利用纠错码对目标数据进行纠错获得第一数据后，若第一数据纠错未成功，那么转而利用目标神经网络对目标数据进行修正，从而获得第二数据，将第二数据确定为目标数据，以便再次利用纠错码对数据进行纠错，从而获得纠错成功的第一数据，那么将第一数据确定为完成纠错的目标数据，也就获得了正确的目标数据。由此可见，本申请可利用纠错码和目标神经网络相配合纠错数据，从而可避免纠错码数据纠错失败和纠错过程中的数据丢失。
参见图2所示，本申请实施例公开了第二种数据纠错方法，包括：
S201、获取待纠错的目标数据；
S202、利用纠错码对目标数据进行纠错，获得第一数据；
S203、判断第一数据是否纠错成功；若是，则执行S206；若否，则执行S204；
S204、分离目标数据的数据部分和校验部分；
S205、将数据部分倒序输入目标神经网络，获得第二数据，将第二数据确定为目标数据，并执行S202；
具体的，将数据部分倒序输入目标神经网络，得到的输出数据与校验部分重新组合，从而获得第二数据。
S206、将第一数据确定为完成纠错的目标数据。
在本实施例中，当获得第二数据后，可进一步判断第二数据是否纠错成功；若是，则可直接将第二数据确定为完成纠错的目标数据；若否，则将第二数据确定为目标数据，并执行S202。
在本实施例中，将目标数据的数据部分倒序输入目标神经网络能够使待处理数据的位置与该数据被编码的位置更接近，如此便于信息记录和编 码。例如：待处理的目标数据的数据部分为：“ABSD”，将其倒序后，即可得到“DSBA”，那么将“DSBA”输入目标神经网络，目标神经网络按照“DSBA”的顺序编码数据，但按照“ABSD”的顺序解码数据，这样对于“A”来说，其编码位置与解码位置就比较近。同样，“B、S、D”编码位置与各自的解码位置也比较近。按照常用的编码和解码方式，按照“ABSD”的顺序编码和解码数据，对于其中的每个字符来说，编码位置与解码位置之间都隔着3个字符，所以不便于字符信息的记录。需要说明的是，上述举例仅是为了说明该方式的技术效果，实际应用过程中数据的编码和解码不应如此简单理解。
在一种具体实施方式中，利用纠错码对目标数据进行纠错，获得第一数据，包括：分离目标数据的数据部分和校验部分；利用纠错码对数据部分的出错数据位进行校正，获得第一数据。
需要说明的是，在数据传输或存储时，每个数据包括数据部分与校验部分。其中，数据部分的数据及错误模式相关性更大，并且在一个数据中的比重较大(一般超过90％)，而校验部分对于同一系统中的每个数据而言，都是相同或类似的。因此可利用纠错码仅对数据的数据部分进行处理，在处理完成后，将处理后的结果与原来的校验部分再合并，从而获得修正后的数据。
需要说明的是，本实施例中的其他实现步骤与上述实施例相同或类似，故本实施例在此不再赘述。
由上可见，本实施例在利用纠错码对目标数据进行纠错获得第一数据后，若第一数据纠错未成功，那么转而利用目标神经网络对目标数据进行修正，从而获得第二数据，将第二数据确定为目标数据，以便再次利用纠错码对数据进行纠错，从而获得纠错成功的第一数据，那么将第一数据确定为完成纠错的目标数据，也就获得了正确的目标数据。由此可见，本申请可利用纠错码和目标神经网络相配合纠错数据，从而可避免纠错码数据纠错失败和纠错过程中的数据丢失。
基于上述任意实施例，需要说明的是，目标神经网络的生成过程包括：构建初始神经网络，初始神经网络包括：编码子网和解码子网；利用包含 不同类型数据的训练数据训练初始神经网络；若训练过程中，初始神经网络的错误损失函数(比如：交叉熵损失函数)收敛到稳定状态，则将当前初始神经网络确定为目标神经网络。
具体的，错误损失函数也叫loss函数。交叉熵损失函数可在神经网络中作为损失函数，其使用sigmoid函数在梯度下降时能避免均方误差损失函数学习速率降低的问题，因为学习速率可以被输出的误差所控制。具体的错误损失函数可参照现有技术的介绍，本说明书不再赘述。
需要说明的是，利用包含不同类型数据的训练数据训练获得的目标神经网络具有良好的通用性，若想要获得针对性更好的神经网络，可按照在将当前初始神经网络确定为目标神经网络之后，还包括：将训练数据划分为多个数据集合，每个数据集合中的数据具有关联关系；利用每个数据集合分别训练目标神经网络，获得与每个数据集合对应的多个目标神经网络。如此获得的每个目标神经网络将具有特有的特征，如：若数据集合中的数据具有空间上相邻的关联关系，那么由该数据集合训练获得的目标神经网络将能够对空间上相邻数据具有更好的修正效果，因此该目标神经网络的特征可确定为空间上相邻。因此当待处理数据是空间上相邻的数据时，该数据的数据特征与该目标神经网络的匹配度就是最高的，那么该目标神经网络对该数据的处理效果更好。目标神经网络可降低数据中的错误数据量，从而为纠错码的纠错成功提供了保障。
其中，关联关系包括：空间上相邻和/或时间上相邻，还可以包括：业务应用上关联、数据格式相关等。
在一种具体实施方式中，利用目标神经网络对目标数据进行修正，获得第二数据，包括：确定目标数据的数据特征，并在多个目标神经网络中选择与数据特征匹配度最高的目标神经网络对目标数据进行修正，获得第二数据。具体的，数据的数据特征与目标神经网络的匹配度可根据数据的数据特征和目标神经网络的网络特征计算获得。
在神经网络中，编码子网和解码子网独立存在，其中的编码子网可实现共享。也就是多个目标神经网络共享一个编码子网，但各自又具有特定的解码子网，如此在同时使用多个神经网络修正同一个数据时，只需进行一次编码的计算，从而避免重复编码。
按照本申请提供的数据纠错方法可实施如下实例。
在本实施例中，纠错码选用LDPC，神经网络选用LSTM(Long Short-Term Memory，长短期记忆网络)，二者结合纠错数据的过程请参见图3。
图3所示的纠错过程利用固态硬盘的主控芯片实现。经实验验证，仅利用LDPC和固态硬盘的主控芯片，输出数据中的数据错误位为90errors/4KB，具体请参见图4。而图3所示的纠错过程输出数据中没有错误数据位，其中LSTM输出数据中错误数据位减少为42个，具体请参见图5。可见，LDPC无法纠正的错误，本实施例能够解决。图4和图5中的黑色小点表示错误数据位。
在本实施例中，LSTM修正数据的过程请参见图6。在图6中，LSTM将“ABSD”修正为“ABCD”。假设一个文件用ASCII格式保存了“ABCD”四个字母，一共是32bits。由于在SSD中保存时间过久，再读出时，变为“ABSD”。首先，LSTM通过编码识别到这是一个四个字母的字符串，但是“ABSD”是很少连续出现在文件中的，这有可能是读取错误引起的；那么，接下来在解码过程中，LSTM将“ABSD”改为了“ABCD”。图6中的LSTM编码可实现编码复用，适应不同类型错误纠错编解码配对的情况。
其中，图6中的每个LSTM Cell请参见图7。在图7中，x t和h t-1为LSTM Cell的输入数据，x t为当前LSTM Cell待处理数据，h t-1是前一个LSTM Cell输出的结果，输入数据还包括前一个LSTM Cell输出的c t-1。图7中“Input Gate”、“Forget Gate”、“Cell Gate”和“Output Gate”为当前LSTM Cell的四个处理门，这四个处理门分别处理x t和h t-1。“Input Gate”的处理结果i t与“Cell Gate”的处理结果c t进行内积运算，获得第一个内积结果，“Forget Gate”的处理结果f t与前一个LSTM Cell输出的c t-1进行内积运算，获得第二个内积结果，进而第一个内积结果与第二个内积结果相加，获得一个新的c t，这个新的c t经过激活函数后，与“Output Gate”的处理结果o t进行内积，从而获得当前LSTM Cell输出的h t。其中，当前LSTM Cell输出的h t和c t会同时输入LSTM Cell下一个计算循环。
图7中的
Figure PCTCN2020102018-appb-000001
表示向量矩阵乘法运算，
Figure PCTCN2020102018-appb-000002
表示相加，
Figure PCTCN2020102018-appb-000003
表示哈达玛积(Hadamard Product)，
Figure PCTCN2020102018-appb-000004
表示激活函数。W x为x t在当前处理门中的权重值，W h为h t-1当前处理门中的权重值。例如，W xi为x t在“Input Gate”中的权重值，W hi为h t-1“Input Gate”中的权重值，其他以此类推。LSTM中的每个LSTM Cell中有四个处理门。
若将“ABSD”倒序输入图6所示的网络，则编码中的信息位“A”与解码中的“A”更近；同样，这样编码中的信息位“B”与解码中的“B”也更近。倒序输入的场景请参见图8。其中，图6、图8和图9中的每个框为一个LSTM Cell集合。图6、图8和图9中的每个LSTM Cell的结构均可参见图7。
具体的，LSTM的网络参数可按照如下示例进行选择。例如：文件采用的是UTF-32(32-bit Unicode Transformation Format)编码格式，那么采用下面的网络参数可能会更有效：
输入数列长度：128(如果按照2KB/32计算，应为512，但不宜过长；因此，这里选取128)；输出数列长度：128(同输入数列长度)；编码时每步input向量：32行×1列binary；解码时每步input向量：32行×1列binary(需要使用指示函数(indicator function)进行后处理)；Basic LSTM Cells：32个。
其中，还可以设置多层结构，这样网络纠错能力得到增强。如图9所示的两层结构。图9中的每个框即为一个LSTM Cell集合。还可以据此设置更多LSTM Cell，以形成三层、四层、甚至更多层的网络结构，从而提高LSTM的修正精度。
LSTM的训练：收集尽可能全面的、符合错误分布的、多类型的数据对构建的初始LSTM进行训练，从而获得通用的LSTM。进一步地，将训练数据按照空间邻居或时间前后进行划分，得到不同的训练集，用这两个训练集分别训练上述通用的LSTM，从而获得更有针对性的两个LSTM，这样的LSTM针对性更强，成功纠错的概率更高，在数据极其重要的场景下，亦有其价值。
其中，还可以在纠错过程中用待纠错的数据对当前使用的LSTM进行再训练，具体请参见图10。
可见，本实施例在不增加纠错码冗余位的情况下，提高了编解码的纠错能力，在编解码纠错失败的情况下，LSTM依然可以辅助其正确纠错，提高了纠错成功率。
下面对本申请实施例提供的一种数据纠错装置进行介绍，下文描述的一种数据纠错装置与上文描述的一种数据纠错方法可以相互参照。
参见图11所示，本申请实施例公开了一种数据纠错装置，包括：
获取模块1101，用于获取待纠错的目标数据；
纠错模块1102，用于利用纠错码对目标数据进行纠错，获得第一数据；
判断模块1103，用于判断第一数据是否纠错成功；
修正模块1104，用于若第一数据纠错未成功，则利用目标神经网络对目标数据进行修正，获得第二数据，将第二数据确定为目标数据，并将目标数据传输至纠错模块；
确定模块1105，用于若第一数据纠错成功，则将第一数据确定为完成纠错的目标数据。
在一种具体实施方式中，修正模块包括：
第一分离单元，用于分离目标数据的数据部分和校验部分；
修正单元，用于利用目标神经网络对数据部分进行修正，获得第二数据。
在一种具体实施方式中，修正单元具体用于：
将目标数据的数据部分倒序输入目标神经网络，获得第二数据。
在一种具体实施方式中，纠错模块包括：
第二分离单元，用于分离目标数据的数据部分和校验部分；
校正模块，用于利用纠错码对数据部分的出错数据位进行校正，获得第一数据。
在一种具体实施方式中，修正模块具体用于：
将目标数据倒序输入目标神经网络，获得第二数据。
在一种具体实施方式中，还包括用于生成目标神经网络的生成模块，该生成模块包括：
构建单元，用于构建初始神经网络，初始神经网络包括：编码子网和 解码子网；
第一训练单元，用于输入包含不同类型数据的训练数据训练初始神经网络；
第三确定单元，用于若训练过程中，当前初始神经网络的错误损失函数收敛到稳定状态，则将当前初始神经网络确定为目标神经网络。
在一种具体实施方式中，生成模块还包括：
划分单元，用于将训练数据划分为多个数据集合，每个数据集合中的数据具有关联关系；
第二训练单元，用于利用每个数据集合分别训练目标神经网络，获得与每个数据集合对应的多个目标神经网络。
在一种具体实施方式中，修正模块具体用于：
确定目标数据的数据特征，并在多个目标神经网络中选择与数据特征匹配度最高的目标神经网络对目标数据进行修正，获得第二数据。
其中，关于本实施例中各个模块、单元更加具体的工作过程可以参考前述实施例中公开的相应内容，在此不再进行赘述。
可见，本实施例提供了一种数据纠错装置，该装置在利用纠错码对目标数据进行纠错获得第一数据后，若第一数据纠错未成功，那么转而利用目标神经网络对目标数据进行修正，从而获得第二数据，将第二数据确定为目标数据，以便再次利用纠错码对数据进行纠错，从而获得纠错成功的第一数据，那么将第一数据确定为完成纠错的目标数据，也就获得了正确的目标数据。由此可见，本申请可利用纠错码和目标神经网络相配合纠错数据，从而可避免纠错码数据纠错失败和纠错过程中的数据丢失。
下面对本申请实施例提供的一种数据纠错设备进行介绍，下文描述的一种数据纠错设备与上文描述的一种数据纠错方法及装置可以相互参照。
参见图12所示，本申请实施例公开了一种数据纠错设备，包括：
存储器1201，用于保存计算机程序；
处理器1202，用于执行所述计算机程序，以实现上述任意实施例公开的方法。
下面对本申请实施例提供的一种可读存储介质进行介绍，下文描述的一种可读存储介质与上文描述的一种数据纠错方法、装置及设备可以相互参照。
一种可读存储介质，用于保存计算机程序，其中，所述计算机程序被处理器执行时实现前述实施例公开的数据纠错方法。关于该方法的具体步骤可以参考前述实施例中公开的相应内容，在此不再进行赘述。
本申请涉及的“第一”、“第二”、“第三”、“第四”等(如果存在)是用于区别类似的对象，而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换，以便这里描述的实施例能够以除了在这里图示或描述的内容以外的顺序实施。此外，术语“包括”和“具有”以及他们的任何变形，意图在于覆盖不排他的包含，例如，包含了一系列步骤或单元的过程、方法或设备不必限于清楚地列出的那些步骤或单元，而是可包括没有清楚地列出的或对于这些过程、方法或设备固有的其它步骤或单元。
需要说明的是，在本申请中涉及“第一”、“第二”等的描述仅用于描述目的，而不能理解为指示或暗示其相对重要性或者隐含指明所指示的技术特征的数量。由此，限定有“第一”、“第二”的特征可以明示或者隐含地包括至少一个该特征。另外，各个实施例之间的技术方案可以相互结合，但是必须是以本领域普通技术人员能够实现为基础，当技术方案的结合出现相互矛盾或无法实现时应当认为这种技术方案的结合不存在，也不在本申请要求的保护范围之内。
本说明书中各个实施例采用递进的方式描述，每个实施例重点说明的都是与其它实施例的不同之处，各个实施例之间相同或相似部分互相参见即可。
结合本文中所公开的实施例描述的方法或算法的步骤可以直接用硬件、处理器执行的软件模块，或者二者的结合来实施。软件模块可以置于随机存储器(RAM)、内存、只读存储器(ROM)、电可编程ROM、电可擦除可编程ROM、寄存器、硬盘、可移动磁盘、CD-ROM、或技术领域内所公知的任意其它形式的可读存储介质中。
本文中应用了具体个例对本申请的原理及实施方式进行了阐述，以上实施例的说明只是用于帮助理解本申请的方法及其核心思想；同时，对于本领域的一般技术人员，依据本申请的思想，在具体实施方式及应用范围上均会有改变之处，综上所述，本说明书内容不应理解为对本申请的限制。

Claims (10)

  1. 一种数据纠错方法，其特征在于，包括：
    S11、获取待纠错的目标数据；
    S12、利用纠错码对所述目标数据进行纠错，获得第一数据；
    S13、判断所述第一数据是否纠错成功；若是，则执行S15；若否，则执行S14；
    S14、利用目标神经网络对所述目标数据进行修正，获得第二数据，将所述第二数据确定为所述目标数据，并执行S12；
    S15、将所述第一数据确定为完成纠错的目标数据。
  2. 根据权利要求1所述的数据纠错方法，其特征在于，所述利用目标神经网络对所述目标数据进行修正，获得第二数据，包括：
    分离所述目标数据的数据部分和校验部分；
    利用所述目标神经网络对所述数据部分进行修正，获得所述第二数据。
  3. 根据权利要求2所述的数据纠错方法，其特征在于，所述利用所述目标神经网络对所述数据部分进行修正，获得所述第二数据，包括：
    将所述数据部分倒序输入所述目标神经网络，获得所述第二数据。
  4. 根据权利要求1所述的数据纠错方法，其特征在于，所述利用目标神经网络对所述目标数据进行修正，获得第二数据，包括：
    将所述目标数据倒序输入所述目标神经网络，获得所述第二数据。
  5. 根据权利要求1至4任一项所述的数据纠错方法，其特征在于，所述目标神经网络的生成过程包括：
    构建初始神经网络，所述初始神经网络包括：编码子网和解码子网；
    输入包含不同类型数据的训练数据训练所述初始神经网络；
    若训练过程中，当前初始神经网络的错误损失函数收敛到稳定状态，则将当前初始神经网络确定为所述目标神经网络。
  6. 根据权利要求5所述的数据纠错方法，其特征在于，所述将当前初始神经网络确定为所述目标神经网络之后，还包括：
    将所述训练数据划分为多个数据集合，每个数据集合中的数据具有关联关系；
    利用每个数据集合分别训练所述目标神经网络，获得与每个数据集合 对应的多个目标神经网络。
  7. 根据权利要求6所述的数据纠错方法，其特征在于，所述利用目标神经网络对所述目标数据进行修正，获得第二数据，包括：
    确定所述目标数据的数据特征，并在所述多个目标神经网络中选择与所述数据特征匹配度最高的目标神经网络对所述目标数据进行修正，获得所述第二数据。
  8. 一种数据纠错装置，其特征在于，包括：
    获取模块，用于获取待纠错的目标数据；
    纠错模块，用于利用纠错码对所述目标数据进行纠错，获得第一数据；
    判断模块，用于判断所述第一数据是否纠错成功；
    修正模块，用于若所述第一数据纠错未成功，则利用目标神经网络对所述目标数据进行修正，获得第二数据，将所述第二数据确定为所述目标数据，并将所述目标数据传输至所述纠错模块；
    确定模块，用于若所述第一数据纠错成功，则将所述第一数据确定为完成纠错的目标数据。
  9. 一种数据纠错设备，其特征在于，包括：
    存储器，用于存储计算机程序；
    处理器，用于执行所述计算机程序，以实现如权利要求1至7任一项所述的数据纠错方法。
  10. 一种可读存储介质，其特征在于，用于保存计算机程序，其中，所述计算机程序被处理器执行时实现如权利要求1至7任一项所述的数据纠错方法。
PCT/CN2020/102018 2019-11-06 2020-07-15 一种数据纠错方法、装置、设备及可读存储介质 WO2021088403A1 (zh)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US17/733,165 US12107601B2 (en) 2019-11-06 2022-04-29 Data error correction method, apparatus, device, and readable storage medium

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN201911077492.3 2019-11-06
CN201911077492.3A CN110798225B (zh) 2019-11-06 2019-11-06 一种数据纠错方法、装置、设备及可读存储介质

Related Child Applications (1)

Application Number Title Priority Date Filing Date
US17/733,165 Continuation US12107601B2 (en) 2019-11-06 2022-04-29 Data error correction method, apparatus, device, and readable storage medium

Publications (1)

Publication Number Publication Date
WO2021088403A1 true WO2021088403A1 (zh) 2021-05-14

Family

ID=69442922

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2020/102018 WO2021088403A1 (zh) 2019-11-06 2020-07-15 一种数据纠错方法、装置、设备及可读存储介质

Country Status (3)

Country Link
US (1) US12107601B2 (zh)
CN (1) CN110798225B (zh)
WO (1) WO2021088403A1 (zh)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN110798225B (zh) * 2019-11-06 2024-07-23 深圳大普微电子科技有限公司 一种数据纠错方法、装置、设备及可读存储介质
CN114389620A (zh) 2020-10-20 2022-04-22 华为技术有限公司 一种译码方法和装置

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102124527A (zh) * 2008-05-16 2011-07-13 弗森-艾奥公司 用于检测和替代失效的数据存储器的装置、系统和方法
CN102881334A (zh) * 2012-09-14 2013-01-16 东南大学 核探测机器人控制系统中数据存储的纠错方法
CN103019883A (zh) * 2012-12-31 2013-04-03 华为技术有限公司 一种内存纠错方法及系统
CN103377664A (zh) * 2012-04-27 2013-10-30 日立民用电子株式会社 数据记录再现装置和数据库装置
US20190311254A1 (en) * 2019-06-21 2019-10-10 Intel Corporation Technologies for performing in-memory training data augmentation for artificial intelligence
CN110798225A (zh) * 2019-11-06 2020-02-14 深圳大普微电子科技有限公司 一种数据纠错方法、装置、设备及可读存储介质

Family Cites Families (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5253328A (en) * 1989-11-17 1993-10-12 Microelectronics & Computer Technology Corp. Neural-network content-addressable memory
TW201521676A (zh) * 2013-12-13 2015-06-16 Nat Inst Chung Shan Science & Technology 一種使用類神經網路產生判斷麻醉意識清醒程度指標的方法
CN104835535B (zh) * 2015-05-15 2017-12-12 华中科技大学 一种固态盘自适应纠错方法与系统
JP6606039B2 (ja) * 2016-09-09 2019-11-13 東芝メモリ株式会社 メモリシステムおよび制御方法
US11030530B2 (en) * 2017-01-09 2021-06-08 Onu Technology Inc. Method for unsupervised sequence learning using reinforcement learning and neural networks
US11308397B2 (en) * 2018-02-16 2022-04-19 Ilya Sorokin System and method of training a neural network
KR102659892B1 (ko) * 2018-07-24 2024-04-24 삼성전자주식회사 객체 인식 장치, 이를 포함하는 전자 장치 및 객체 인식 방법
TWI684106B (zh) * 2018-09-28 2020-02-01 大陸商深圳衡宇芯片科技有限公司 訓練人工智慧更正儲存裝置的對數概度比的方法
US10740165B2 (en) * 2018-12-03 2020-08-11 Western Digital Technologies, Inc. Extending the error correction capability of a device using a neural network
CN110058956A (zh) * 2019-04-22 2019-07-26 广东工业大学 一种nand闪存读取方法、系统及电子设备和存储介质
US11080152B2 (en) * 2019-05-15 2021-08-03 Western Digital Technologies, Inc. Optimized neural network data organization

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN102124527A (zh) * 2008-05-16 2011-07-13 弗森-艾奥公司 用于检测和替代失效的数据存储器的装置、系统和方法
CN103377664A (zh) * 2012-04-27 2013-10-30 日立民用电子株式会社 数据记录再现装置和数据库装置
CN102881334A (zh) * 2012-09-14 2013-01-16 东南大学 核探测机器人控制系统中数据存储的纠错方法
CN103019883A (zh) * 2012-12-31 2013-04-03 华为技术有限公司 一种内存纠错方法及系统
US20190311254A1 (en) * 2019-06-21 2019-10-10 Intel Corporation Technologies for performing in-memory training data augmentation for artificial intelligence
CN110798225A (zh) * 2019-11-06 2020-02-14 深圳大普微电子科技有限公司 一种数据纠错方法、装置、设备及可读存储介质

Also Published As

Publication number Publication date
CN110798225B (zh) 2024-07-23
US20220255558A1 (en) 2022-08-11
CN110798225A (zh) 2020-02-14
US12107601B2 (en) 2024-10-01

Similar Documents

Publication Publication Date Title
US12107601B2 (en) Data error correction method, apparatus, device, and readable storage medium
TWI710220B (zh) 使用硬選取硬解碼模式下的解碼器產生軟資訊的方法
CN104282340B (zh) 一种固态盘闪存芯片阈值电压感知方法及系统
US10348334B2 (en) Reducing a decoding error floor by post-processing codewords encoded by binary symmetry-invariant product codes
CN113168560A (zh) 用于对编码数据进行解码的神经网络和系统
KR20110037842A (ko) 메모리 시스템 및 메모리 시스템의 제어 방법
CN110768751B (zh) 译码的方法与电路、存储器控制器、储存装置、控制器
CN103297061A (zh) 决定同位元检查阵列的方法以及快闪存储器系统
EP3629482A1 (en) Decoding data using decoders and neural networks
WO2022127289A1 (zh) 基于高斯消元进行校验恢复的方法、系统、设备及介质
WO2008043282A1 (fr) Procédé et système de décodage de correction d'effacement ldpc
US11418219B2 (en) Learning device
KR20160084232A (ko) 메모리 장치, 메모리 시스템 및 메모리 장치의 동작 방법
CN115858235A (zh) 循环冗余检验处理方法及装置、电路、电子设备和介质
US10908994B2 (en) Memory system and method of controlling nonvolatile memory
CN118508976A (zh) 编码方法、译码方法、编码器、译码器及可读存储介质
WO2012092902A2 (zh) 一种译码方法和译码装置
TW202024907A (zh) 資料處理
US20240296090A1 (en) Interleaved reed-solomon (irs) with collaborative decoding
CN103986476A (zh) 一种用于二维图形码的级联纠错编码方法和装置
CN113595560B (zh) 一种信息纠错方法、装置、设备及存储介质
CN116662063B (zh) 一种闪存的纠错配置方法、纠错方法、系统、设备及介质
US12119841B2 (en) G-LDPC decoder and G-LDPC decoding method
US20240313806A1 (en) Deep neural network implementation for concatenated codes
US20220190846A1 (en) Single error correct double error detect (secded) error coding with burst error detection capability

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20885595

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20885595

Country of ref document: EP

Kind code of ref document: A1