Embodiment
Hereinafter will and combine embodiment to specify the present invention with reference to accompanying drawing.Need to prove that under the situation of not conflicting, embodiment and the characteristic among the embodiment among the application can make up each other.
A kind of decoding processing method is provided in the present embodiment, and Fig. 1 is the flow chart according to the decoding processing method of the embodiment of the invention, and is as shown in Figure 1, and this flow process comprises the steps:
Step S102 confirms to comprise the data to be decoded of former data of part and checking data, wherein, and the former data that this checking data correspondence is lost;
Step S104; According to the above-mentioned checking data and first generator matrix that is used to encode; Obtain the little generator matrix (and this your pupil become inverse of a matrix matrix) corresponding with above-mentioned former data of losing, and the new to be decoded data corresponding with above-mentioned former data of losing;
Step S106 according to above-mentioned new data to be decoded and above-mentioned little generator matrix (perhaps the your pupil becomes the inverse of a matrix matrix), obtains the former data of losing.
Pass through above-mentioned steps; Only obtain corresponding little generator matrix to the data of losing, and corresponding new data to be decoded, thereby the data that obtain losing; Promptly irrelevant with the quantity of data fragmentation in the process of calculating inverse matrix; Only relevant with the verification burst quantity of using, and in the Code And Decode process, use the memory cache Galois field multiply to replace original double meter method.With respect to original method; Solved not only that the decoding method based on the RS correcting and eleting codes exists the encoding and decoding amount of calculation huge in the correlation technique; Speed is slow; And the low problem of effective rate of utilization of system storage, and, reached the effect of the effective rate of utilization of the treatment effeciency that improved encoding and decoding and system's storage.
Based on checking data and above-mentioned first generator matrix that is used to encode, obtain the little generator matrix corresponding and can adopt more processing mode, for example with the former data of losing; For some simple little generator matrixes,, can be directed against checking data and above-mentioned first generator matrix according to rational mathematical logic reasoning; Extract little generator matrix in first generator matrix artificially,, also need obtain according to tight mathematical programming certainly for some complicated little generator matrixes; More excellent ground; Can adopt following processing mode: treat the decoded data processing of standardizing, wherein, this normalization is handled and is comprised: the position of the former data of reserve part in data to be decoded and the position consistency of the former data of part in coding initial data before; And with checking data be filled in the data to be decoded rest position (wherein; The order of above-mentioned filling can be to fill successively, also can inverted order fill, and concrete operations do not limit); Obtain and the second corresponding generator matrix of data to be decoded through standardizing after handling; Wherein, the data of importing during coding and this second generator matrix multiply each other can obtain standardizing and handle data to be decoded afterwards, based on above-mentioned second generator matrix and checking data, obtain the little generator matrix corresponding with the former data of losing.Adopt such processing mode, go for several scenes, and operating principle is simple, also more or less freely during processing.
When implementing, when adopting above-mentioned tight mathematical programming to handle, obtaining second generator matrix also can be according to following comparatively simple processing mode: keep in the unit matrix of first generator matrix columns certificate with former data respective column; In the check column of the unit matrix of first generator matrix, fill the checking data after the intact former data respective column of above-mentioned reservation.In addition, according to second generator matrix and checking data, comparatively simple processing mode below the step of obtaining the little generator matrix corresponding with the former data of losing also can adopt: in second generator matrix, extract row and the row that checking data occurs; The row that the checking data that extracts occurs is formed the corresponding little generator matrix of losing of initial data with row.
After obtaining little generator matrix; Relation between checking data that can comprise according to little generator matrix and the corresponding initial data of losing judges corresponding your pupil becomes matrix inversion matrix file whether to exist, if exist; Through reading this document, directly obtain the your pupil and become the inverse of a matrix matrix; If do not exist,, calculate the your pupil and become the inverse of a matrix matrix, and be written to the inverse matrix content in the respective file through the inverse matrix computational methods.The inverse matrix Galois Field array that the your pupil becomes 0 to 65535 formation in each element and Galois Field in the matrix inversion matrix after obtaining the your pupil and becoming the matrix inversion matrix data, can also be set up in more excellent ground, is used for decoding subsequently.
Preferably,, obtain after the little generator matrix corresponding, also can obtain the new new to be decoded data corresponding through variety of way with former data that lose with the former data of losing according to the checking data and first generator matrix that is used to encode.The embodiment of the invention provides a kind of processing mode, promptly obtains the new to be decoded data corresponding with the former data of losing through following formula:
J=1..m, wherein, if j position is not former data D
jBut checking data P
x, when then calculating multiplication skip over it P
i' be these new data to be decoded, the quantity of input data when m is coding, P
iThe checking data that uses during for decoding, D
jThe data of input during for coding, P
IjBe the row of the checking data in second generator matrix,
Be xor operation.
Preferably; Because the coding/decoding method that is provided in the embodiment of the invention specifically reduces with respect to the correlation technique amount of calculation; And the high characteristics of decoding processing efficient, therefore, when treating decoded data and encode; Can adopt following coded system to obtain this data to be decoded: at first; The generator matrix Galois Field internal storage data structure that foundation is made up of 0 to 65535 in each element except that unit matrix and the Galois Field in first generator matrix, array or Hash table for example below is the example explanation with the array; When encoding; Judge earlier the multiply operation result who whether has 0 to 65535 in each element except that unit matrix and Galois Field in first generator matrix in the generator matrix Galois Field array; More simply, can judge whether the file of storing this operating result exists; Be judged as under the situation that is, directly reading the multiply operation result who is stored in this generator matrix Galois Field array.Adopt such processing mode, only need simply to judge and read operation, the repeatedly continued operation of Galois multiplication in the prior art, its operational ton significantly reduces, and has improved the efficient of decoding to a certain extent.
In addition; When in this array, not existing in first generator matrix and the Galois Field from 0 to 65535 multiply operation as a result; Obtain 0 to 65535 the multiply operation result of each element except that unit matrix and Galois Field in first generator matrix, promptly calculate in first generator matrix except that unit matrix each element and Galois Field in 0 to 65535 multiply operation result; The multiply operation result of 0 to 65535 in each element except that unit matrix and the said Galois Field in first generator matrix that obtains is saved in the above-mentioned generator matrix Galois Field array.Data in the generator matrix Galois Field array; Not only be used for coding, and be used for adopting after such processing in the step of the new data to be decoded of decode procedure calculating; After adopt the first identical generator matrix to the input data when encoding; Just can directly read corresponding multiply operation result, practice thrift the processing time, raise the efficiency.
Need to prove; In the foregoing description and preferred implementation; The Galois field multiply data that constitute by 0 to 65535 in each element except that unit matrix and the Galois Field in first generator matrix; Except by the array mode in internal memory, preserve and addressing, also can show mode and in internal memory, preserve and addressing according to Hash.Certainly, other comparatively easily storage mode be applicable to the present invention too.
A kind of decoding processing device also is provided in the present embodiment, and this device is used to realize the foregoing description and preferred implementation, has carried out repeating no more of explanation.As following employed, the combination of the software and/or the hardware of predetermined function can be realized in term " module ".Although the described device of following examples is preferably realized with software, hardware, perhaps the realization of the combination of software and hardware also maybe and be conceived.
Fig. 2 is the structured flowchart according to the decoding processing device of the embodiment of the invention, and is as shown in Figure 2, and this device comprises first determination module 22, first acquisition module 24 and second acquisition module 26, describes in the face of this device down.
First determination module 22 is used to confirm to comprise the data to be decoded of former data of part and checking data, wherein, and the former data that this checking data correspondence is lost; First acquisition module 24; Be connected to above-mentioned first determination module 22; Be used for obtaining the little generator matrix corresponding with the former data of losing according to the above-mentioned checking data and first generator matrix that is used to encode, and the new to be decoded data corresponding with the former data of losing; Second acquisition module 26 is connected to above-mentioned first acquisition module 24, is used for obtaining the former data of losing according to these new data to be decoded and above-mentioned little generator matrix.
Fig. 3 is the preferred structure block diagram according to first acquisition module 24 in the decoding processing device of the embodiment of the invention; As shown in Figure 3; This first acquisition module 24 comprises first processing unit 32, first acquiring unit 34 and second acquisition unit 36, describes in the face of this first acquisition module 24 down.
First processing unit 32; Be used for the processing of standardizing of above-mentioned data to be decoded; Wherein, This normalization is handled and is comprised: the position of the former data of reserve part in above-mentioned data to be decoded and the position consistency of the former data of this part in coding initial data before, and checking data is filled into the rest position in these data to be decoded; First acquiring unit 34 is connected to above-mentioned first processing unit 32, is used to obtain and the second corresponding generator matrix of data to be decoded through standardizing after handling; Second acquisition unit 36 is connected to above-mentioned first acquiring unit 34, is used for according to above-mentioned second generator matrix and checking data, obtains the little generator matrix corresponding with the former data of losing.
Fig. 4 is the preferred structure block diagram according to first acquiring unit 34 in first acquisition module 24 in the decoding processing device of the embodiment of the invention; As shown in Figure 4; This first acquiring unit 34 comprises and keeps subelement 42 and fill subelement 44, describes in the face of this first acquisition module 24 down.
Keep subelement 42, be used for keeping the unit matrix of first generator matrix and the columns certificate of former data respective column; Fill subelement 44, be connected to above-mentioned reservation subelement 42, be used for filling checking data at the check column of the unit matrix of above-mentioned first generator matrix.
Fig. 5 is the preferred structure block diagram according to second acquisition unit 36 in first acquisition module 24 in the decoding processing device of the embodiment of the invention; As shown in Figure 5; This second acquisition unit 36 comprises extraction subelement 52 and forms subelement 54, describes in the face of this second acquisition unit 36 down.
Extract subelement 52, be used for extracting row and the row that above-mentioned checking data occurs at second generator matrix; Form subelement 54, be connected to said extracted subelement 52, the row that is used for the above-mentioned checking data that extracts is occurred is formed the corresponding little generator matrix of losing of initial data with row.
Fig. 6 is the preferred structure block diagram one according to the decoding processing device of the embodiment of the invention; As shown in Figure 6; This device all modules in comprising Fig. 2, also comprise first judge module 62, the 3rd acquisition module 64 and/or first writing module 66, describe in the face of this device down.
First judge module 62 is connected to above-mentioned second acquisition module 24, and the relation between checking data that is used for comprising according to little generator matrix and the corresponding initial data of losing judges corresponding your pupil becomes matrix inversion matrix file whether to exist; The 3rd acquisition module 64 is connected to above-mentioned judge module 62, and being used in judged result is under the situation that is, becomes matrix inversion matrix file to obtain the your pupil to become the inverse of a matrix matrix through reading the your pupil; And/or first writing module 66 is connected to above-mentioned the 3rd acquisition module 64, be used for judged result for situation not under, calculate the your pupil and become the inverse of a matrix matrix, and become the your pupil inverse of a matrix matrix to be written to the your pupil to become matrix inversion matrix file.
Fig. 7 is the preferred structure block diagram two according to the decoding processing device of the embodiment of the invention, and is as shown in Figure 7, this device all modules in comprising Fig. 6, comprises that also first sets up module 72, down first sets up module 72 and describes in the face of this.
First sets up module 72, is used for setting up the inverse matrix Galois Field array that 0 to 65535 in each element that the your pupil becomes the inverse of a matrix matrix and the Galois Field constitutes.
A kind of coding/decoding system also is provided in the present embodiment; Fig. 8 is the structured flowchart according to the coding/decoding system of the embodiment of the invention, and is as shown in Figure 8, and this system is except that comprising above-mentioned each described decoding processing device; Also comprise a code device; Wherein, this code device comprises: second sets up module 82, is used for setting up the generator matrix Galois Field array that is made up of 0 to 65535 in each element and the Galois Field of first generator matrix except that unit matrix; Second judge module 84 is connected to the above-mentioned module 82 of setting up, and is used for judging whether generator matrix Galois Field array exists the multiply operation result of 0 to 65535 in each element except that unit matrix and Galois Field in first generator matrix; Read module 86 is connected to above-mentioned second judge module 84, and the judged result that is used at judge module is under the situation that is, reads the multiply operation result who is stored in the generator matrix Galois Field array.
The foregoing description and preferred implementation proposed when file storage or data communication; Method based on fast coding and fast decoding under the RS correcting and eleting codes system; This method can realize through software, promote Code And Decode speed effectively, and finding the inverse matrix, document No.; And the operand of file decoding is all irrelevant with burst number m, and it is relevant that the verification burst that uses during only with coding, decoding is counted n.In the method; Normalization through using data to be decoded is handled; Can be with the less generator matrix G1 that generates equivalence under the RS correcting and eleting codes system, this generator matrix size is n*n, can be far smaller than the generator matrix G of original m*m; Therefore can calculate its inverse matrix fast, and be used for fast decoding.In addition, all Galois field multiplies are changed into by double meter method look into the internal memory method, and memory consumption is very little, can be applied in the existing computer system.Adopt the foregoing description and quick decoding method that preferred implementation proposed, can (m+n be m) under the correcting and eleting codes system at Read-Solomon; Improve the m value effectively; Thereby improve the memory space utilance, perhaps reduce the checking data ratio, when data communication, improve communication speed.For example, at n=6, and decoding the time is used under the situation of whole 6 block check bursts; Support m up to 64 to 128; The respective coding rate reaches 91.4% and 95.5%, and m, and n can be provided with arbitrarily under the condition that satisfies RS correcting and eleting codes condition restriction and Computing capabilities limits.Since finding the inverse matrix, file encoding and decoding all and burst quantity m irrelevant, so m no maximum constraint is calculated restriction as long as guarantee m+n less than 2^8 or 2^16 Galois Field.The present invention is to not restriction of generator matrix, and Fan Demeng matrix and Cauchy matrix are all applicable, and any generator matrix that other meets RS correcting and eleting codes system is also applicable, uses but use Cauchy matrix can reduce internal memory, obtains better effect.
Through the foregoing description and preferred implementation; Generator matrix finding the inverse matrix and encoding and decoding performance under the RS correcting and eleting codes system have greatly been improved; Breakthrough is not only arranged in theory, and have bigger practical value, be applicable in cloud storage and the data communication field and use in the system of RS correcting and eleting codes.
The fast coding coding/decoding method that is provided in the present embodiment under the situation that keeps use RS correcting and eleting codes know-why, is made up of data fast coding and two kinds of methods of data fast decoding.These two kinds of methods were both separate, were related again.They can use separately or unite use, are respectively applied for data fast coding and data fast decoding.
Data decode in data fast decoding method and the correlation technique exists than big-difference, and this data fast decoding method roughly comprises following processing: (1) with data requirementization to be decoded, and through file cache inverse matrix data; (2) inverse matrix of the equivalent minor matrix of rapid solving generator matrix; (3) decoding and replacement checking data.
Need to prove that above-mentionedly adopt above-mentioned quick decoding method, to not restriction of generator matrix, Fan Demeng matrix, Cauchy matrix or other generator matrix are all applicable, this are not limited, but use Cauchy matrix can obtain better effect.Be that Cauchy matrix is that example describes in the present embodiment with the generator matrix.
For given m, the n parameter, the generator matrix G of system uses Cauchy matrix, for example, and m=5, n=3, i.e. (8,5) correcting and eleting codes, generator matrix G is the combination of unit matrix I and Cauchy matrix Q.The following demonstration, use 3 row check matrixes altogether, P
IjThe j row element of the i columns certificate in the expression Cauchy matrix except that unit matrix.
If the input data are the D that comprises m element, use matrix notation to be D=[D
1, D
2... .D
m], use P to represent checking data, the use matrix notation is [P
1, P
2... P
n], then the RS coded system is:
D* [I, Q]=[D, P], perhaps: [D
1, D
2... .D
m] * G=[D
1, D
2... .D
m, P
1, P
2... P
n].
Describe to fast coding and fast decoding method respectively below.
1, fast encoding method:
System's all multiplication X*Y data among the Galois Field G (2^16) that are prepared in advance, X wherein, Y from 0 to 65535.These size of data are 65536*65536*2=8GB, leave among the file Galois.dat.For reading conveniently, these data can be placed in some files, logically form single Galois.dat file during actual the use.
Because m, after n confirmed, generator matrix G also just confirmed, according to the column element of Cauchy matrix G wherein these data carried out counts, amounted to m*n element, numbered from 0 to m*n-1.For m*n element set up coding two-dimensional array Code [] [], to each element P
IjCalculate its in Galois Field to 0 to 65535 multiply operation result, preserve hereof in advance, filename and m, n are relevant.Generator matrix during coding and the file that specifically will encode are irrelevant, and only and m, n is relevant, therefore can generate the generator matrix Galois Field cache file under the special parameter in advance.Like this, during system initialization, at first read corresponding generator matrix Galois Field cache file.If this document does not exist, then the array element with needs calculates and writes in the file, so that read later on.Calculate certain element P
IjAt whole multiplication results of Galois Field, be exactly appropriate location (P in Galois.dat
Ij* read continuous 65536 words 65536*2 byte place), amount to the 128K byte.
Following table 1 is that generator matrix Galois Field array Code [m*n] [65536] forms signal.The first dimension all elements i is the element numbering that occurs among the generator matrix G among the Code [i] [k], the second dimension element k from 0 to 63335, and Code [i] [k] deposits element P among the G
IjGalois field multiply result with k.Behind all elements in the generation matrices algorithm acquisition generator matrix, when needs calculate D
iMultiply by P
IjThe time, only need find P
IjNumbering x, return array Code [x] [D then
i] value get final product.Only need 1 operation, practice thrift 4 times than 5 operations of original Galois field multiply.
Table 1
For RS coding, when needing to preserve former data, all want an extra preservation n effect piece at every turn, need m Galois field multiply and m-1 sub-addition during each effect blocks of data calculating.Because 1 Galois field multiply equals 5 basic operations, therefore, the about 6m of each effect blocks of data amount of calculation.Total amount of calculation is 6mn.After adopting the fast encoding method of the foregoing description; Use is looked into the array method and is replaced Galois field multiply, and a Galois field multiply only need be looked into the internal memory operation of array for 1 time, therefore; The about 2m of each effect blocks of data amount of calculation; Total amount of calculation is 2mn, and than the operational ton of original minimizing 2/3, coding rate improves twice.The Code array size of using is m*n*128KB.Typically, m=64, during n=10, the Code array size is the 81.92M byte.
For length is the file of L, gets m word (two bytes) at every turn and encodes.Press word code rather than press byte code, be because in Galois Field G (2^16), calculate, and press the word length that the word encoding and decoding adapt to computer system itself, can improve encoding and decoding speed.
If L is not the m integer multiple, then mend the multiple of 0 polishing at last to m.Therefore, need cycle calculations L/m time, former amount of calculation is 6Ln, and improving the back amount of calculation is 2Ln.It is irrelevant that encoding calculation amount that file is total and burst are counted m, only relevant with file size L with check block n.Therefore, improving m quantity can not cause document No. speed to descend.Opposite because after improving burst m quantity, the ratio that extra generation and the checking data size that writes disk account for original descends, so coding efficiency improves on the contrary.Following table 2 is the encode test results under the Dell E6420 notebook, and unit is second:
Table 2
Can find out, the coding of identical big small documents, when the verification burst was identical, along with the increase of m, coding rate was accelerated on the contrary.
Because the character of Cauchy's generator matrix; Can pass through the proper parameters setting; Make m*n element in generator matrix m+n-1 different element only to occur, like this can be with the Code array size; Becoming Code [m+n-1] [65536] from Code [m*n] [65536], promptly approximately is original 1/n.
2, fast decoding method:
The fast decoding method that present embodiment provided comprises following steps: (1) with data requirementization to be decoded, and calculates the data D to be decoded under this situation " and corresponding little generator matrix G
1', read G through file
1' the inverse matrix data.(2), then calculate this your pupil and become the inverse of a matrix matrix, and write file to the inverse matrix data if the inverse matrix file does not exist.(3) fast decoding checking data.Describe in the face of above-mentioned steps down.
" and corresponding little generator matrix G (1):, and calculate the data D to be decoded under this situation with data requirementization to be decoded
1' filename, read G through file
1' the inverse matrix data.
For a given data D ' to be decoded who comprises m piecemeal, the use matrix notation is [D
i... D
j, P
x..P
y], data to be decoded are standardized as follows, generate unique arrangement.Its method is:
For data D ' to be decoded=[D
i... D
j, P
x..P
y], for original data D wherein
i, make it to be arranged in D ' i position, i.e. its original position that occurs.For remaining position among the D ', use a plurality of check block P data, from left to right the order of original check block (promptly according to) carries out order to D ' and fills.For example, m=5, n=3 uses data block D to be decoded
2, D
5And P
1, P
2, P
3, the D '=[P after then standardizing
1, D
2, P
2, P
3, D
5].This normalized number certificate is to use the unique arrangement of former data block and checking data piece combination by this rule.
Calculate data D to be decoded " principle be:
Wherein, D
1, D
3, D
4Be unknown data, P
1, P
2, P
3, D
2, D
5Be given data:
D
1P
11+D
3P
13+D
4P
14=P
1–D
2P
12-D
5P
15=P
1’
D
1P
21+D
3P
23+D
4P
24=P
2–D
2P
22-D
5P
25=P
2’
D
1P
31+D
3P
33+D
4P
34=P
3–D
2P
32-D
5P
35=P
3’
That is:
As general formula generally speaking:
Wherein, P
iThe checking data that uses during the expression decoding, D
jAnd P
IjThose row of checking data among expression input data and the generator matrix G.If j position is not former data D
jBut checking data P
x, when then calculating multiplication skip over it.
expression is an xor operation.In Galois Field, plus-minus all is an xor operation.
Little generator matrix G
1Be n*n data among the former generation matrix G.The check block quantity of n for using.If S={1,3,4} is the position that check block occurs, the then P in the generator matrix
IjForm little generator matrix G
1, i, j ∈ S.
Table 3 has been illustrated the generation of little generator matrix, and as shown in the table: in those generator matrixes, the data that the row that checking data occurs, row are formed are formed little generator matrix G
1 final little generator matrix G1 is as shown in table 4.
Table 3
Table 4:
P
11 |
P
21 |
P
31 |
P
13 |
P
23 |
P
33 |
P
14 |
P
24 |
P
34 |
As long as record burst m number position and numbering occur with each checking data, can constitute unique filename.The numbering that position (in the table 3 italic overstriking show) is exactly the former deblocking lost appears in checking data.
After having obtained the inverse matrix filename of little generator matrix G1, read this document.If this document exists, read this document content to inverse matrix G
1' in, directly accomplish the evaluation work of inverse matrix, enter into step (3).If this document does not exist, then do sign, accomplish etc. step (2), generated inverse matrix after, again the inverse matrix data are write in this document.Also can calculate the corresponding inverse matrix file of some typical arrangement in advance, be kept in the storage system, run into the back for system and directly use.
Step (2): calculate inverse matrix
Give an example the employed normalization example arrangement of the foregoing description [D for top
1, D
2, P
1, P
3, D
5], obtained little generator matrix G
1, but do not have the corresponding cache file, and then use conventional calculating inverse matrix method, obtain little generator matrix G
1Inverse matrix G
1'.
The algorithms most in use of calculating the square formation inverse matrix has Gauss-Ruo when the elimination and the adjoint matrix tactical deployment of troops.
Work as the elimination for Gauss-Ruo, the inverse matrix amount of calculation of calculating square formation m is O (m^3).Because the generator matrix size drops to n*n element now, amount of calculation drops to O (n^3).Relative theory and concrete realization are studied clear already, repeat no more at this.The advantage of the foregoing description is, can generator matrix be reduced to the n*n of equivalence from m*m, during less than former burst m, can significantly reduce the workload of calculating inverse matrix at the verification burst n that uses.Only when using verification burst n=m, just equate with original amount of calculation.
Use adjoint matrix tactical deployment of troops finding the inverse matrix, amount of calculation is higher than Gauss-Ruo and works as the elimination.No matter adopt which kind of finding the inverse matrix method, obtain inverse matrix after, all the inverse matrix data are write the corresponding minor mistake in treatment matrix D of normalization row ' in the pairing file.Run into later on and use identical former data block and check block combination, can directly from file, read and generate inverse matrix.An inverse matrix file comprises the n*n*2 byte, and for n=6, the inverse matrix file size is 72 bytes, and under the current computer disposal ability, it is little of ignoring to read document time.Therefore, after the employing inverse matrix metadata cache mechanism, have only when running into for the first time this normalization data arrangement, need to calculate inverse matrix, later just no longer the needs calculates.Because depending on the Data Position of damage and the verification burst of use, the inverse matrix data arrange; Therefore the inverse matrix data are relevant with actual operating position; At m; Can not all calculate in advance under the big situation of n, therefore adopt the mechanism of the embodiment of the invention and preferred implementation, when running into for the first time, calculate and preserve the inverse matrix data.Owing to adopted standardized method, can unique specified data piece damage the verification burst of position and use and number, therefore, run into the combination of same case later on, just can obtain inverse matrix through the mode that reads the inverse matrix file.Along with the continuous operation of system, can generate and preserve the inverse matrix data under more and more combinations, systematic function is also just become better and better.
Employing the foregoing description and the method that preferred implementation provided are dwindled than originally much owing to calculate the inverse matrix workload, therefore, can effectively increase the value of verification piecemeal quantity n during actual coding.
After calculating decoded data, these elements are numbered from 0 by row.The n*n of an inverse matrix element, reference numeral are 0 to n*n-1.Set up decoding array Decode [n*n] [65536], deposit the multiplication data of inverse matrix all elements Galois Field.From Galois Field full dose file Galois.dat, read the corresponding Galois field multiply result of these inverse matrix elements.Each element X
IjCorresponding Galois field multiply result has 128K, is positioned at Galois.dat and begins from the x*65536*2 byte.The Decode matrix size is n*n*128k.If 10 verification bursts are supported in decoding at most, then the Decode matrix size is the 12.8M byte.Owing to separating byte number that the inverse matrix elapsed time much needs to preserve seldom, therefore very big and rise time of inverse matrix Galois field multiply array data amount seldom, need not preserve inverse matrix Galois field multiply result, only needs preservation inverse matrix data to get final product.
Step (3), the fast decoding checking data
For the decoding system that is adopted in the embodiment of the invention, only need to recover k check block data, all the other are former data, and the position is also identical.
For example, 3 effect blocks of data P have been used in the example
1, P
2, P
3, when then decoding, residue D
2, D
5All can directly use, and the position is also identical in former data D.And D
1, D
3, D
4Data preparation and inverse matrix multiplication according to the front calculate.During decoding, use the Decode matrix to carry out Galois field multiply.For example, calculate P
i' value and inverse matrix X
IjMultiplication the time, at first find P
i' numbering x, return Decode [x] [P then
i'].Decoding calculating quantity is O (mk), than original O (m^2), drops to original k/m.Equal 5 basic operations by a Galois Field and calculate, former decoding calculating quantity adds up to 6mm.
Use k check block (k=n) time, decoding calculating quantity is O (mk), than original O (m^2), drops to original k/m.Equal 5 basic operations by a Galois Field and calculate, former decoding calculating quantity adds up to 6mm; Use coding/decoding method of the present invention, Galois field multiply all becomes looks into the internal memory mode, and amount of calculation drops to 2mk.As m>during k, decoding speed improves 3m/k doubly.
The fast decoding checking data specifically comprises the steps:
1) the data preparatory stage
If use k checking data during decoding, then need calculate k data, each data need be calculated m-k Galois field multiply and m-k XOR addition.Because each Galois field multiply is D
jMultiply by P
Ij, and P
IjBe the generator matrix element, therefore, can utilize " generator matrix Galois Field array Code " in the aforementioned fast encoding method, multiply operation is changed into directly look into array manipulation, when calculating D
jMultiply by P
IjThe time, only need find P
IjNumbering x, return array Code [x] [D then
j] value get final product.Only need 1 operation, add the XOR addition, the data preparatory stage needs 2 (m-k) k operation altogether.
2) decode phase
Decode phase is k P
i' multiply by G
1Inverse matrix G
1', this matrix size is k*k.Owing to directly use when calculating and look into Decode [] [] array mode, add the add operation of XOR, need 2kk operation.Therefore the decoding total needs 2 (m-k) k+2kk=2mk operation, and amount of calculation is identical with coding.Obviously and former 2mm operation compared, during less than data fragmentation m, total operation amount significantly reduces the embodiment of the invention at the check block k that uses.Even compare with follow-on decoding 6mk amount of calculation, the embodiment of the invention also can reduce by 2/3 amount of calculation owing to utilized coding buffer memory Matrix C ode [] [] and decoding buffer memory matrix D ecode [] [].
For length is the file of L, gets m word at every turn and decodes.Need circulation L/m time altogether, total amount of calculation is: O (mk) * L/m=O (Lk) particularly is to be slightly larger than 2Lk.
Therefore, adopt the fast decoding method of the embodiment of the invention, total amount of calculation only with the redundant block k of file size and use (k=n) relevant and burst m has nothing to do.When inverse matrix was calculated, also only relevant with the redundant slices k that uses and m had nothing to do.Therefore, the raising burst is counted the m value can not influence decoding speed.
On the contrary, because m increases the ratio decline that the back checking data accounts for original, so decoding performance slightly improves on the contrary.Under the situation of supporting 10 verification bursts, need use the 12.8M byte of memory to deposit the Decode matrix data during decoding, under the situation of m=64, need the 81.92M byte of memory to deposit the Code matrix data.If m+n-1=73 different elements only appear in generator matrix, then need the 73*128K=9.344M byte to deposit the array data of Code coding.
Following table 5 during for encoding and decoding required memory data and the encoding and decoding of different parameters under being provided with checking data account for the ratio of general act.The Code array is calculated according to m+n-1 different elements.Can find out, under the situation of 10 verification bursts, be no more than at 128 o'clock in burst quantity, only need at most the internal memories of 30.336M, all Galois field multiply data that run in the time of can be with encoding and decoding all are loaded into internal memory, do not need the 8G internal memory.
Table 5
During coding, need checking data be write file.During decoding, need checking data be reverted to original data and write file.Therefore, the ratio of checking data when encoding and decoding is low more good more.As can be seen from the above table, under the constant situation of verification burst n, m is big more, and the ratio that checking data accounts for when coding, decoding is just low more, and the coded data that need additionally write file is just few more with decoded data.Therefore though RS correcting and eleting codes amount of calculation is irrelevant with m during encoding and decoding, only relevant with the check block number, m is big more, and the extra ratio data that writes disk is just few more, encodes, decoding performance is also just good more.
In sum; The method of fast coding and fast decoding under the RS correcting and eleting codes system that the embodiment of the invention provided; Can realize through software; Promote Code And Decode speed effectively, and the encoding and decoding speed of matrix inversion and file is all irrelevant with m, it is relevant that the verification burst that uses during only with coding, decoding is counted n.Through using the standardized method of data to be decoded, the equivalent your pupil that can be used for calculating fast under the RS correcting and eleting codes system becomes the inverse of a matrix matrix, and fast decoding.With the inverse matrix amount of calculation from before O (m^3) be reduced to O (n^3).Through using internal memory seldom, Galois multiplication data are put into internal memory, when calculating Galois field multiply, use the method for looking into memory array to replace double meter method in the past, further improved encoding and decoding speed.
The quick decoding method that uses the embodiment of the invention to propose; Can be at Read-Solomon (m+n; M) under the correcting and eleting codes system, improve encoding and decoding speed and m value effectively, thereby improve the memory space utilance; Perhaps reduce the ratio of checking data in total data, and when data communication, improve communication speed.At n=6, and decoding the time uses under the situation of whole 6 block check bursts, supports m up to 64 to 128, and the respective coding rate reaches 91.4% and 95.5%, and m, and n can setting arbitrarily under the condition that satisfies RS correcting and eleting codes condition restriction and Computing capabilities limits.The present invention is to not restriction of generator matrix, and Fan Demeng matrix, Cauchy matrix or other generator matrix are all applicable.But when using Cauchy matrix, can reduce the size of generator matrix multiplication array Code, thereby reduce memory consumption, obtain more excellent effect through parameter is set.
A kind of method of the quick encoding and decoding based on the RS correcting and eleting codes is provided in the present embodiment, wherein, has comprised coding and decoding, specified below data.Before the encoding and decoding of explanation, at first the Cauchy's generator matrix in the fast coding is described data.
Can know by disclosed mathematical knowledge, the Cauchy's generator matrix behind the square formation I of the generator matrix unit of removing shown in the first half of above-mentioned matrix G, matrix element P
Ij=1/ (x
i+ y
j).
According to open knowledge, the Cauchy matrix element is restricted to:
For X={x
1, x
2... x
m, Y={y
1, y
2... y
n,
1) to any i and j, x is arranged
i+ y
j!=0;
2) to any two element x among the X
i, x
j, have i! During=j, x
i!=x
j
3) to any two element yi among the Y, yj, have i! During=j, yi!=yj
Therefore, select x
i=i, y
j=j-1, wherein i is from 1...m, and j obtains the latter half of the above-mentioned matrix G of Cauchy matrix from 1..n.Like this, m+n-1 different elements only appear in this Cauchy's generator matrix.These elements are converted into the Galois Field corresponding element, and number by row, from 0 to m+n-2.Can confirm first element of every row P like this
IjCorresponding array is numbered j-1, and the array numbering that other element of these row is corresponding increases progressively.
Following table 6 is the data cached Code of Galois Field generator matrix [m+n-1] [65536] example after improving.
Table 6
For m=64, n=10, the Code array size is (m+n-1) * 128k=9.344M byte.For existing computer system, can all load into internal memory.
After obtaining Cauchy's generator matrix, data are carried out encoding operation through following steps:
S1, the needs coded data is encoded: in the RS correcting and eleting codes, confirmed burst number m and checking data n after, can pass through vandermonde matrix or Cauchy matrix, through calculate obtaining generator matrix G.With m=5, n=3 is an example in the present embodiment.
Wherein, the input data to be encoded are with D=[D
1, D
2, D
3, D
4, D
5] expression, the generator matrix G of use, the left side is unit matrix I, the right is the 3*5 matrix that comprises three row.Through the coding computing, the coded data of generation is [D
1, D
2, D
3, D
4, D
5, P
1, P
2, P
3], wherein preceding 5 is former data fragmentation, back 3 is redundant slices.
The data receiver, data are decoded: in the embodiment of the invention " based on the quick decoding method of RS correcting and eleting codes ", when fast decoding; Treat decoded data and format processing: promptly when needs are decoded, at first will appoint, use D ' expression to 5 bursts; D '=[D2, D5, P1 for example; P2, P3].Adopt standardized method of the present invention then, D ' is standardized, its method is: with former data fragmentation Di playback, put back to the home position.Insert in order remaining the vacant check block data of using then.For example, in the present embodiment: obtain later D '=[P1, D2, P2, P3, the D5] of normalization through following processing.
S2 according to normalized D ', calculates generator matrix G in the case.Method is with those columns certificates of using checking data among the unit matrix I, inserts with corresponding check columns certificate in the former generation matrix.The characteristics of generator matrix G are: those have used the row x of legacy data piece, or corresponding row x in the unit matrix, and this is listed as x element is 1, and all the other are 0.For example, in the present embodiment:
After obtaining generator matrix G, can calculate new data D to be decoded "=[P
1', P
2', P
3'].Computing formula is following:
Wherein, when the i position is former data among the D ', use D
i, otherwise skip over.
P wherein
iThe checking data that uses during the expression decoding, D
jAnd P
IjThose row of checking data among expression input data and the generator matrix G.If not being former data but checking data, j position when then calculating do not skip over it.Calculating D
j* P
IjThe time, can use " generator matrix Galois Field array Code " in the fast encoding method, replace the Galois field multiply operation through looking into the array mode, so that raise the efficiency.
Afterwards, calculate the little generator matrix G of equivalence
1Method is following:
Wherein, P
1'=P
1-D
2P
12-D
5P
15, P
2'=P
2-D
2P
22-D
5P
25, P
3'=P
3-D
2P
32-D
5P
35,
Obtain new data D to be decoded thus "=[P
1', P
2', P
3'], and little generator matrix:
Little generator matrix G
1Be the n*n row among the former generation matrix G.The check block quantity of n for using.If S={1,3,4} is the position that check block occurs, the then P in the generator matrix
IjForm little generator matrix G
1, i, j ∈ S.Promptly in generator matrix, the data that the row that checking data occurs, row are formed are formed little generator matrix G
1This matrix is k*k under the situation of using k checking data.
(3) calculate G
1Inverse matrix, the former data that the decoding acquisition is lost
Adopt conventional finding the inverse matrix method to calculate G
1Inverse of a matrix matrix G1 ' after the acquisition inverse matrix data, reads the corresponding Galois field multiply data of inverse matrix element afterwards one by one from Galois Field file Galois.dat, put into " inverse matrix Galois Field array Decode ".Use D "=[P
1', P
2', P
3'] multiply by inverse matrix G
1', the initial data D that promptly can obtain to lose through following formula
1, D
3, D
4:
And, only need calculate k data during decoding.Subsequently, the initial data that will lose replaces the P in original input data
1, P
2, P
3, can obtain whole initial data, decode successfully.
To importing the processing of standardizing of data to be decoded, be the core concept of the embodiment of the invention " based on the quick decoding method of RS correcting and eleting codes ".In RS correcting and eleting codes decoding algorithm in the past, its data to be decoded generally are that back checking data mode is arranged, i.e. D '=[D according to the former data of elder generation
1, D
2, D
5, P
1, P
3].Former data D when calculating inverse matrix, decoding like this
iThe position is not fixed, and causes unit matrix row corresponding in the inverse matrix, the former Data Position in decoding back not to fix, and can't confirm need calculate which row when therefore calculating inverse matrix actually, can only treat according to the m*m matrix; When decoding, former data D
iNeed duplicating or exchange, just can get back on original position, cause a large amount of judgements and replication work, therefore, is to calculate inverse matrix or decoding efficiency is not high.This in addition mode can't be found out the corresponding relation between the checking data of the former data of losing and use intuitively, therefore can't confirm unique Rankine-Hugoniot relations, also just can't use file cache inverse matrix data.
Behind the standardized method of the input data to be decoded of the employing embodiment of the invention, the benefit of at least four aspects below can obtaining: at first, and can be through record checking data position and numbering, unique fast little generator matrix G that confirms that this arrangement is corresponding
1And the inverse matrix file name, and read the inverse matrix file content, directly generate inverse matrix G
1'.Through the file cache mode, avoided matrix computations running under the repetition situation, improved performance.Secondly, when inverse matrix does not exist, can be according to G
1Calculate inverse matrix G fast
1'.Because this matrix has only the k*k size, k < less than original generator matrix m*m, has therefore significantly reduced the inverse matrix amount of calculation during m, has been reduced to O (k^3) from O (m^3).After obtaining inverse matrix, all inverse matrix element multiplication data are put into Decode [] [] array.Once more, when decoding is calculated, only need to calculate k checking data and get final product, remain m-k data and can directly use the data on the origin-location, improved decoding efficiency.And when data are prepared; Can use the Galois field multiply array of generator matrix buffer memory, thereby further improve decoding efficiency, last; No matter encode, decode; All multiply operations of Galois Field all are placed into internal memory, adopt the mode of looking into memory array to replace former double meter method, have further improved encoding and decoding speed.
Need to prove that the format of the employed data to be decoded of the embodiment of the invention is represented and generation matrices algorithm has nothing to do, and no matter is that vandermonde matrix, Cauchy matrix or other generator matrix all can use.After the embodiment of the invention and preferred implementation are reduced to the k rank square formation of equivalence with generator matrix by m rank square formation; Calculate the inverse matrix of minor matrix again; Also irrelevant with the algorithm that calculates inverse matrix; No matter be the Gaussian elimination method or the adjoint matrix tactical deployment of troops, use the decoding method of the embodiment of the invention, all can improve the speed of calculating inverse matrix.Because the characteristic of Cauchy matrix can pass through the parameter setting, the number of different elements in the Cauchy matrix is reduced to m+n-1 from m*n, thereby the EMS memory occupation that can reduce the Code array is big or small.Therefore among the present invention, generator matrix adopts Cauchy matrix can reduce EMS memory occupation.
Below in conjunction with accompanying drawing, the embodiment of the invention is described.
Fig. 9 is the flow chart based on the file fast encoding method of RS correcting and eleting codes according to the embodiment of the invention, and as shown in Figure 9, this method comprises the steps:
Step S902, the system 8G Galois field multiply data file Galois.dat that is prepared in advance, these data are results of all the multiplication X*Y in the Galois Field, X, Y from 0 to 65535, and multiplication result also is 2 bytes, amounts to 64k*64k*2=8GB.The i*128k that whole Galois field multiply data of i element are placed on file begins the place, continuously the 128k byte.For conveniently reading, can these data be kept in some files, logically constitute unified 8G Galois field multiply data file.
According to given m, n value initialization encoder according to system's generation matrices algorithm, calculates generator matrix, and according to m, the n value is distributed Galois field multiply two-dimensional array Code [m*n] [65536] usage space.
The Code array need be distributed the internal memory of m*n*128k byte altogether.It is the two-dimensional array of m*n*64k, and each element is between 0 to 65535 in the array, a word length of corresponding computer system, perhaps two bytes.
If generator matrix uses Cauchy matrix, then the Code size can be (m+n-1) * 128k byte.
Step S904, according to m, n generates Galois field multiply cache file title, judges whether this document exists.Change step S908 if exist.If there is no, then continue step S906.More excellent ground, system can prepare some typical m in advance, and the Galois field multiply cache file under the n configuration is so that import in the Code array.
Step S906, the Galois field multiply cache file does not exist, to the columns of those except that unit matrix certificate in the generator matrix; Amount to m*n element and increase progressively numbering, calculate each element, be kept among the Code 0 to 65535 Galois field multiply result; To element x, also can be from Galois.dat x*128k, read the multiplication result of this element; Amount to the 128K byte, import in the Code array, carry out step S910.
Step S908, the Galois field multiply cache file exists, and directly reads this document content in Code, gets into step S912.
Step S910 is written to m with Code array content, and the Galois field multiply cache file that n is corresponding gets into step S912.
Step S912 after the completion coding eligible task, opens file F to be encoded.
Step S914, order is read in m data from file:
Read in m word, 2m byte.
If the data deficiencies m that reads in, end of file then has been described, mend the multiple of some 0 back polishings to m.
Reading under the case of successful, getting into step S916, otherwise getting into step S920.
Step S916, when reading successfully, D carries out the RS coding to the input data.Wherein, a preceding m data are directly returned former data.A back n data need to calculate.Each data needs m Galois field multiply operation and m-1 sub-addition (XOR) operation.Wherein, Galois field multiply adopts and looks into the Code array method to raise the efficiency, and its method is, for D
i* P
IjGalois field multiply, at first confirm P
IjNumbering x, return Code [x] [D then
i] element.Only need twice operation, and original Galois field multiply needs 5 operations.
Step S918 writes the former data of the m behind the coding and n checking data in m+n the branch block file respectively.Return step S914, continue to read m data under the file.
Step S920 reads the file failure, and supporting paper has been encoded and finished.Close file F to be encoded and m+n branch block file, finish coding.
It is pointed out that above-mentioned file fast coding flow process based on the RS correcting and eleting codes shown in Figure 9 is for the realization principle of the fast coding flow process that the embodiment of the invention provides is described.Also need provide abnormality processing, file by functions such as piece processing, log file metadata during concrete the realization.
Figure 10 is the flow chart based on the file fast decoding method of RS correcting and eleting codes according to the embodiment of the invention, and is shown in figure 10, should comprise the steps: based on flow process of the file fast decoding method of RS correcting and eleting codes
Step S1002, the system 8G Galois field multiply data file Galois.dat that is prepared in advance, according to m, the n initializing decoder calculates former generation matrix G.
Because in the RS correcting and eleting codes algorithm that each file adopts, m, the n parameter maybe be different, therefore only when the concrete file of decoding, could obtain m according to the metadata or the burst information of this document, n, initializing decoder calculates former generation matrix.Do like this, can make encoding function and decoding function independent mutually.
As previously mentioned, after calculating generator matrix, the array data of generator matrix Galois Field is imported to the Code [] [] from file.If file does not exist, then need to calculate, also direct reading of data from the Galois.dat file.
Step S1004 according to data block numbering to be decoded, treats decoded data and formats arrangement, produces the corresponding little generator matrix G of generator matrix G
1The inverse matrix filename.
Suppose in the data block to be decoded, comprise k checking data, 0 < k ≤n.Then little generator matrix G
1Size is k*k.
Step S1006 reads the inverse matrix file.If the inverse matrix file exists, get into step S1008, otherwise get into step S1010.
Step S1008 reads the inverse matrix file content, recovers inverse matrix G
1', get into step S1022.
Step S1010 arranges according to the format of generator matrix and data to be decoded, generates the generator matrix G and corresponding little generator matrix G that treat finding the inverse matrix
1
Whether step S1012 uses Gaussian elimination method to ask G
1Inverse matrix? If get into step S1014, otherwise get into step S1016.
Step S1014 adopts Gaussian elimination method to calculate G
1Inverse matrix G
1'.Need O (k^3) amount of calculation altogether, get into step S1018.
Step S1016 uses adjoint matrix tactical deployment of troops finding the inverse matrix G
1'.
Step S1018 is with the inverse matrix G that calculates
1' data write in the corresponding minor mistake in treatment matrix file of this formatted data, so that directly read the inverse matrix data after running into next time.
Step S1020, the Galois field multiply data that read all data among the inverse matrix G1 ' are in Decode [] [] array.If Decode array corresponding file exists, then directly read, otherwise read the mode of corresponding data among the Galois.dat through correspondence, calculate Decode [] [] array.
Step S1022 opens format and arranges m corresponding branch block file and treat recovery file F '.
Step S1024 reads 1 data respectively from the m file, amount to m data.If some slicing files does not have data, then mend 0, polishing is to m data.Reading under the case of successful, getting into step S1026, otherwise getting into step S1032.
Step S1026 copies to a m to be decoded data in the output area, calculates new data D to be decoded then ".Computing formula:
J=1..m, wherein, P
iThe checking data that uses during the expression deciphering, D
jAnd P
IjThose row of checking data among expression input data and the generator matrix G.If j position is not former data D
jBut checking data P
x, when then calculating skip over it.
expression is an xor operation.Calculating D
i* P
IjThe time, use the Code array in the fast encoding method, replace the Galois field multiply operation through looking into the array mode, so that raising speed.
With k P
i' data set is made into D ", for example, D "=[P
1', P
2', P
3'].
Step S1028 uses matrix multiplication: D " * G
1', generate k data of losing.Wherein multiplication uses the Decode array directly to search.Replace the position of original k check code in the data of output area after decoding finishes one by one, can obtain whole former numeric data codes.
Step S1030 is written to the data of the m in the output area and treats to return step S1026 among the recovery file F '.
Step S1032 reads the file failure, shows that processing finishes.Close all, finish decoding.
Need to prove that equally the file fast decoding flow process based on the RS correcting and eleting codes shown in Figure 10 is for the realization principle of the fast coding flow process that the embodiment of the invention provides is described.Also need provide abnormality processing, file to handle, handle the last function such as zero of mending more during concrete the realization by file metadata by piece.
Obviously, it is apparent to those skilled in the art that above-mentioned each module of the present invention or each step can realize with the general calculation device; They can concentrate on the single calculation element; Perhaps be distributed on the network that a plurality of calculation element forms, alternatively, they can be realized with the executable program code of calculation element; Thereby; Can they be stored in the storage device and carry out, and in some cases, can carry out step shown or that describe with the order that is different from here by calculation element; Perhaps they are made into each integrated circuit modules respectively, perhaps a plurality of modules in them or step are made into the single integrated circuit module and realize.Like this, the present invention is not restricted to any specific hardware and software combination.
The above is merely the preferred embodiments of the present invention, is not limited to the present invention, and for a person skilled in the art, the present invention can have various changes and variation.All within spirit of the present invention and principle, any modification of being done, be equal to replacement, improvement etc., all should be included within protection scope of the present invention.