AU604545B2 - A method for storing data - Google Patents

Description

COMMONWEALTH OF AUSTRALIA PPi TENTS ACT 1952 COMPLETE SPECIFICATION (Original) 0 FOR OFFICE USE N=h Class I nt. C 1 a s! Application Number: Lodged: Complete specification Lodged: Accepted: Publi,shed: 1 Priority: P~elated Art: his docuent coJn!-tlW t amer~dments made uncl-, Scetion 4 9 and is correct for P rFn ti n.

Name of Applicant: ':h'iress of Applicant: Ac* a 4 nrntrs 04 TdX iass for Service: APPLE COMPUTER, INC.

20525 Mariani Avenue, Cupertinio, California 95014, United States of America RICHARD N. WOOLLEY NEAL GLOVER RICHARD WILLIAMS DAVIES COLLISON, Patent Attorneys, 1 Little~ Collins Street, Melbourne, 3000.

Complete specification for the invention entitled: "A METHOD FOR STORING DATA" The following statement is a full description of this invention, including the best rrsthod of performing it known to us 1- Irving S. R Fpozt ''V Note. Initial all atrations. Associate .General..Co'nsel,.....

DAVIES COLLISON. MELBOURNE and CANBERRA, 2 BACKGROUND OF THE INVENTION .Field The present invention relates to the field of error detection, and more particularly, to a method for storing data to facilitate error detection in a data processing system.

2. Art Background 00 00 o o a o 0 a IThe complexity of modern data processing o a 9 a, systems requires that some means be employed in order 0 to detect errors in stored, retrieved and otherwise 4" 15 manipulated data bits. For example, errora may be introduced into data as a result of mechanical and/or electrical variations in the data processing system, such as foreign material disposed on the magnetic read-write heads of a tape or disk mass media storage device. Other potential sources which may introduce errors in, or otherwise alter, data bits include the nitural wearing of the storage medium, variations in tht power supply voltage for the data processing system, as well as "soft" errors induced by radioactive impurities in the housing of the storage device and cosmic rays.

A variety of methods have been developed in order to detect errors introduced into stored or otherwise manipulated data. One comanon error detection system includes the use of a simple check-sum code. Typically, data is organized in terms of records which comprise a plurality of bytes. Each byte in turn is comprised of a plurality 3 of bits (typically A system employing the use of a simple check-sum method performs an exclusive-OR operation between each sequential byte with,.n the record. Thus, as illustrated in Figure 1, the 8 bits comprising byte A 1 are exclusive-ORed with the 8 bits comprising A 2 Similarly, the resultant binary quantity of the operation between A 1 and A is exclusive-ORed with the bits comprising byte

A

3 and so on. The check-sum is defined as the net resultant binary quantity obtained from these repetitive exclusive-OR operations. The resultant check-sum quantity is typically inserted at the end of the record, as illustrated in Figure 1 as the A i In many data processing systems, data is written onto a magnetic recording media such as a magnetic disk or tape. In those systems employing tho use of a check-sum error detection method, the data in the form of numerous bytes comprising the entire record, is written sequentially onto the magnetic medium such that byte Al is written first with subsequent bytes then written, and finally one or more bytes comprising the check-sum resultant is disposed at the end of the record. When retrieving 25 the data from the magnetic medium, the data processing system sequentially reads the content of the entire record while recalculating the value of the check-sum. After the entire record has been read from the recording medium, the recalculated check-sum value is compared with the check-sum previously recorded as part of the record. If the two check-sums do not identically match, an error in the data is assumed.

00 00 o o 0 0 00ooo00 0o 0 a ea 00 0 0 0 0 0a I0 0 9 Sa0 4O X0 0a0 t 00 The use of a simple check-sum error detection method is not as protective as one naturally assumes. For example, the simple check-sum method has inherent pattern sensitivity for short burst errors randomly distributed throughout the record. Thus use of an exclusive-OR operation between sequential bytes within the record renders it possible for errors to cancel one another, and therefore not be detected. For example, if the bit state of bit #2 within a particular byte of the record is altered as a result of a mechanical and/or electrical malfunction a 1 is shifted to a 0 bit state), and bit #2 of another byte is similarly altered (a 0 bit state is inadvertently shifted to a 1 bit state), the errors in the bytes will cancel during the exclusive-O. operation. In the case of two random bits in error, the probability of an error misdetection is approximately one out of eight.

20 One method for improving the error detection rate for the simple check-sum is to add "interleaving". For example, a 3 byte check-sum generates three separate resultant check-sum values for a record. As illustrated in Figure 2, the first 25 check-sum value may be determined for bytes A 1

A

2

A

3 etc. which represent sequential check-sum interleaved bytes. Similarly, a second check-sum would be calculated for the bytes B 1

B

2 B3 etc., and a third for bytes CI, C 2

C

3 etc.

Although interleaving significantly lowers the misdetection probability for long random error bursts, the misdetection probability for sparse error bursts has been found to be unacceptably high.

~I~

Accordingly, there exists a need to provide a means to increase the detcctability of errors in stored and manipulated data.

SMMARY OF HE INVENTION According to the present invention there is provided a method of processing data in a data processing system for storage to improve error detection in said data, said data being organised into a plurality of sequential interleaved bytes, including first and second bytes, said method comprising: generating first and second check-sunm signals representative of first and second check-sums corresponding to interleaved bytes of said data; applying a randomisation function to said second check-sum signal to generate a randomised check-sum signal; o altering said first check-sum signal by adding said first byte to said first check-sum signal; 15 generating a first storage signal by performing an exclusive-OR operation between said randomised check-sum signal and said first byte; and Qo c storing said storage signal in said data processing system; whereby an error in one of said bytes is propagated through subsequent bytes, thereby increasing the detectability of said error.

0 a a o o0 O BRIEF DESCRI QNlNJOF T R DLWNGS Figure 1 is an illustration of a simple check-sum error detection system.

Figure 2 is an illustration of an interleaved check-sum error detection system.

25 Figure 3 is a block diagram disclosing a computer incorporating the teachings of the present invention, Figures 4 illustrate one preferred 14,ippletJp,S N r$yi 6 storage and retrieval method, Figure 5 illustrates the propagation of an error using the technique of Figures Figures illustrate another preferred storage and retrieval method for propagation and randomization of an error.

Figures illustrate a preferred storage and retrieval method of the present invention for randomizing and propagating an error using interleaved check-sums.

t 4 NOTATION_ AND NOMENCjAT lRE The detailed description which follows is presented largely in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing art to most effectively convey the substance of their work to others skilled in the art.

An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. These steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It proves convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like. It should be kept in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities.

Further, the manipulations performed are often referred to in terms, such as adding or comparing, which aLe commonly associated with mental operations performed by a human operator. No such capability of a human operator is necessary, or :desirable in most cases, in any of the operations described herein which form part of the present invention; the operations are machine operations.

Useful machines for performing the operations of the present invention include general purpose digital computers or other similar devices. In all cases the distinction between the method operations in operating a computer and the method of computation itself should be noted, The present invention "O relates to method steps for operating a computer in processing electrical or other mechanical, chemical) physical signals to generate other desired physical signals.

The apparatus for performing these operations may be specially constructed for the required purposes or it may comprise a general purpose computer as selectively activated or reconfigured by a computer program stored in the computer. The algorithms presented herein are nob inherently related to any particular computer or other apparatus. In particular, various general purpose machines may be used with the teachings herein,, or it may prove more convenient to construct 8 more specialized apparatus to perform the required method steps. The required structure for a variety of these machines will appear from the description given below.

DETAILED DESCRIPTION The following detailed description will be divided into several sections. The first of these 10 will treat a general system arrangement for storing, Qo eQ S, retrieving and manipulating data. Subsequent sections deal with apparatus and methods for S obtaining improved error detection capability in a data processing system.

In addition, in the following description, numerous details are set forth such as algorithmic conventions, specific numbers of bits, binary operations, etc., in order to provide a thorough S 20 understanding of the present invention. However, it will be apparent to one skilled in the art that the present invention may be practiced without these specific details. In other instances, well-known circuits and structures are not described in detail in order not to obscure the present invention unnecessarily.

GENERAL SYSTEM CONFIGURATION Figure 3 illustrates a typical computer-based system for reading, storing and manipulating data. Shown there is a computer which comprises three major components. The first of these is the input/output circuit 22 which is L9 used to communicate information in appropriately 7 structured form to and from the other parts of computer 20. Also shown as a part of computer 20 is the central processing unit (CPU) 24 and memory 26.

As illustrated, CPU 24 also includos a writing/reading means 25 for writing data and reading data out of memory 26. These latter two elements are those typically found in most computers. In fact, the several elements contained within computer 20 are intevided to be representative of this broad category of data processors. Particular examples of suitable data processors to fill the role of computer included machines manufactured by Apple Computer, Inc., in Cupertino, California. Other computers having like capabilities may, of course, be adapted in a straightforward manner to perform the several functions described below.

Also shown in Figure 3 is an input device 30, shown in typical embodiment as a keyboard. It should be understood, however, that the input device may actually be a card reader, magnetic or paper tape reader, or another well-known input device (including, of course, another computer). A mass memory device 32 coupled to the 1/0 circuit 22 provides additional storage capability for the computer 20. The mass memory may include other programs and the like and may take the form of a magnetic or paper tape reader or other well known device. It will be appreciate', that the data retained within mass memory 32, in appropriate cases, may be incorporLtted in a standard fashion into computer 20 as part of memory 26.

In addition, a display monitor 34 is illustrated which is used to display the data being manipulated by the present invention. Such a display monitor may take the form of any of several well-known varieties of CRT displays.

In the course of operation of computer errors may be introduced into data which is stored and retrieved from memory 26 and mass memory 32. The failure to detect errors in stored or otherwise manipulated data may generate spurious computer responses. As will oe disclosed, the present invention provids a method for increasing the detectability of errors in stored or otherwise manipulated data, thereby significantly improving the reliability of computer operations.

EPR EM P <N_QZ.E.RRS..

As previously discussed, the use of interleaving significantly decreases the misdetection probability in the case of long random error bursts within a record. However, it has been found that the likelihood of error misdetection for sparse error bursts randomly distributed throughout the record remains unacceptably high. Therefore, one aspect of the present invention operates to propagate an error throughout the record, thereby increasing the likelihood of detection.

oe Referring now to Figure 4,ta pron avra.ontiAn' method for the propagation of errors is disclosed. Individual bytes of data A, B, C and D, which may comprise sequential interleaved bytes in a record, are to be stored or otberwis mnipulated in a computer system. The .proeont in'ention\directly stores the first byte A in, for example, memory 26 or mass memory 32, which may comprise a magnetic storage medium or the like. As illustrated, the contents ot the next byte, B, is exclusive OR-ed with the contents of previous byte A, and the binary quantity resulting from this exclusive-OR operation between bytes B and A are stored, Similarly, the contents of previous byte B, and that resultant binary quantity is also stored. Thus, unlike prior art encoding methods for string data in a storage device, the SrQ..4ntIW.'o..iG.stores the result of a series of e xclusive-OR operations between sequential (or interleaved) bytes of a record.

Referring now to Figure the recovery of data stored in accordance with a scheme of Figure 4(a) is described. As illustrated, data is decoded 29 from the resultant binary quantities stored in accordcance with the method described in conjunction with Vigure by the use of additional exclusive-OR operations, As shown, the retrieved resultant binary quantities are exclusive-ORed with 2D the decoded results obtained from the immediately preceding (or interleaved) byte. For example, the retrieval from mass memory 32 or memory 26, of byte A results simply in binary quantities representative of the contents of byte A inasmuch as byte A was retrieved without alteration within the storage means. However, the retrieval of the subsequently stored resultant quantity (B 0 A) from storage means 32 or the like is exclusive ORed with the previously decoded byte, namely A in the present 12 example. It will be aparent, that the exc. OR operation between the stored resultant quantA, of Figures and the previously recovered data (for example, byte A) effectively decodes and recovers byte B. Similarly, an exclusive-OR operation between the recovered byte B and encoded quantity (C 0 B) results in the recovery of byte C. Thus, the method illustrated in Figure 4 provides a means whereby data may be encoded and stored in a storage device and later decoded utilizing a series of simple exclusive-OR operations.

Referring now to Figure 5, the propagation of a random error in a data record s illustrated.

In accordance with the teachings of the Sthe first record byte A is stored within the desired storage means such a mass memory 32 or momory 26 of computer 20, Subsequent (whether consecutive or interleaved) byte B is exclusive-ORed with the contents of byte A with the resultant quantity stored in the desired storage means.

Similarly, subsequent bytes C, D, etc., are exclusive-ORed with previous data bytes in accordance with the method of Figure 4(a).

Assume for sake of example, that an inadvertent error, "e"t is present within byte A of a :-aord. This error may be introduced during the readback or original write cycle or as a result of any one of many possible previously described causes. Thust the retrieval of the first byte of a record results in a quantity representative of byte A e.

*-h-'quantity (A I2 e) is exclusive-oRed

VALIA,

13 with the subsequent stored resultant quantity (B G The net result from the exclusivc-OR operation is the quantity (B 0 Thus, error e Las been propagated into subsequent byte B, and as is apparent from Figure 5, error c is similarly propagated throughout each byte of the record. In operation, a data processing system incorpoiating a simple chec, sum procedure would compare the recalculated check sum value based on the retrieved data illustrated in Figure with the previously stored check sum quantity for the data originally stored pursuant to the procedures of Figure Due to the propagation of the error in accordance wvith the preferred method, the check sums would not match and an error would be assuimed, aE PATLANID hilRAND MlZATIM Referring now to Figure 6, another preferred method will be described which both propagates and randomizes errors within a data record, As previously used, A, B, C and D represent consecutive bytes (whether sequential or interleaved) within a rcoau. The notation prime represents the function applied twice, and so on, As pteviously disclosed, the first byte A is directly stored in the particular memory nmans used. As illustrated, the contents of the next sequential byte B, is S 2( exdlusive-ORed with the contents of the previous

A>

xC WMA'A'sin'013'appleo spe'13 ,a 13a byte A to which a randomizing function has been applied. The binary quantity resulting from this exclusive-OR operation between bytes B and A' are then stored in the storage means. Similarly, the contents of sequential byte C are exclusive-ORed with the contents of previous byte B to which the function has been applied, and the resultant binary quantitites are then also stored. It will be appreciated, that the particular randomizing function may comprise a variety of well known randomization algorithms, however, in the present preferred method a linear randomizing function is utilized.

Referring now to Figure the effect of an error randomly located in a data record is illustrated, Assume for sake of example, that an inadvertent error is present within byte A of the record. As shown in Figure the retrieval of the first byte of a record results in a quantity representative of byte A 0 e. The randomizing function is applied to the quantity A 0 e resulting in the quantity (A 0 This quantity (A is then exclusive-ORed with the subsequently stored resultant quantity B 0 The net result from this exclusive-OR operation is the quantity B 0 where c' equals A' 0 (A 0 Thus, error e has been propogated into subsequent byte B, and has been randomized in accordance with a chosen randomizing funciton, Similarly, the randomizing function is applied to the resultant B 0 c' thereby obtaining (B which is in ,urn exclusive-ORed with the previously stored quantity C thereby resulting in the retrieved binary quantity C 0 where e" equals B' 0 (B 0 Thus, the randomizing function function has been applied to error e twice thereby randomizing the previously randomized error as it is propogated through the record, This procedure is repeated as data is sequentially read from the record, thereby randomizing and propagating the error throughout the entire record and significantly S wsp01apple 13 ~0 44 o 4 4 q 2 II 4 4, 4 4 444 14 increasing the probability of detection utilizing check-sum techniques.

Referring now to Figure 7(a) at) embodiment of tlhe present inet' ~idslsd Although the p .ew -4vonio has~been discussed utilizing randomizion and propagation as applied to sequential or interleaved bytes in a data record, it has been found that high detection probabilitvy is achieved by applying the teachings of the~p.e4sea64 -Inent-en directly to the check-sum quantities themselves, As shown in Figure a data record.

includes bytes A, B, C, D, etc. Check-sum quantities Cl, C2 and C3 are located at the end of the data record, As illustrated, the preferred embodiment ci the present invention utilizes a 3 byte interleaved check-sum system, wherein bytes A, D, G, etc., comprise sequential interleaved bytes of check-sum Cl. Similarly, sequential interleaved bytes B, Ee H, etc., comprise bytes of check-sum C2. Finally, bytes C, F, I, etc., comprise sequential interleaved bytes of check-sum C3. In addition, it will be noted that indicates a carey from a previous operation.

As illustrated in Figure the randomizing function of the embodiment in Figure 7 comprises a "rotate" wherein the contents of check-sum C3 are shifted one bit to the left (nomely least significant bit toward most significant bit) and the most significant bit is rotated to the least significant bit position, It has been found that the use of a rotate for randomization permnits real time implementation in a variety of computer systems.

i As shown in Figure 7(c, data encodi in accordance with the teachings of the present invention includes an initial rotation of check-sum C3 (which at the beginning of the procedure will simply result in the rotation of binary zeros j inasmuch as no data has previously been stored).

Check-sum Cl is then set equal to check-sum Cl plus the contents of byte A (plus any carry At intialization, Cl will simply equal the cont'-ts of byte A. As shown, the first byte is set equal to binary quantity A exclusive-ORed with the contents of check-bum C3, and this binary quantity is then stored in an appropriate storage means. For the first byte of a record, the contents of byte A will simply be stored in mass memory 32 or memory 26 of computer since C3,is set initially to zero. Check-sum quantity C2 is set equal to the previous content of C2 plus the binary quantity comprising byte B as well as any carry which exists. Byte B of the record is then exclusive-ORed with the contents of check-sum Cl and this resultant quantity is stored in the appropriate storage means. Check-sum C3 is then set equal to its previous contents plus the contents of byte C plus carry. The contents of byte C is exclusive-ORed with the contents of check-sum C2 and this quantity is then stored. Once the first three bytes, A C3, B Q C1, and C C2, are stored in the storage means in accordance with the previous discussion, the contents of check-sum C3 are then once again rotated by a one bit shift as shown in Figure 7(a).

The contents of check-Gum Cl are then added to the contents of byte D of the record plus any 16 resultant carry. The contents of byte D are exciusive-ORed with the contents of check-sum C3 and this resultant quantity is then stored in the storage means. As shown, the contents of check-sum C2 are t-hen added to the contents by byte E of the record plus any carry in order to redefine the contents of check-sum C2. Similarly, subsequent bytes comprising the record illustrated in Figure 7(a) are stored in the appropriate storage means, and check-sums, Cl, C2 and C3 are accordingly updated.

inReferring now to Figure recovery iaccordance with the encoding scheme disclosed in Figures will be discussed. Prior to retrieval, check-sums Cl, C2 and 03 are defined within computer memory 26 of computer 20, and are set equal to zero. it will be recalled, that the resultant end of record check-sum quantities Cl, C2 and 03 are stored at the end of the record illustrated in Figure As shown the retrieved quantity A 0D 3 is exciusive-ORed with the contents of check-sum C3 (which equals 0) thereby resulting in a true retrieval of the contents of byte A if A 0NC3 contains no errors. Once byte A is retrieved, as shown, CI is updated by add~lng contents of CI with byte A plus any carry (none in the present example).

Similarly, the previous binary quantity B QCl is exolusive-ORed with the newly updated value of Cl, resulting in the decoding and retrieval of byte B.

02 is then updated by adding the contents of byte B (plus any carry). Byte C is retrieved by exclusive-QRing the previously stored quantity C 02 with the newly updated value of C2. 03 is then updated by adding the value of byte C, and then 1C 17 performing a rotate as previously described. In this fashion, sequential bytes comprising the record are decoded and retrieved.

CODING DETAILS No particular programming language has been indicated for carrying out the various procedures described above. This Is in part due to the fact that not all languages that might be mentioned are universally available. Each user of a particular computer will be aware of the language which is most suitable for his immediate purposes. In practice, it has proven useful to substantially implement the present invention in an assembly language which provides a machine executable object code.

Because the computers and the monitor systems which may be used in practicing the instant invention consist of many diverse elements, no detailed program listing have been provided. It is Sconsidered that the operations and other procedures i described above and illustrated in the accompanying drawings are sufficiently disclosed to permit one of ordinary skill to practice the instant invention or so much of it as is of use to him.

Thus, methods and apparatus which are most advantageously used in conjunction with a digital computer to provide improved error detection have been disclosed,, The present invention's randomization and propagation of errors throughout a data record significantly reduces the error misdetection probability and thereby increases the reliability of a data processing system.

Claims (8)

1. A method of processing data in a data processing system for storage to improve error detection in said data, said data being organised into a plurality of sequential interleaved bytes, including first and second bytes, said method comprising: generating first and second check-sum signals representative of first and second check-sums corresponding to interleaved bytes of said data; applying a randomisation function to said second check-sum signal i generate a randomised check-sum signal; altering said first check-sum signal by adding said first byte to said first check-sum signal; 4 generating a first storage signal by performing an exclusive-OR operation between said randomised check-sum signal and said first byte; and storing said storage signal in said data processing system; whereby an error in one of said bytes is propagated through subsequent bytes, thereby increasing the detectability of said error.

2. A method as claimed in claim 1, further including: °o 20 altering said second check-sum signal by adding said second byte to said 0o li;econd check-sum signal; generating a second storage signal by performing an exclusive-OR operation between said first check-sum signal and said second byte; and storing said second storage signal in said data processing system. 00 00 0 0 oo°

3. A method as claimed in claim 2, further including generating a third check- sum signal representative of a third check-sum of bytes of said data interleaved by three.

4. A method as claimed in claim 3, wherein said randomisation function comprises a rotational shift operation applied to bits of said second check-sum signal, ZE !N Z 9l2,dbwsp.0j4,pplelpm,l18 19 A method as claimed in claim 4, wherein said nethod is applied to each of said sequential interleaved bytes, such that encoded representations of each of said bytes is stored.

6. A method as defined in any one of the preceding claims, further including a method of recovering stored data, comprising: altering said first and second check-sum signals so as to represent a zero value; accessing said first storage signal and recovering said first byte by performing an exclusive-OR operation between said first storage signal and said second check-sum signal; and and altering said first check-sum signal by adding said first check-sum signal to said first byte.

7. A method as claimed in claim 6, further including: accessing said second storage signal and recovering said second data byte by performing an exclusive-OR operation between said second storage signal and said first check-sum signal; altering said second check-sum signal by adding said second check-sum signal to said second byte; and altering said second check-sum signal again by applying said randomisation function thereto. i

8. A method as claimed in claim 7, wherein said method is applied to each of said sequential interleaved bytes to recover said data from storage in said data processing system, u It4,ippkl Jspe,19 L 20

9. A method of processing data in a data processing systrn substantilly as hereinbefore described with reference to Figures 7(a) to 7(d). DATED this 12th day of September, 1990. APPLE COMPUTER, INC. By its Patent Attorneys DAVIES COLLISON q~ 04 0 8t 4 0* 0~ 90912,bwspeO14,appl