GB2295255A

GB2295255A - Data compression and decompression

Info

Publication number: GB2295255A
Application number: GB9422856A
Authority: GB
Inventors: Paul Stuart Adams; Vincent Joseph Singleton; Benedict Daniel Gladwyn; Ian Charles Edwards
Original assignee: International Business Machines Corp
Current assignee: International Business Machines Corp
Priority date: 1994-11-12
Filing date: 1994-11-12
Publication date: 1996-05-22
Also published as: GB9422856D0

Abstract

Data representing differences of successive video frames (for example), Figs. 2, 3, is compressed by putting the positions and lengths of runs of zeros of at least a predetermined length into a data structure 70, Fig. 4, removing these runs from the original data, and providing the result 71, Fig. 4, together with the data structure 70, as the compressed data. The removed runs are put back in decompression. <IMAGE>

Description

DATA COMPRESSION AND DECOMPRESSION The present invention relates to a system and method of compressing and decompressing data.

In order to be able to efficiently store or transmit, for example, a sequence of digital images it is desirable to compress the sequence such that the amount of data constituting the compressed sequence is less than that constituting the original sequence. It is further desirable to be able to recover the original sequence from the compressed sequence.

Such compression can be utilised in, for example, video conferencing systems, video telephones or collaborative working systems.

The users of such systems communicate both aurally and visually. The ability to see the other party gives a more natural and interactive quality to the communication and allows the parties to more confidently evaluate their counter-part. In such systems there is a need to exchange large amounts of data to keep the displayed images of the parties updated. Further, a sequence of compressed digital images requires less bandwidth to transmit as compared to a sequence of complete digital images.

Digital image compression also finds application in storing image data on mass storage media, suc as CD-ROMS o magnetic storage media, to reduce the amount of storage space required to record a sequence of digital images.

Successive images generated, by image capture appatatiis, such as a video camera, dllt-ini a video conference or derived from a mass storage media are often substantially similar and vary only by small amounts.

Accordingly, attempts have been made use tllis pl-opel-ty in order to reduce the amount of data which needs to be exchanged to support n video conference o to record a sequence of digital images. The reduction is achieved by exc])anginq or storing, for example, only selective areas of the digital images 01 eliminating the areas which do not change between contiguous images.

Many comp1essio11 and decompression methods use run length encoding, such as techniques are described in, for example, "Data Compression Techniques and Applicatons , 1tardwal-e and Software Consiclel-ations'', by Gilbert Held, published by John Wiley & Son, second edition. A special character and a number are used within the data to be compressed to indicate the beginninq of a compressed run and the number of data units constituting the compressed run respectively.However, the use of such a special character is not appropriate within the context of compressing data comprising all possible values which can be represented using a single data unit as the result of such a use does not necessarily guarantee that the number of compressed data and decompression data will comprise less data units that the original data. For example, if each pixel of a digital image is represented using a single byte, the pixel can take values within the range zero to 255 inclusive. If a system is arranged to compress runs of zeros, the zero value could be used as the special character to denote the beginning of a run followed by a number indicating the length of the run. In this instance, all runs of zeros would have to be compressed even when it is not efficacious to do so.

Therefore, such compression could expand the data as a consequence of purportedly compressing any runs comprising a single byte. Any such expansion is undesirable.

US 5,298,992 discloses a method for coding a plurality of compressed video streams in a time ordered sequence. Each compressed data stream includes coding of frame to frame differences of a video segment, which are represented as a compressed MxN exclusive-or plane of pixel change values and location displacement control values for an output pointer into a decompressed video frame.

The above prior art is either unsuitable for compressing certain types of data or requires a complex algorithm to be followed in order to compress an image or given set of data. Further, in many systems the amount of memory available in which such compression and decompression can be effected is limited. There is no guarantee that the requisite memory will be available when compression or decompression is to be performed. Further, if more memory is available the acquisition or reservation thereof may be a relatively time consuming process which can slow down the overall operation of, for example, a data compression system.

Accordingly the present invention provides a method for compressing a sequence of data units comprising the steps of searching said sequence to determine the lengths and positions therein of runs of at least a predetermined number of contiguous data units having the same value, storing in a data structure said lengths and positions, producing a reduced set of data units comprising the sequence of data units less said runs of at least a predetermined number of contiguous data units having the same value, and forming a compressed data entity comprising the reduced set of data units and said lengths and positions stored in said data structure.

Compression according to the invention advantageously ensures that the number of bytes constituting the compressed data is always less than or equal to the number of bytes constituting the corresponding uncompressed data. Therefore, a fixed amount of memory can be reserved prior to compression in which the compressed data can be stored. The first set of data units and reduced or second set of data units are guaranteed to comprise no more data units than the original data from which they were derived.

Compression and decompression as described herein can be effected using any data set in which runs of zeros or other values occur and is not limited to processing digital image data. The sequence of data units or data set to be compressed can represent, for example, tie digital images used in a video conferencing system. Such data can take many forms. For example, the digital image may comprise many independently addressable data units, such as bytes, having values between zero and 255 inclusive. Accordingly, an exclusive-or operation or comparison or subtraction operation performed on a byte-by-byte basis to identify the differences between the current digital image and the preceding digital image would yield data which is amenable to compression using the present invention.

Compression and decompression according to the present invention is very fast and simple to implement. It finds particular application, when processing, for example, digital images, as a second stage of compression. The first stage being a comparison or an exclusive-or operation effected between to images to be compressed to identify any differences therebetween. The second stage is the removal of the runs of redundant data.

By using the method of the present invention in a screen sharing or collaborative working environment, a significant reduction in the amount screen data or bandwidth required to update the screens can be realised.

This advantage comes about as a consequence of only a small portion of the screen or a window in such an environment changing as between successive images. The remainder of the screen or the area outside the window currently of interest generally remains unchanged. Accordingly, when successive screens are subjected to an exclusive-or operation or a comparison or subtraction operation a significant number of data units will have the value zero.

The step of producing the second or reduced set of data units can comprise the step of removing from said sequence, using said stored lengths and positions, said runs of at least said predetermined number of contiguous data units having the same value to produce said reduced data set units. This step allows all maniuplations of data units and the storage of the lengths and positions to be effected within the same area of memory. For example, the buffer used to store the sequence of data units to be compressed can be used for such a purpose.

Alternatively, the step of producing the reduced or second set of data units comprises the step of copying from said sequence, for inclusion into said reduced set of data units, all data units except said runs of at least said predetermined number of contiguous data units having the same value. This step can requite additional memory to be allocated for storing the reduced set of data unit before compression is instigated.

Having compressed an original data set it is desirable to be able to reconstruct the original data set from the compressed data.

Accordingly the present invention provides a method for decompressing a reduced set of data units to produce an original sequence of data units using a data structure comprising lengths and positions of runs of at least a predetermined number of data units having the same value which were removed from said original sequence of data units, said method comprising the steps of identifying within the data structure said runs lengths and positions of runs of at least a predetermined number of contiguous data units having the same value, and producing an original sequence of data units using the reduced set of data units and said identified lengths and positions of runs of at least said predetermined number of contiguous data units having the same value.

Embodiments of the present invention will now be described, by way of example only, with reference to the accompanying drawings in which: figure 1 is a high level block diagram of a data processing system utilised to realise the present invention; figure 2 is a diagram of successive video frames for a video segment and associated exclusive-or planes; figure 3 is a diagram of an exclusive-or plane including data representing a segment of the plane; figure 4 illustrates two data sets representing first and second sets of data units derived as a result of compressing the exclusive-or plane of figure 3; figure 5 illustrates schematically a flow diagram for effecting compression according to an embodiment; figure 6 illustrates schematically a flow diagram for effecting decompression according to an embodiment;; figure 7 illusttates compression carried out within the same area of memory or buffer as is used to store the data to be compressed.

Referring to figure 1, there is depicted a block diagram of a personal computer system 10, such as an IBI1 PS/2 personal computer. The computer system 10 includes a 16-bit wide system bus 12. Connected to system bus 12 is a central processing unit (CPU) 14, for example an Intel 486 or equivalent microprocessor. CPU 14 passes data to and receives data from other devices attached to system bus 12 ovet- the system bus.

Traffic on the bus is controlled by a bus controller 16. An interrupt controller 18 handles interrupts passed between CPU 14 and the remaining devices.

Read only memory (ROM) 20 is non-volatile memory storing power on test processes and a basic input/output system (BIOS or ROM-BIOS). The system memory 22 is random access memory into which an operating system, such as the IBM OS/2 operating system, programs and data can be loaded for execution or manipulation. Additional hardware components of computer 10 attached to system bus 12 include a memory controller 24 and a system configuration store 26, provided by random access memory (RAM).

Auxiliary data storage is provided by peripheral controllers and associated storage devices including, a floppy disk or diskette controller 28 and drive 30, a hard drive controller 32 and hard drive device 34, and a compact disk (CD) read only memory (ROM) controller 36 with CD-ROM drive 38. Digital video signals, compressed as described herein, may be supplied to computer system 10 from compressed video data stored on portable media mounted in CD-ROM drive 38. After decompression by CPU 14, visual reproduction of video segments is carried out using an extended graphics array (XGA) controller 40 and suitable display 42. The XGA controller 40 includes a RAM buffer 44 in which the current frame for display is stored. In some computers, the buffer 44 may be loaded directly from system memory 22 or from an auxiliary storage controller.

In system 10, the CPU 14 handles loading of decompressed data for each new frame to the appropriate locations in buffer 44 for updating the frame.

Direct memory access (D!5A) controller 46 handles data transfers between auxiliary storage devices or other input/output devices, and system memory 22 without interaction by CPU 14. Keyboard controller 48 provides an interface to a keyboard 50, for user entries, and may be used to provide an interface to a "mouse". Parallel controller 52 is a device controller for an input or output device (e.g. a printer connected to computer 10 by a parallel cable).

Video signals may be derived from a number of sources, including graphics generated by an application program executing on CPU 14 or from a video camera 56 and an associated video interface 54 as used in video conferencing ot. collaborative working systems. The video interface 54 converts an analogue video signal, such as NTSC or PAL, into a digital video signal. The digital signal comprises a series of frames each being an array of bytes conveying intensity or colour information (e.g. RGB values) for respective pixels. The frames are arranged in a time ordered sequence.Digital video signals supplied by video input interface 54 are processed by CPU 14 to produce a compressed video signal comprising an initial frame and subsequent frame difference information for storage on a hard drive device 34 or other processing such as being transmitted to a remote terminal of a video conferencing or collaborative working system.

Figure 2 illustrates schematically the operation of the present invention in compressing a data set which comprises inter alia runs of zeros or some other value. A frame Fj of a digital video signal comprises an H by N array of pixels. Further illustrated are a time ordered sequence 62 of exclusive-or planes Pj. Exclusive-or planes Pi are M by N arrays of bytes related to adjacent frames Fj and Fjtl by the following function: Fl XOR Fj+1 = Pj where the XOR operator represents the exclusive or operation.A data unit or pixel in an exclusive-or plane Pj assumes a particular value where corresponding data units or pixels of consecutive frames Fj and Fj+ differ, and the value zero where the corresponding data units or pixels of consecutive frames Fi and F11 are the same. As a consequence of the exclusive-or operation an exclusive-or plane Pj contains data units (or bytes) or pixels which are, in the case where compared pixels were identical, zero or, in the case where the compared pixels were different, some other value. The data units of the exclusive-or plane Pj which are not zero represent the differences between the consecutive frames Fj and F1+1.

Figure 3 illustrates schematically an exclusive-or plane 64 in which a segment thereof 66 and 68 represents the result of an exclusiveor operation between corresponding data units of two consecutive video frames Fj and Foal. The exclusive-or plane 64 is represented at the byte level and comprises zeros and other values. Such an exclusive-or plane P1, comprising runs of zeros or some other value constitutes, a data set which is amenable to compression.

The CPU 14 compresses the exclusive-or plane Pj into two sets of data units 70 and 71. The first set of data units 70 comprises data units representing runs of at least a predetermined number of contiguous data units having the same value, such as zero, and a respective indication of the position of said runs within the exclusive-or plane Pi.

For example, a run of seven zeros commencing at position fourteen in the exclusive-or plane would be represented in the first 70 set of data units as 14,7.

Runs of data units in order to be efficaciously compressed or processed must comprise at least a predetermined number of contiguous zeros. The predetermined number is at least one more data unit than the total number of data units being used to represent the position and length of a corresponding run of contiguous zeros. For example, if the position and length of a run of zeros are represented using respective bytes, it would only be efficacious to compress runs comprising at least three bytes or at least three contiguous zeros. Using a single data unit to represent the length of a runs of zeros means that the maximum length of a run is governed by the number of bits constituting the data unit.

For example, if a single byte is used the maximum length of a run is 256.

The first 70 set of data units can be stored in tabular form or using some other data structures, such as a linked list, as are well known within the art. The data units used to represent the lengths and positions need not necessarily be the same size as the data units containing the data to be compressed. For example, the data to be compressed may be represented using bytes whereas the positions and lengths of compressible runs may be represented using words comprising 32 bits. The second or reduced set of data units 71 comprises data units constituting the exclusive-or plane Pi, or data to be compressed, less said runs of at least said predetermined number of contiguous data units having the same value.

Figure 4 depicts the first 70 and second or reduced 71 sets of data units after compression. Note that the example given has assumed that single bytes are being used to represent the lengths and positions of the runs of contiguous zeros and hence only runs comprising at least three zeros will be compressed. The first set of data units 70 comprises, for the forty data units illustrated in figure 3, the following pairs of values 1,6: 8,3; 14,7; 26,13 which represent the positions and lengths of the runs of consecutive zeros within the illustrated block of forty data units. Note that although in the embodiment described the positions are deemed to commence at location one, the positions can equally well start at zero in other embodiments.The second or reduced set of data units 71 comprises the vales 34, 35, 36, 32, 32, 45, 0, 0, 32, 43, 23 which represent the forty data unit less the runs of contiguous zeros which are amenable to compression. It can be seen that the two consecutive zeros at position twenty-three are not compressed as it is not efficacious to do so.

Although the above embodiment specifies the actual number of zeros constituting a run, the byte or data unit used to store the length of a run can be more effectively utilised as follows. An entry is only made in the table when a run comprising at least a predetermined number of contiguous zeros have been encountered. Therefore, the fact that such an entry has been made is indicative of the occurrence of such a run. The occurrence of subsequent zeros can therefore be represented using table entries of 0, 1, 2, 3, etc which should be decoded accordingly. For example, a table entry of 5 can be indicative of a run length of (predetermined number+6). It can be seen that the maximum length of a run which can be compressed is more than the maximum value which can be represented using the byte or data unit used to store the length by an amount equal to the predetermined number.Therefore, in the case in which the run lengths and the positions of respective runs are represented using single bytes and the data to be compressed comprises bytes, it is efficacious to compress runs of at least three zeros and the maximum length of a run which can be compressed is 259 bytes.

The first 70 and second or reduced 71 data sets can be output for further processing. For example, they can be transmitted to a video conferencing terminal in order to update an image of a party to the conference. Separate packets can be used to transmit the first and second sets in the conventional manner. The order of transmission of the first 70 and second 71 sets of data units has an impact upon the time taken to effect decompression. If the first set of data units is received after the second set, decompression is delayed until the first set of data units is received. Having separate first 70 and second 71 sets of data units provides the option of transmitting the decompression or first 70 set before the compressed or second 71 set or visa versa.

Alternatively, the two sets can be incorporated into the same packet and separated by a suitable delimiter or separately identified for subsequent transmission to a video conferencing terminal. A further means by which the two data sets can be combined is to provide an indication of the start or end of the first or second set of data within the combined data set. For example, the first two bytes of such a combined data set may be arranged to represent the position of the start of the second data set or the length of the first data set from which such a position can be calculated and the data units preceding such a given or calculated position will constitute the first set of data units 70 or visa versa.

Still further, the first 70 and second 71 sets of data units, after combination, can be separately identified by including a single zero value therebetween. When decompression is required, assuming the second data set 71 precedes the first 70, the beginning of the first data set 70 can be determined by searching backwards through the first data set 70, commencing with the last value, until the single zero is found. The single zero will be contiguous with the first value of the first data set 70 assuming that position values of compressed runs commence at position one.

The first 70 and second 71 sets of data units are associated or can be combined, either logically or physically, and constitute or can constitute a data entity. The data entity can then be manipulated, logically or physically, as such. For example, both the reduced or second set of data units and the data structure containing the lengths and positions can be stored on a storage medium as records or in some other suitable manner.

Figure 5 shows a flow diagram schematically illustrating the compression steps of an embodiment. The data 64 to be compressed is received and stored in a buffer at step 500. The memory required for the buffer is allocated prior to instigating compression or calling a routine which performs such compression. Such allocation can be effected if the maximum size of the data to be compressed is known in advance of compression. At step 510 a check is made to determined whether or not all of the data 64 to be compressed has been searched for runs of zeros which are amenable to compression. If all data has been so searched the process terminates otherwise processing continues at step 520. At step 520 the data 64 to be compressed is searched to identify a run of zeros comprising at least a predetermined number of contiguous zeros.At step 530 the start position and length of any such identified run is recorded in the first set of data units 70 or other data structure such as a table, linked list or an area of memory reserved for such a purpose.

Step 540 removes from the data 64 to be compressed any such identified run. The above steps are repeated until all data contained within the buffer has been processed and all runs which are amenable to compression have been identified and compressed.

Although the above embodiment removes runs from the data 64 to be compressed as they are identified, an embodiment can equally well be realised in which all such runs are first identified and recorded in the first set of data units 70 and then removed from the data 64 to be compressed in a later step using the data contained in the first set of data units 70.

Decompression according to an alternative embodiment is schematically illustrated by the flow diagram of figure 6. At step 600 the first 70 and second 71 sets of data units are received and respectively stored in a data structure, such as a table, and a buffer.

A second buffer is created and initialised to contain only zeros and pointers to the data contained in the first 70 and second 71 sets of data units are initialised to point to respective first data units contained therein at step 610. The pointer to the first 70 set of data units is used to access the lengths and positions contained within the table. The pointer to the second 71 set of data units is used to copy data therefrom into the second buffer. At step 620 a check is made to determine whether or not all of the runs contained within the table have been decompressed.

The first set of data units 70 is used to determine the appropriate positions of the data units constituting the second set of data units 71 within the second buffer. Pairs of values of the first set of data units 70 identify the position and length of respective runs of contiguous zeros. At step 630 the start position of a number of data units constituting part of the second set of data units 71 within the second buffer is determined using the first set of data units 70 as follows: START=POSITION OF RUN OF ZEROS + LENGTH OF RUN. At step 640 the number of data units which need to be copied from the second set of data units 71 into the second buffer is determined using START and the position of the beginning of the next run of contiguous zeros as follows: NUMBER = BEGINNING OF NEXT RUN OF ZEROS - START.Step 650 copies the above determined number of data units from the second set of data units 71 into the second buffer at a location indicated by START. Execution of the process then continues at step 620 until all of the data units contained within the first 70 and second 71 sets of data units have been processed.

Although the above embodiment decompresses the compressed data by copying data units from the second set of data units 71 into the second buffer, an embodiment can equally well be realised in which the second set of data units 71 is expanded by incorporating therein appropriate numbers of contiguous zeros at appropriate positions according to the data contained within the first set of data units 70.

A CD-ROM may store a video segment as data compressed in accordance with the invention. The CPU 14 may thereafter recover the segment in the form of the compressed video signal from drive 34 or drive 38 for reproduction of the digital video on display 42.

An embodiment of the present invention is illustrated in the portions of code shown in tables 1 and 2.

Table 1 illustrates C code which is suitable for implementing the compression embodiment. The compression code operates as follows.

1. Function call, Squish, compresses the data set pointed to by pointer data~buffer. Upon return from the function the pointer table~buffer points to the first data set comprising the lengths and positions of any runs of zeros which were compressed and the variable compressed~size contains an indication of the number of entries constituting the first data set 70. The compressed~size variable is initially set to equal the number of bytes constituting the data to be compressed.

Squish((char huge *)data~buffer, //The Buffer (long far *) & ompressed~size, //Size of compressed data (long huge *)table~buffer); //Buffer area for table It can be seen from the function parameters that the data set contained in data~buffer comprises data units which are eight bits long or single bytes. The data units which used to represent the lengths and positions of runs of zeros are each four bytes long. Therefore, it is only efficacious, using this embodiment, to compress run comprising at least nine bytes. Hence the value of MIN~COMPRESS should be set to nine.

Similarly, compressed~size uses four bytes to represent the number of entries in the table pointed to by table~buffer.

The reference numerals incorporated into the code are for the purposes of explanation only and do not form part of the code.

2. Line 1 of the function handles the passing and equating of parameters as is conventional in the art.

3. Lines 2 to 11 are declarations of variables used within the function. "ZeroCount" is used to record the total number of zeros constituting a run. "TableIndex" is used to access the entry position of the table pointed to by pTable. "SpaceSaved" is used to determine the total number of zeros compressed. "NonZeroCount" is used to determine the number of data units, or bytes in this case, encountered between consecutive runs of zeros, said runs being amenable to compression.

"CopyTo" defines a pointer which is used to access a location in data~buffer to which a byte can be copied. Pointer "CopyFrom" defines a pointer which is used to access a location in the data~buffer from which a byte can be copied. Pointer "pBaseAddr" points to the beginning of the buffer "pBuf" (which in turn, due to parameter handling in C, points to data~buffer). Pointer "pBufferEnd" defines a pointer which points to the end of the buffer which is accessed using "pBuf". The boolean variable "fCopyWaiting" indicates whether or not a compression of a run is outstanding. The boolean variable "fCanCompress" is used to indicate whether or not the current run of zeros is amenable to compression.

4. Lines 12 and 13 initialise the values of "CopyTo" and "CopyFrom".

5. Line 14 creates a for-loop which is arranged to step through the buffer, pBuf, a byte at a time.

5.1 At line 15 a check is made to determined whether or not the current byte, pointed to by pBuf, within the buffer is a zero.

5.1.1 If so, at line 16 the number of zeros encountered thus far is incremented by one and a check is made to determine whether or not enough zeros have been encountered to justify compression. As indicated above the value of MIN~COMPRESS is, for this embodiment, nine.

5.1.1.1 If so, a check is made to determine whether or not there exist an outstanding compression at line 17.

5.1.1.2 If such an outstanding compression exist, a block of contiguous, bytes defined by NonZerosCount, is copied from a starting location pointed to by CopyFrom to locations starting at CopyTo using the function hmemcpy as is well known in the art at line 18. The variable "fCopyWaiting" is set to FALSE at line 19.

5.1.2 If compression is justified, the beginning of the current run of zeros is determined and recorded in "CopyTo" at line 20 and the variable "fCanCompress" is set to equal TRUE at line 21. The variable "CopyTo" is set only once per run of contiguous zeros. The execution of the code then continues at line 14 as described in point 5 above.

5.2 If the current byte pointed to by pBuf is not a zero execution of the code continues at line 22.

5.2.1 A check, at line 23, is made to determine whether or not a run of zeros which is amenable to compression has been encountered. The variable "fCanCompress" will be equal to TRUE if compression is so justified.

5.2.1.1 If a such run of zeros has just been encountered the following action is taken. The boolean variable FCanCompress is set to FALSE at line 24. The variable NonZerosCount is set to zero at line 25.

The pointer CopyFrom records the location in the buffer, pBuf, of first non-zero byte after a run of compressible zeros at line 26. The table containing the lengths and positions of runs of compressible zeros is updated, at lines 27 and 28, to include the position and length of the most recently encountered run of compressible zeros. The variable SpaceSaved is updated, at line 29, to reflect the total number of zeros compressed thus far. As the program has reached the end of a run of zeros, the run is ready for compression. Accordingly, the "fCopyWaiting" variable is set to TRUE at line 30.

The if statement at line 31 is optional. It is used to determined whether or not a current run comprises more zeros than can be recorded in a single length entry of the table. For example, if a single byte were used to record the length of a run, then the value of MAX~COHP~TAB~ENTRIES may be 255. If such a single byte were used the table entries could be interpreted in a different manner in order to be able to accommodate runs of zeros comprising more than 256 data units.

Several possible alternative interpretations of the table entries now follow. If the convention that a zero is not ordinarily used to represent the position of a run of zeros, the occurrence of such a zero entry might indicate that the number of data units constituting the current run is to be extended by the number of zeros defined by the associated length value. For example, if a run of contiguous zeros comprises 290 bytes and had a position of forty-five, a table entries representing the run would be 45, 255 and 0, 35. Generally for each multiple of 256 bytes encountered other than the first multiple, an addition table entry of 0,255 would indicate that the current run of zeros should be extended by 256 bytes. Alternatively, rather than the zero being used to indicate a continuation of the current run, the actual position of the next run of 256 bytes can be recorded.Still further, the position value of the table entry can be used to represent the position of the current run of zero relative to the end of the previous run of zeros. The break statement exits, at line 32, the for-loop of a run of contiguous zeros has been encountered which potentially comprises more zeros than can be represented by a single entry in the table.

5.2.1.2 If a run of zeros has been encountered which is not amenable to compression, such zeros are classed as "non-zero" bytes for the purposes of compression or decompression according to the embodiment described. Therefore, NonZerosCount is updated to include the run of zeros which are not amenable to compression at line 34.

5.2.2 The number of zeros is reset to 0 at line 35.

5.2.3 The NonZerosCount is incremented by one at line 36.

5.3 The above is repeated until all of the data contained in the data~buffer has been processed i.e. execution continues at point 5 above.

6. Lines 37 to 44 deal with the scenario in which all of the data units in the buffer been processed and yet there is still an outstanding compression to be performed. The steps have the same effect as the above described equivalent steps.

7. Line 45 determines how much space was saved or the total number of zeros compressed.

8. Line 46 exits the function and returns the size of the table in terms of bytes.

Table 2 illustrates C code which is suitable for implementing the decompression according to an embodiment. The decompression code operates as follows.

1. The function call below instigates decompression of the second data set 71 using the first data set 70.

UnSquish((char huge *)data~buffer, //Compressed data (long far *)compressed~buffer, //Destination Buffer (long huge *)table~buffer, //Compression Table pointer (long)table~size, //Size of Table (long)original~size); //Original Buffer Size The second data set 71 or compressed data comprising various values and runs of zeros having less than the predetermined number of zeros is contained in a first buffer which is accessed using a pointer, "data~buffer". The decompressed data will be contained in a second buffer which is accessed using a pointer thereto, "compressed~buffer".

The pointer "table~buffer" points to the table comprising the lengths and positions of the zeros which have been compressed. The number of entries contained within the table pointed to by table~buffer is indicated contained in "table~size". The number of data units originally constituting the data set to be compressed is contained in the variable "original~size".

2. Lines 2 to 5 contain various variable declarations which are used within the function. "LoopIndex" is a count which is used to step through the entries contained within the table pointed to by PTable.

"IdxCompBuf" is used to access data units contained within the second data set 71. "IdxNonCompBuf" is used as an index to determine the position within the second buffer of data units which are proposed to be copied from the second data set 71 into the second buffer.

"NonZeroDataLen" is used to determine the number of contiguous data units of second data set 71 which are to be copied into the second buffer.

3. Line 6 determines whether or not compression was actually effected.

If the number of table entries is zero the data submitted for compression was not amenable thereto. Accordingly, at line 7, the data contained within the first buffer, pBufln, is copied into the second buffer, pUncompBuf and line 8 returns a TRUE value indicating that the second buffer, pUncompBuf, contains decompressed data. The return instruction exits the function.

4. As the functions Squish returns the number of bytes constituting the table, it is necessary to determine how many entries the table containes. Such a determination is conducted at line 9 of the code.

Each length and position is stored used a variable having a type of uint32. Therefore, the number of entries in the table is calculated by dividing the number of bytes constituting the table by the number of bytes used to represent a variable which has a type of unit32.

5. Line 10 determines whether or not a run of compressed zeros commenced at the first position of the data to be compressed. Such a determination is effected by checking to see if the position value of the first entry in the table is a one.

5.1 If not, the first run of compressed zeros did not start at the first data unit position of the data to be compressed and the first block of data from the second data set 71 should be copied into the buffer pUncompBuf. Such a copy is effected using the function hmemcpy at line 11. The number of bytes, in the instant case, to copied from the compressed data into the decompressed data buffer or second buffer is determined from the position information stored in the first position of the table as illustrated at line 11. The indices to the data units contained within the second buffer and the second data set are altered accordingly at lines 12 and 13.

5.2 If so, the indices to the data units contained within the second buffer and the second data set 71 are initialised to zero at lines 15 and 16.

6. Line 17 provides for termination of the following loop with one outstanding table entry to process.

7. Line 18 establishes a for-loop which steps through the table entries in order to decompress the compressed data.

7.1 Line 19 calculates the appropriate number of data units to be copied from the second data set 71 into the second buffer. The number is given by beginning of next run of zeros - beginning of current run of zeros length of current run of zeros.

7.2 Line 20 copies the above appropriate number of data units from the second data set 71 into the second buffer, puncompBuf, at positions determined by the respective indices associated therewith.

7.3 Lines 21 and 22 adjust the above indices according to the number of data units copied from the second data set to the second buffer.

8. Lines 23 and 24 determined whether or not there is any data left in the second data set which needs to be included in the buffer containing the decompressed data. If so, the remaining data is copied into the buffer pUncompBuf.

9. Line 25 exits the decompression code and returns a value TRUE upon exit.

Referring to figure 7, there is schematically illustrated compression which is conducted within the buffer storing the data to be compressed. The first and second or reduced sets of data units are created and manipulated within the same are of memory used store an image or sequence of data to be compressed. The data to be compressed is contained within a buffer 700. A pointer 710, pStart, points to the location of the first data unit 720, or byte, of the data to be compressed. The pointer 710, pStart, is used to access the data units contained within the buffer 700. It can be seen that some compression has already taken place. The table 730, containing the lengths, Li, and positions, Pj, of any runs of contiguous zeros which have been compressed, is stored in the buffer 700 at the end of the data to be compressed. A pointer 740, called pTable, points to the next available table position for receiving the position, P,, and length, L,, of the next run of contiguous zeros 760 which is amenable to compression. The pointer 740, pTable, is initialised to point to the eighth from last location of the buffer 700 containing the data to be compressed. Pointer 740, pTable, points to the eighth from last data unit because the position and length of a run of zeros are each represented using four bytes. A pointer 750, pEnd, points the location of the last data unit of the data to be compressed. Pointer 750, pEnd, is initialised to point at the last location of the buffer 700 containing the data to be compressed.

Compression is effected as follows. The data to be compressed is searched, using pointer 710, start, to identify the next run of contiguous zeros 760 which is amenable to compression. The position, P and length, L,, of the run 760 are determined. The location 770, D, of the data unit following the run 760 is determined. The data 780 between location 770 and location 750, pointed to by pEnd, is copied to overwrite the run of contiguous zeros. That is, the data stored at location 770 is copied into position P,, data stored in the location following location 770 is stored in the position following Pn and so on. This is repeated for all data units contained within the data 780 to be moved.As each data unit is moved, the pointer 750, pEnd, is decremented accordingly such that it always points to the end of the data to be compressed.

After the run 760 of contiguous zeros has been compressed the position, P,, and length, L,, are stored in the table at the location pointed to by the pointer 740, pTable. The pointer 740 is then decremented to point to the next potentially available table position.

During the searching, a check is made, as each data unit is encountered, to determine whether or not all of the data to be compressed has been processed. Such a check is effected by comparing the values of pointers pStart and pEnd. If they are equal, all of the data to be compressed has been processed. Therefore, the search for runs of contiguous zeros is continued until such a time as pStart equals pEnd.

Once all of the data to be compressed has been processed tlie buffer contains a second set of data units comprising only values other than zero and runs of zeros which are not amenable to compression. The table 730, or first set of data units comprises the lengths and positions of all runs of contiguous zeros which were amenable to compression. The buffer area 795 contains redundant data and can be ignored in any further processing. The first and second sets of data units can be output for further processing.

Decompression according to this embodiment is the converse of compression.

TABLE 1 * Function : Squish * * Purpose : Compress out as many zeros as possible from an XOR'd image * * inputs : pBuf - Pointer to the data to compress * * *lpSize - Size of data buffer ( changed to compressed size ) * * *pTable - on return contains the compression table * * return : the size of the compression table in bytes * 1 long Squish(char huge *pBuf,long far *lpSize, long huge *pTable) 2 uint32 ZerosCount=O: // number of adjacent zeros 3 uint32 TableIndex=O; // index into compression table 4 uint32 SpaceSaved=0; // total num of zeros removed 5 uint32 NonZerosCount=O; // Count of non compress data 6 char huge *CopyTo; // position to compress to 7 char huge *CopyFrom; // position to compress from 8 char huge *pBaseAddr=pBuf; // position of the beginning 9 char huge *pBufferEnd=pBuf+*lpSize; // position of the end 10 BOOL fCopywaiting=FALSE; // copy block to compress 11 BOOL fCanCompress=FALSE; // compress worth doing flag 12 CopyTo=pBuf; // point to start of buffer 13 CopyFrom=pBufferEnd; // point to the end of the buffer /* Main loop to ZIP though all the data looking for squishable zeros 14 for(;pBuf != pBufferEnd: pBuf++ ) 15 if ( !(+pBuf) ) // > ) do we have a zero ?? { 16 if ( ++ZerosCount == IN~CO?IPRESS ) // o compless ? 17 { if ( fCopyWaiting ) // outstanding copy? 18 hmemcpy( CopyTo, CopyFrom NonZerosCount );// compress zeros 19 fCopyWaiting = FALSE; // done it } /* Remember where these zeros started and remember we /* will need to compress these zeros out later 20 CopyTo = pBuf - (MIN~COMPRESS - 1) - SpaceSaved; // ptr to start of zeros 21 fCanCompress = TRUE; } //if many zeros } 22 else // was not a zero 23 { if ( fCanCompress ) // check to see if we have just had run of 0's 24 ç fCanCompress = FALSE; // > doing it now 25 NonZerosCount = 0; 26 CopyFrom = pBuf;// remember where this data starts /* > create a compression table entry [ offset, NumZeros] < */ 27 pTableiTablelndexj = (uint32)((CopyTo + SpaceSaved) pBaseAddr); 28 pTable[TableIndex+1] = ZerosCount; TableIndex += 2; 29 SpaceSaved += ZerosCount; // > rem how much we have saved 30 fCopyWaiting = TRUE; /* > Check to see if we haved used all tlle comp table space '*/ 31 if ( TableIndex > = N MAX~COMP~TAB~ENTRIES ) {// thats all we can do so exit 32 break; } 33 else if ( ZerosCount ) 34 NonZerosCount += ZerosCount; // zeros not compresses are NonZeros; 35 ZerosCount = 0; // > reset counter 36 NonZerosCount++; // > count the number of Non zeros } ) //for all data /* on the way out we could still be counting zeros, or just finished */ /* or .. etc etc ... so we clean up depending on the loop exit state */ 37 if ( fCopyWaiting ) // > data left to move 38 if ( (NonZerosCount > 0) || ( !fCanCompress ) ) 39 hmemcpy( CopyTo, CopyFrom, NonZerosCount ); 40 if ( fCanCompress ) // > some zeros on the end, so add table entry 41 { pTable[TableIndex] = (uint32)((CopyTo+SpaceSaved) pBaseAddr); 42 pTable(Tablelndex+1] = ZerosCount; 43 TableIndex += 2; 44 SpaceSaved += ZerosCount; 45 *1pSize -= SpaceSaved; 46 return( (Tablelndex * sizeof(uint32)) ); // > return size of table in bytes TABLE 2 * Function : UnSquish * * Purpose : Decompress a data buffer, compressed using Squish * * inputs : *pBufln - Pointer to the compressed data * * *pBufOut - The uncompressed buffer * * lpTable - Ptr to the compression table * * lTabSize - Size of the compression table * * return :BOOL : okay or fail * 1 BOOL UnSquish (char huge *pBufIn, char huge *pUncompBuf, long huge *pTable,long lTabSize, long OldSize ) 2 long Looplndex =O; // counter though data 3 uint32 IdxCompBuf; // Index into compressed buffer 4 uint32 IdxNonCompBuf =O; // Indx to non compressed buffer 5 uint32 NonZeroDataLen=O; // size of non zero block /* > check its worth processing < */ 6 if ( lTabSize == 0 ) 7 hmemcpy( pUncompBuf, pBufIn, OldSize ) 8 return(TRUE); } /* > convert the table size to be the number of uint32 entries < */ 9 lTabSize = (lTabSize / sizeof(uint32) ); 10 if ( pTable[Oi != 0 ) // > any data at start? { /* yes, so copy the first block of data to the front < */ 11 hmemcpy( pUncompBuf, pBufln, pTable[0] ); 12 IdxNonCompBuf = pTableioi+pTable (1); 13 IdxCompBuf = pTable[Ol; 14 else // NN first bit of data was compressed zeros 15 IdxNonCompBuf = 0: 16 IdxCompBuf = 0: 17 lTabSize -= 2; // exit the loop with one more table entry to process *1 /* whiz though the compression table, copying out real data between */ /* the zero'd gaps ( the new buffer must have been calloc'd 1* 18 for( ; LoopIndex < lTabSize; LoopIndex+=2) { /* len = next-data-ptr MINUS curr-data-ptr MINUS num-of zeros 19 NonZeroDataLen = pTable[LoopIndex+2] - pTabletLooplndex] pTable[LoopIndex+1]; 20 hmemcpy( pUncompBuf+IdxNonCompBuf, pBufIn+IdxCompBuf, NonZeroDataLen ); 21 IdxCompBuf+= NonZeroDataLen; 22 IdxNonCompBuf+= NonZeroDataLen + pTableiLooplndex+3]; // next spaces 23 if ( ((OldSize - IdxNonCompBuf) > 0) ) // any data left to copy ? 24 hmemcpy( pUncompBuf+IdxNonCompBuf,pBufln+IdxCompBuf, (OldSize - IdxNonCompBuf) ); 25 return( TRUE );

Claims

CLAIMS 1. A method for compressing a sequence (64) of data units comprising the steps of searching said sequence (64) to determine the lengths and positions therein of runs of at least a predetermined number of contiguous data units having the same value, storing in a data structure said lengths and positions (70), producing a reduced set of data units (71) comprising the sequence (64) of data units less said runs of at least a predetermined number of contiguous data units having the same value, and forming a compressed data entity comprising the reduced set of data units (71) and said lengths and positions (70) stored in said data structure.
2. A method as claimed in claim 1, wherein the step of producing comprises the step of removing from said sequence (64), using said stored lengths and positions (70), said runs of at least said predetermined number of contiguous data units having the same value to produce said reduced data set units (71).
3. A method as claimed in any preceding claim wherein, all of said steps are such that all data units are manipulated within the same area of memory.
4. A method as claimed in claim 1, wherein the step of producing comprises the steps of copying from said sequence (64), for inclusion into said reduced set of data units (71), all data units except said runs of at least said predetermined number of contiguous data units having the same value.
5. A method as claimed in any preceding claim, further comprising the step of performing a comparison between two digital images to create said sequence (64) of data units, said sequence (64) of data units comprising data representing the difference between said two digital images,
6. A method for decompressing a reduced set of data units (71) to produce an original sequence (64) of data units using a data structure comprising lengths and positions of runs of at least a predetermined number of data units having the same value which were removed from said original sequence (64) of data units, said method comprising the steps of identifying within the data structure said runs lengths and positions (70) of runs of at least a predetermined number of contiguous data units having the same value, and producing an original sequence (64) of data units using the reduced set of data units (71) and said identified lengths and positions (70) of runs of at least said predetermined number of contiguous data units having the same value.
7. A method as claimed in claim 6, further comprising, prior to said step of identifying, the step of identifying in a data entity said reduced set of data units (71) and said data structure comprising said lengths and positions (70) of runs of at least a predetermined number of contiguous data units having the same value.
8. A method as claimed in either of claims 6 or 7, wherein the step of producing comprises inserting into said reduced set of data units (71), using said identified lengths and positions (70), said runs of at least a predetermined number of contiguous data units to produce said sequence (64) of data units.
9. A method as claimed in claim either of claims 6 or 7, wherein the step of producing comprises creating a storage area which is initialised to contain all zeros, copying selectable data units from said reduced set of data units (71) into respective positions in said storage area, said positions being determined from said stored lengths and positions (70) identified in said data structure.
10. A method as claimed in any preceding claim, wherein said lengths stored in said data structure represent runs having lengths equal to said predetermined number plus a number of data units corresponding to the maximum value which can be represented using said data structure.
11. A method as claimed in any preceding claim, wherein said positions represent the number of data units between successive runs of at least a predetermined number of contiguous data units having the same value.
12. A system for compressing a sequence (64) of data units comprising means for searching said sequence (64) to determine the lengths and positions therein of runs of at least a predetermined number of contiguous data units having the same value, means for storing in a data structure said lengths and positions (70), means for producing a reduced set of data units (71) comprising the sequence (64) of data units less said runs of at least a predetermined number of contiguous data units having the same value, and means for forming a compressed data entity comprising the reduced set of data units (71) and said lengths and positions (70) stored in said data structure.
13. A system for decompressing a reduced set of data units (71) to produce an original sequence (64) of data units using a data structure comprising lengths and positions of runs of at least a predetermined number of data units having the same value which were removed from said original sequence (64) of data units, said system comprising means for identifying within the data structure said runs lengths and positions (70) of runs of at least a predetermined number of contiguous data units having the same value, and means for producing an original sequence (64) of data units using the reduced set of data units (71) and said identified lengths and positions (70) of runs of at least said predetermined number of contiguous data units having the same value.