GB2300537A

GB2300537A - Differential JPEG encoder

Info

Publication number: GB2300537A
Application number: GB9508788A
Authority: GB
Inventors: Baruh Hason; Zufnat Hagin; Mordechay Shor
Original assignee: Motorola Israel Ltd
Current assignee: Motorola Solutions Israel Ltd
Priority date: 1995-04-29
Filing date: 1995-04-29
Publication date: 1996-11-06
Anticipated expiration: 2015-04-29
Also published as: GB9508788D0; GB2300537B

Abstract

A transform operation (40) is performed on a block of image data to provide a block of transformed image data. A subtraction operation (41) is performed on the block of transformed image data relative to a stored block (44) of transformed image data to provide a block of difference data. A quantization operation (42) is performed on the block of difference data to provide a block of quantized difference data and the block of quantized difference data is encoded (43). A video encoder is also provided. The encoder performs a subtraction operation (41) to provide difference data in the transform domain which reduces the amount of computation required in prior art differential JPEG encoders.

Description

METHOD OF ENCODING IMAGES AND VIDEO ENCODER Field of the Invention This invention relates to a method of encoding images and is particularly applicable to, but not limited to, differential JPEG encoding.

Background of the Invention JPEG is a standard for compression of still pictures. JPEG is designed for compression of either full-colour or grey-scale images of natural real world scenes and is designed to exploit known limitations of the human eye, mainly the fact that small colour changes are perceived less accurately than small changes in brightness. Therefore, it is intended to compress natural pictures (as opposed to cartoons or text) which will be viewed by humans.

JPEG is a "lossy" procedure (there is also a lossless procedure but the compression rate is significantly lower), meaning that the decoded picture will not be exactly the same as the original picture. A useful feature of JPEG is that the quality, or in other words the degree of lossiness can be varied by the setting of the compression parameters. Therefore, the user can trade off compression rate against image quality.

Differential JPEG is a variation on JPEG applicable to sequential image frames enabling much greater compression per frame. In differential JPEG the difference between two consecutive frames is encoded, using the JPEG algorithm. Since JPEG is a lossy procedure the difference between two consecutive frames is defined as the difference between the current frame and the decoded former frame.

Differential JPEG is used for video encoding at frame rates of typically 25 or 30 frames per second. Existing differential JPEG algorithms are highly computationally demanding.

When transmitting video over a radio link, typical current standard radio links (eg. 25 KHz) have insufficient bandwidth to carry 25 or 30 frames per second even with full differential JPEG coding. In practice, it is possible to transmit one frame every few seconds, but reduction of the frame rate does not reduce the computational requirement per frame.

There is a need for an improved computation method of encoding differential images.

Summarv of the Invention According to the present invention, a method of encoding images is provided, comprising: performing a transform operation on a block of image data to provide a block of transformed image data; performing a subtraction operation on the block of transformed image data relative to a stored block of transformed image data to provide a block of difference data; performing a quantization operation on the block of difference data to provide a block of quantized difference data and encoding the block of quantized difference data.

By performing the subtraction operation to provide difference data in the transform domain, substantial simplification of the computation is achieved.

A preferred embodiment of the invention is now described, by way of example only.

Glossarv of terms JPEG - A standardized image compression mechanism named after the Joint Photographic Experts Picture Group.

DCT - Discrete Cosine Transform.

FDCT - Forward Discrete Cosine Transform.

IDCT - Inverse Discrete Cosine Transform.

Pixel - A basic picture element. In the preferred embodiment each pixel is made up of 24 bits. These 24 bits represent 3 colour components, 8 bits represent a Y component, 8 bits represent a Cr component and the remaining 8 bits represent a Cb component.

Component Block - A group of pixels representing only one components made up 8 rows and 8 columns. (64 pixels 8 bits each).

MCU - Minimum Coded Unit. In the preferred embodiment an MCU consists of two blocks of Y component, one block of Cb component and one block of Cr component (4:2:2).

Brief Description of the Drawings FIG. 1 illustrates a prior art JPEG encoder.

FIG. 2 illustrates a prior art JPEG decoder.

FIG. 3 illustrates a theoretical simplified version of a prior art differential JPEG encoder and FIG. 4 illustrates an encoder and encoding method in accordance with the preferred embodiment of the invention.

Detailed Description of the Drawings DCT based JPEG coding is the procedure for compressing a still picture to the JPEG compressed format. The main procedures for the DCT based coding procedure are illustrated in FIG. 1. The procedure described is done for each colour component.

FIG. 1 illustrates that the prior art JPEG encoder 10 comprises an FDCT function 11, a quantizer function 12 a Huffman encoder function 13.

The quantizer function 12 has associated quantization tables 14 and the Huffman encoder 13 has associated Huffman tables 15.

In the encoding process the input component is divided into blocks of 8*8 samples. Each block is transformed by the FDCT function 11 to 64 DCT coefficients. These coefficients are then quantized using specific quantization tables 14. After quantization the quantized coefficients are compressed even further by going through an Entropy encoding procedure (Huffman coding function 13).

It is worth noting that the only lossy procedure in the encoding is the quantization function 12. The FDCT is not a lossy transform since it is a unitary transform and therefore Parsevals' Theorem applies. In addition Huffman coding is also a lossless procedure. Since the only lossy procedure is the quantization procedure, each DCT coefficient is retrieved with an error proportional to its quantization step in the dequantization procedure.

DCT Based JPEG decoding is the procedure for decoding a compressed still picture back to a sampled picture. The main procedures for the DCT based decoding procedure are described in FIG. 2. In FIG. 2, a JPEG decoder 20 is shown comprising a Huffman decoder function 23, a dequantizer function 22 and an IDCT function 21. Each procedure 21, 22, 23 in the decoder 20 is the inverse procedure for the corresponding procedure 11,12,13 in the encoder 10.

The Huffman decoder 23 uses the same tables 15 as the encoder 13.

The dequantizer function 22 uses the same quantization tables 14 as the quantizer 12.

The Huffman decoder 23 decodes the quantized DCT coefficients.

The dequantizer 22 dequantizes the DCT coefficients with an error related to the quantization step of each coefficient. After the dequantization the DCT coefficient are transformed back to a block of 8*8 samples by the IDCT module 21.

A differential JPEG algorithm encodes the difference between two consecutive frames. Based on the assumption that there is little difference between two consecutive frames, the compressed data of the difference is significantly smaller than the compressed frame data.

Since JPEG is a lossy procedure the difference encoded is the difference between the current source image frame and the reconstructed former image frame. The reason that this difference is taken (and not the difference between two source frames) is because the compression is a lossy procedure. If the difference between two source frames is encoded and the current frame is decoded by adding the decoded difference to the former decoded frame, there is an accumulating error. On the other hand if the difference between the current source frame and the former decoded frame is encoded, there is still an error from the compression of the difference itself but since the difference is always measured in relation to the decoded picture it is not accumulative and is only related to the difference being decoded.

Differential JPEG encoding is actually implemented in video compression standards as MPEG and H.261. However, in these video compression standards additional control is applied in order to compensate for motion and blocks with no change are not encoded at all.

The main procedures of the Differential JPEG Encoder (disregarding motion compensation) are described in FIG. 3. The figure illustrates a theoretical differential JPEG encoding procedures, employing the encoder 10 of FIG. 1 and the decoder 20 of FIG. 2. FIG. 3 illustrates a theoretical direct application of differential JPEG encoding assuming that the JPEG encoder 10 and the JPEG decoder 20 are available as closed modules A subtraction function 30 is provided, receiving source image data and providing differential data to the JPEG encoder 10. A delay or storing function 31 is provided at the output of the JPEG encoder 10, to store encoded difference data from one frame to the next. The JPEG decoder 20 receives the previous difference frame from the storing function 31. A further storing function 32 and an adding function 33 are provided.The adding function 33 sums the output of the storing function 32 to the output of the decoder 20 to provide reconstructed image data which is passed to the subtraction function 30 for subtraction from the incoming image.

Adding function 33 forms a reconstruction of the previous frame by adding the previous frame (retrieved from storing function 32) to the reconstructed difference data for the present frame from decoder 20. The reconstructed image is stored in storing function 32 for the next frame and at the same time is passed to the subtraction function 30 described.

By way of further explanation, it may be noted that for the first frame, storing function 31 stores a complete reconstructed image (rather than difference data), because there is no previous image to be subtracted in subtraction function 30. Upon the occurrence of the second frame being input to subtraction function 30, the complete image from storing function 31 is decoded in decoder 20 and stored in storing function 32, for later adding to the next reconstructed difference data from the decoder 20. Thus it is that the storing function 32 always stores a reconstructed image.

The reconstructed image fA(i1) is subtracted in subtraction function 30 from the source image f(i) and the difference DifF(i) is encoded by the JPEG encoder 10. The reconstructed image is decoded from the former difference Diffr'(i-l) and the former reconstructed image fA(i2). The procedure of decoding the reconstructed image is the same procedure that is done in the decoder 20. This means that a decoder 20 must be included in the encoder. Notice that in order to encode the difference Diff(i) the encoded difference DiffA(i1) and the reconstructed image f^(i-2) are needed. These are saved in memory stores 31 and 32 respectively by the encoder. The frame needed for DiffA(i1) is not actually a complete frame.

Memory for a single MCU is sufficient.

FIG. 3 also illustrates the decoding procedure for the reconstructed frame f^(i-1). The encoded difference Diff^(i-1) is decoded using the JPEG decoder. The former reconstructed image fA(i2) is added in adder 33 to the decoded difference to receive the decoded frame f^(i-1).

Although it is not illustrated in FIG. 3, the existing differential JPEG encoding procedure employs motion compensation. Motion compensation is a procedure in which the source image is compared with images in the vicinity of the previous frame, until the best match is found.

A motion vector is calculated between the present frame and the best match in the vicinity of the previous frame. The difference data calculated by subtraction function 30 is between the present frame and a transposed previous frame, that is to say a frame which is transposed by the motion vector. Motion compensation allows for a high rate of compression.

When encoding images at low frame rates, the search area necessary for calculating a motion compensation vector is large and the resultant benefit from employing algorithm has been found to be low. The inventors have identified that by removing motion compensation from an existing Differential JPEG encoder, and by rearrangement of the elements of FIG. 3, the improved algorithm of FIG. 4 can be provided.

The algorithm illustrated in FIG. 4 comprises a DCT function 40, a subtraction function 41, a quantization function 42 and Huffman decoder function 43. A storage function 44 receives the output of the quantization function 42 and stores the output for the next frame. Storing function 44 outputs a previous frame to an inverse quantization function 45 which provides its output to an adding function 46, which provides reconstructed difference frames to either storing function 47. Adding function 46 adds a previously stored reconstructed image frame from storing function 47 to a present reconstructed difference frame from inverse quantization function 45 and supplies a reconstructed image frame to subtraction function 41 for subtracting from a new source image from DCT function 40.

It may be noted that all of the operations 41 to 47 are carried out in the cosine transform domain.

Huffman coding is a lossless operation. Therefore the Huffman encoder and decoder pair of FIG. 3 can be eliminated from the loop and instead a single Huffman encoder 43 is placed at the output. The DCT and IDCT pair 11 and 21 present in the feedback loop can be eliminated as well and replaced by a single DCT operator 40 at the input. The resultant block diagram shown in FIG. 4 implements the differential JPEG compression by saving the computation needed for one IDCT and one Huffman decoder.

The differentiation procedure is performed in the DCT coefficients domain.

Comparing Figs 1 and 4, it can be seen that the only computational addition to conventional JPEG is one dequantization unit and two addition units. The computational complexity added by these extra modules is less than 20% with respect to original JPEG compression.

The memory needed for DiffA(i1) actually is not a complete frame.

Memory for a single DCT coefficient is sufficient.

In addition, if this algorithm is implemented by a VLSI circuit, silicon space can be saved due to the elimination of the IDCT and Huffman decoding operation.

The encoder output of the algorithm of FIG. 4 is the same as in the algorithm in FIG. 3, so the decoder is the same as decoder 20 shown the lower part of FIG. 3.

Claims

1. A method of encoding images comprising: performing a transform operation on a block of image data to provide a block of transformed image data; performing a subtraction operation on the block of transformed image data relative to a stored block of transformed image data to provide a block of difference data; performing a quantization operation on the block of difference data to provide a block of quantized difference data and encoding the block of quantized difference data.

2. A method according to claim 1, further comprising the steps of: storing the block of quantized difference data to provide a block of stored quantized difference data; inverse quantizing the block of stored quantized difference data at a later time to provide a block of reconstructed difference data; and reconstructing a block of transformed image data from the block of reconstructed difference data for performing a subsequent subtraction operation on a subsequent block of transformed image data.

3. A method according to claim 2, comprising the further step of storing the block of reconstructed difference data and, at a later time, adding it to a new block of reconstructed difference data to provide a new block of reconstructed transformed image data.

4. A method according to any one of the preceding claims where the step of performing the transform operation comprises performing a discrete cosine transform operation.

5. A method according to any one of the preceding claims wherein the step of encoding comprises Huffman encoding.

6. A video encoder comprising an input for receiving blocks of image data; a processor having a transform operation function for performing transform operations on consecutive blocks of image data received at the input, to provide consecutive blocks of transformed image data; and a storage element for storing blocks of transformed image data; the processor further having: a subtraction function operatively coupled to the storage element for subtracting a stored block of transformed image data from a block of transformed image data from the transform operation function to provide a block of difference data; a quantization function for performing quantization of the block of difference data to provide a block of quantized difference data and an encoding function for encoding the block of quantized difference data.