WO2005107265A1

WO2005107265A1 - Method and device for post-processing image data

Info

Publication number: WO2005107265A1
Application number: PCT/IB2005/051322
Authority: WO
Inventors: Johan Puttenstein
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2004-04-30
Filing date: 2005-04-22
Publication date: 2005-11-10
Also published as: EP1757102A1

Abstract

The present invention relates to a method and a device for post-processing image data that has been compressed using a block transform scheme. The luminance values of pixels (12a) on one side of a block boundary (11) are made to adopt luminance values of pixels (12b) on the other side of the block boundary (11). Preferably, this is done by interchanging the values. This reduces the perceptual annoyance of blocking artefacts in still or video images, while high frequency content in the image data is retained.

Description

METHOD AND DEVICE FOR POST-PROCESSING IMAGE DATA

FIELD OF THE INVENTION The present invention relates to a method of post-processing image data, which image data is obtained by decoding compressed data, encoded in accordance with a block transform scheme, said method including changing picture element values, on opposite sides of a block boundary in the image data, in order to reduce the perceptual annoyance of blocking artefacts in said image data. The invention further relates to a corresponding device. BACKGROUND OF THE INVENTION Such a method is disclosed e.g. in US, 2003/0035586, Al. In this method, the image data is analysed to determine an artificial edge threshold. Pixel pairs along block boundaries are analysed in order to find out whether their luminance difference exceeds the threshold. If this is the case, their values are changed in order to "smooth" the image data and to reduce or eliminate blocking artefacts in the image data. A problem with such a method is that it requires extensive analysing of the image data, and is thus calculation intensive.

SUMMARY OF THE INVENTION It is an object of the present invention to provide a less calculation intensive method, and a corresponding device, for post-processing image data in order to reduce the perceptual annoyance of blocking artefacts. This object is achieved by means of a method according to claim 1 and a device according to claim 8. More specifically, according to a first aspect, the invention relates to a method of post-processing image data, which image data is obtained by decoding compressed data, encoded in accordance with a block transform scheme, said method including changing picture element values, on opposite sides of a block boundary in the image data, in order to reduce the perceptual annoyance of blocking artefacts in said image data, wherein the luminance value of a first picture element, on one side of the block boundary, is made to adopt the luminance value of a second picture element on the other side of the block boundary. This reduces, in an effective manner, the perceptual annoyance of blocking artefacts in image data, without requiring extensive analysis of the characteristics of the image data. Moreover, the higher spectral energy of the image is retained to a much greater extent compared to "smoothing" techniques, thus resulting in more crisp appearing images. In a preferred embodiment, the luminance values of the first and second picture elements, which preferably are mutually adjacent, are interchanged. This entails a simple method, where the total luminance is maintained. Preferably, the luminance values of picture elements in an image, which picture elements are situated in the vicinity of a boundary, are made to adopt, or are not made to adopt, luminance values of picture elements on the other side of the boundary in accordance with a rule, which corresponds to a pseudo-random distribution. This makes the processing itself virtually invisible to the user. In a preferred embodiment, the rule is chosen, depending on characteristics of said image data, from a set of rules. This allows optimised post-processing to be carried out. In an embodiment, the image data comprise video data, divided into a frame sequence, and at least two different rules are used alternately, in such a way that different rules are applied to two successive frames in the sequence. This makes the processing itself less visible to the user. Alternatively, the image data comprise a still image. According to a second aspect, the invention relates to a device corresponding to the method.

These and other aspects of the invention will be apparent from and will be elucidated with reference to the embodiments described hereinafter.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention will now be described in more detail, by way of example, with reference to the accompanying drawings, wherein: - Fig. 1 illustrates a context in which a method and a device in accordance with an embodiment of the invention may be used,

- Fig 2 illustrates block encoding of an image,

- Fig 3a and 3b illustrate an example where a method according to an embodiment of the invention is carried out, - Figs 4a-4c illustrate the method according to two different embodiments of the invention,

- Fig 5 illustrates, schematically, a device according to an embodiment of the invention,

- Fig 6a and 6b show an image before and after post-processing according to an embodiment of the invention. DETAILED DESCRIPTION OF THE INVENTION The present invention relates in general to a method and a device for post-processing image data. Fig 1 illustrates a context in which a method and a device in accordance with an embodiment of the invention may be used. Original image data, I₀, is encoded in blocks of picture elements (hereinafter called pixels), e.g. using a DCT (Discrete Cosine Transform) unit 1. The DCT unit 1 generates, for each block of pixels, a frequency data block, corresponding to the pixel values in the block. The most important information, with regard to the visual quality of an image, in the frequency data block is extracted first, generating a bit-stream. This extraction of information may, at a desired stage for each block, be stopped or the bit stream may be truncated, by a truncating device 2, thus leaving the least important frequency data behind. This accomplishes a compressed bit stream I_c, comprising less information than the original image data I₀. The compressed image data I_c may be stored in a memory 3 or transmitted, e.g. over a narrowband channel 4. The illustrated scheme may be used for still images, for example encoded in the JPEG format or for video, for example encoded in the MPEG-2 format. When the compressed image data I_c is to be viewed, an JDCT (Inverse Discrete Cosine Transform) unit 5 decodes the compressed data I_c into restored data I_r, which corresponds to the original data I₀, but with a lower image quality, depending on the level of compression. The restored data I_r may be viewed by a user on a display 6. Preferably, the restored data I_r is however processed by a post-processing device 7 in order to compensate for the lower image quality, thus obtaining post-processed data I_p. The present invention relates to such a post-processing device 7 and a method intended to be carried out by such a post-processing device. Fig 2 illustrates block encoding of an image. A part of an image 8 is shown comprising a large number of pixels 9, arranged in an array. When the image 8 is encoded, the array is divided into a number of rectangular blocks 10 of pixels, e.g. comprising 8x8 pixels as illustrated in fig 2. Boundaries 11 between pixel blocks 10 are illustrated with bold lines, forming a grid in fig 2. At high levels of compression, the blocks often become visible, causing blocking artefacts. The present invention serves to reduce the perceptual annoyance of such artefacts by processing pixels close to block boundaries 11, particularly pixels adjacent to boundaries 11, hereinafter called boundary pixels 12a. Except at the outer periphery of the image and at the corners of each block, each boundary pixel 12a will, together with one adjacent pixel on the other side of the boundary, form a boundary pixel pair 13. At block corners where boundaries 11 intersect, a boundary pixel 12a may be part of two such pairs. In accordance with an embodiment of the present invention, the perceptual annoyance of blocking artefacts are in general reduced in image data by letting a first pixel on one side of a boundary 11 adopt the luminance value of a pixel on the other side of the boundary 11.

Preferably, this can be done by interchanging luminance values of two picture elements, on opposite sides of a block boundary. Fig 3 a and 3b illustrate an example where a method according to an embodiment of the invention is carried out. Fig 3 a illustrates for an image luminance values of boundary pixels 12 along a boundary 11. The pixels above the boundary 11 have the values 5, 6, 5, 7,

6, 5, 5, 6, i.e. their average value is about 5.6. The pixels below the boundary 11 have the values 2, 1, 2, 2, 1, 2, 1, 1, i.e. their average value is 1.5. A user watching the displayed image will therefore recognise a distinct step at this boundary, forming part of a blocking artefact. In accordance with an embodiment of the invention, a rule 15 is applied to the image data in fig 3a. This rule may be expressed as a Boolean vector with eight elements Y(=yes), N(=no), N, Y, N, Y, N, Y. Each of these elements express whether the luminance values in the corresponding boundary pixel pair should be interchanged or not. By corresponding is here meant that the rightmost element in the rule 15 is applied to the rightmost boundary pixel pair 16 in the image data in fig 3a, etc. Fig 3b shows an example where the rule 15 has been applied to the image data in fig 3a. As can be seen in fig 3b, the luminance values in the rightmost boundary pixel pair 16 have been interchanged, while this has not occurred in the second rightmost boundary pixel pair, etc., in accordance with the rule 15. The average value of the pixels above the boundary 11 is now about 3.6, while the average value of the pixels below the boundary 11 is 3.5. Thus the "step" at the boundary 11 has now been virtually eliminated. This scheme may be used both for vertical and horizontal boundaries. As compared to methods where the image data is processed by "smoothing", e.g. by applying a low-pass filter to the image data, the invention data retains the higher frequency components to a much greater extent, which results in a crisper, more sharp appearing image. The scheme is further "self-adjusting" in the sense that, if no substantial step exists along a boundary, this boundary will be affected a lot less, since pixels with similar values are interchanged. Therefore in general no complex analysis need to be performed on the image to find blocking artefacts, all block boundaries may be treated the same way. In an embodiment, however, real edges in an image are found and, areas, where those edges coincide with a block boundaries, may be kept unprocessed. In general, a rule which is applied to image data tells for each pixel in the image data, or only for pixels close to or adjacent to a boundary in an image, whether or not their luminance values should adopt the value of another pixel in another block and, if so, of which pixel. The rule should preferably not be repeating or periodical, such as e.g. ynynynynyn along a boundary, since this may in some cases itself be perceptually annoying. Rather, the rule should be a pseudo-random distribution, i.e. it should appear as if the rule was determined randomly, e.g. as illustrated at 15 in fig 3a. If the image data relates to video data, divided into a frame sequence, at least two different rules should preferably be used alternately. This should be done in such a way that different rules are applied to two successive frames in the sequence in order not to make the processing of the image data visible to a user. Of course, more than two different rules can be used. It should be understood that the rule also could be chosen depending on properties of the image content, either in the overall image or in a sub-area of the image. The rule can be determined, e.g. based on how detailed the image is, if there is movement in a video sequence, or, as mentioned earlier, if real edges coincide with block boundaries, etc. The rules may be different both as regards the probability that a given pixel adopts the luminance value of another pixel and the distance between the two pixels. Fig 4a-c illustrate methods according to two different embodiments of the invention.

Fig 4a illustrates the image data prior to post-processing, namely luminance values abcdef of pixels on opposite sides of a block boundary 11. Luminance values c and d relate to boundary pixels, b and e relate to pixels one step away from the boundary 11, and a and f relate to pixels two steps away from the boundary 11. In the first embodiment, illustrated in fig 4b, the luminance values of the boundary pixels c, d are interchanged, resulting in the sequence abdcef corresponding to the example in fig 3b. In the embodiments described here, the pixels are made to adopt values for pixels on the other side of a boundary in a perpendicular direction. It should however be noted that other schemes are feasible, e.g. where a boundary pixel adopts a value from a boundary pixel situated diagonally over the boundary. Fig 4c illustrates another embodiment where pixels at a greater depth, as seen from a boundary, are affected. In this embodiment, the luminance values of pixels situated one step away from the boundary (b, e in fig 4a) are made to adopt the luminance values of the respective boundary pixel on the other side of the boundary 11, resulting in the sequence adcdcf. Of course, many other schemes are possible. Fig 5 illustrates a post-processing device 7 according to an embodiment of the invention. The post-processing device 7 is placed between a decoder 5 and a display 6. The device 7 comprises a grid unit 20 and a processor 21. The grid unit 20 provides information to the processor 21 regarding the position of the boundaries in restored image data I_r which is fed from the decoder 5 to the processor 21. In a case where the boundaries always are situated in the same way in the restored image data L, the grid unit 20 may be left out, since the boundary positions can then be programmed into the processor 21 once and for all. The grid unit 20 may receive boundary position data directly from the decoder 5, or may extract boundary position data from the restored image data L, using known image analysis techniques, as illustrated by the hatched arrow. The processor 21 applies rules to the restored image data I_r in order to obtain post- processed image data I_p. As described above, the rules affect pixels close to block boundaries. In accordance with the rules, pixel luminance values on one side of a boundary adopts values of another pixel on the other side of the boundary, such as by interchanging two pixel values. As indicated earlier, the rule may be chosen based on image characteristics, which is done by means of an optional control unit 22. The control unit may chose the used rule based on information from the decoder 5 and/or from the restored image signal I_r. The grid unit 20, the processor 21, and the control unit 22 are functional modules of the post-processing device 7. These functional modules may be integrated with each other in different ways and may be realised as software or hardware. A real example where a method according to an embodiment of the invention has been used for post-processing an image (300x300 pixels) is illustrated in figs 6a-6b. The post-processing in this embodiment affects pixels up to two steps from a block boundary in accordance with a pseudo-random rule. Fig 6a shows the image before post-processing, while fig 6b shows the image after post-processing. As can easily be seen, blocking artefacts in fig 6b are a lot less annoying. In summary, the invention relates to a method and a device for post-processing image data that has been compressed using a block transform scheme. The luminance values of pixels on one side of a block boundary are made to adopt luminance values of pixels on the other side of the block boundary. Preferably this is done by interchanging the values. This reduces the perceptual annoyance of blocking artefacts in still or video images, while high frequency content in the image data is retained. The invention is not restricted to the described embodiments. It can be altered in different ways within the scope of the appended claims.

Claims

1. A method of post-processing image data, which image data is obtained by decoding compressed data, encoded in accordance with a block transform scheme, said method including changing picture element values, on opposite sides of a block boundary (11) in the image data, in order to reduce the perceptual annoyance of blocking artefacts in said image data, wherein the luminance value of a first picture element (12a), on one side of the block boundary (11), is made to adopt the luminance value of a second picture element (12b) on the other side of the block boundary (11).

2. A method according to claim 1, wherein the luminance values of the first and second picture elements (12a, 12b) are interchanged.

3. A method according to claim 2, wherein the first and second picture elements (12a, 12b) are mutually adjacent picture elements.

4. A method according to claim 1, wherein, in an image, the luminance values of picture elements, which are situated in the vicinity of a boundary are made to adopt, or are not made to adopt, luminance values of picture elements on the other side of the boundary in accordance with a rule (15), which corresponds to a pseudo-random distribution.

5. A method according to claim 4, wherein the rule (15) is chosen, depending on characteristics of said image data, from a set of rules.

6. A method according to claim 4, wherein the image data comprise video data, divided into a frame sequence, and wherein at least two different rules are used alternately, in such a way that different rules are applied to two successive frames in the sequence.

7. A method according to claim 1, wherein the image data comprise a still image.

8. A device (7) for post-processing image data, which image data is obtained by decoding compressed data, encoded in accordance with a block transform scheme, the device comprising means for changing picture element values, on opposite sides of a block boundary in the image data, in order to reduce the perceptual annoyance of blocking artefacts in said image data, wherein the device comprises processing means (21) for making a luminance value of a first picture element (12a), on one side of the block boundary (11), to adopt the luminance value of a second picture element (12b) on the other side of the block boundary

(11).

9. A device according to claim 8, wherein the processing means (21) is arranged to interchange the luminance values of the first and second picture elements (12a, 12b).

10. A device according to claim 9, wherein the first and second picture elements (12a,b) are adjacent picture elements.