WO2006056940A1

WO2006056940A1 - Reduction of compression artifacts in displayed images

Info

Publication number: WO2006056940A1
Application number: PCT/IB2005/053849
Authority: WO
Inventors: Wilhelmus H. A. Bruls; Lincoln Lobo
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2004-11-25
Filing date: 2005-11-21
Publication date: 2006-06-01
Also published as: EP1817913A1

Abstract

Compression artifacts are known to be especially annoying on high end TV's, e.g. because the sharpness enhancement settings increase the visibility of block boundaries. A system arranged for receiving a video data stream for displaying an image, the system comprising an enhancer for enhancement of the displayed image, wherein the system comprises an extractor to extract compression parameters from the video data stream, and comprises an adjustor to adjust, in dependence on the extracted compression parameters, settings of the enhancer.

Description

Reduction of compression artifacts in displayed images

The present invention relates to a system arranged for receiving a video data stream for displaying an image, the system comprising an enhancer for enhancement of the displayed image.

The present invention also relates to a display device comprising a system arranged for receiving a data stream for displaying an image, the system comprising an enhancer for enhancement of the displayed image.

The present invention also relates to a method of processing an image data stream in which method en enhancement algorithm is used.

Video display systems comprise means for enhancement of a picture. Current systems especially high-end TV's perform a lot of signal processing on a received data stream before displaying it. In particular resolution enhancement algorithms are frequently used. Resolution enhancement algorithms enhance, as the words already indicate, the resolution, in practice this means that edges, i.e. areas or lines in the image where there is a transition between one object and another in the displayed image are more sharply displayed. Other types of enhancement system are e.g. 3-D enhancement systems in 3-D systems, meaning that the depth of view is enhanced, i.e. the difference in apparent distance between objects in the image is more sharply defined and displayed. Other type of enhancements such as contrast enhancement are also possible.

Although such enhancement algorithms usually substantially improve the quality of the displayed image, it has been noted by inventors that in some, often encountered circumstances, especially in case of compressed digital video, image processing algorithms aimed at improving an image characteristic, such as in particular resolution, contrast and/or depth of view (in 3-D applications), in fact reduce the quality of the image, leading to a poorer image quality rather than an improved image quality. The invention has as an object to improve on average the image quality. To this end the system in accordance with the invention comprises an extractor to extract compression parameters, for instance from the data stream and/or an encoder, and comprises an adjustor to adjust, in dependence on the extracted compression parameters, settings of the enhancer.

Compression techniques are often used to compress the data stream, i.e. reduce the amount of data within the data stream. In particular, consumer recorder devices (DVD recorders, hard-disk recorders etc.) use digital compression algorithms to provide digitally compressed streams such as MPEG2 streams. Such compression techniques may be lossless techniques, but often, when an appreciable amount of compression is used, some loss of data is deemed acceptable. Typically data compression techniques are arranged such that the loss in data is kept relatively small so that no apparent visible effect of the data compression is seen in the displayed image. Thus it would seem that enhancing the quality of the image based on the resolution enhancement techniques, given the feet that in a non- enhanced image the effects of the data compression are not or hardly visible, would improve the image quality of the image. The inventors have, however, realize that a substantial negative co-operative effect often occurs, i.e. rather than enhancing the image quality, the net effect of image enhancement, in particular of contrast and/or resolution enhancement of an image based on a compressed data stream, is an image in which compression artifacts are clearly visible. The non- visible effects of the compression are made visible by the image enhancement. Similar effects occur when contrast or color is enhanced or 3-D is enhanced, where the compression artifacts can become visible because they can move in depth which catches the viewer's attention.

In the system in accordance with the invention compression parameters are extracted from the incoming data stream.

"Extracting of compression parameters' may take the form of reading a particular part of the video data stream which carries information on compression parameters established beforehand, or which carries information on the system with which the data was compressed, or the algorithms used for compression of the data, since from such data the compression parameters may be deduced for instance by calculation or a look-up table or comparing the incoming signal with internal data of the compressor. In such cases the extractor is arranged to read information related to compression from a part or parts of the data stream designated for such information. Information indicating the compression of the digital stream can be embedded in the data stream and passed from the receiving digital -STB or -recorder to e.g. the display or TV-set, so within the display or TV-set the enhancement settings/parameters can be adapted in order to prevent magnification of compression artifacts due to picture enhancements techniques. When the information indicating the compression of the digital stream is embedded in the data stream 'extracting' thus simply means receiving the data stream and identifying the embedded information, which is then passed on to the adjustor. Alternatively 'extraction of compression parameters' may take the form of extracting from the data stream itself (for instance the number of bits per image, which is a measure of compression) compression parameters.

The compression parameter of the digital stream can e.g. be determined in the encoder by comparing the original input with the local decoded version.

Extraction of compression parameters, may also take the form of a separate signal sent via a connector, which separate signal indicates the compression factor. The extraction of compression parameters may be done once, at the onset of receipt of a data stream, or it may be done, in more sophisticated systems, on a regular basis, i.e. it may change during the data stream, e.g. frame by frame, or even per group of pictures in a frame.

In the system in accordance with the invention image enhancement is performed in dependence on the compression parameters. Thus it can be avoided that for instance a highly compressed data stream is paired to a high resolution enhancement, which combination, inventors have realized, often leads to magnification of disturbing artifacts in the displayed image to a clearly visible and thereby disturbing level. The system in accordance with the invention automatically avoids or at least reduces such artifacts. The method in accordance with the invention is characterized in that a data stream for an image is processed wherein a data stream is compressed, one or more compression parameters are determined and wherein an image enhancement is performed, and that the image enhancement is performed in dependence of the compression parameters and/or vice versa. Known systems do not have means to extract compression parameters from an incoming data stream, nor do they have any means to make the image enhancement dependent on the compression parameters or vice versa.

In an embodiment of the invention the enhancer is a resolution and/ or contrast enhancer and/or brightness enhancers. Resolution and/or contrast and/or brightness enhancement in combination with data compression often leads to a 'block-like' appearance of the image. The data reduction is commonly performed on a block-by block basis. In reducing the amount of data a small loss in image quality often occurs. In particular high order image information is lost. Such loss introduces small, to the naked eye not or hardly perceivable, changes in image intensity or another image parameter (e.g. color point, depth of view for 3-D images) near or at the edges of the blocks. The enhancer enhances such differences, so that the edge between the blocks becomes slightly more visible. Looking at each image block separately this enhancement is often hardly visible. However, the human eye is very perceptive for small recurrent changes in the image, and the edges of the blocks become visible or at least have a disturbing effect.

Within the concept of the invention an 'enhancer', 'compressor', 'extractor', etc are to be broadly understood and to comprise e.g. any piece of hard- ware (such as a extractor, enhancer, etc), any circuit or sub-circuit designed for performing an image enhancement, data compression, compression parameters extraction iunction as described as well as any piece of soft-ware (computer program or sub program or set of computer programs, or program code(s) or algorithms) designed or programmed to perform an enhancement, compressing, compression parameter extraction etc operation in accordance with the invention as well as any combination of pieces of hardware and software acting as such, alone or in combination, without being restricted to the below given exemplary embodiments. One program or device or circuit or algorithm may combine several functions, or distinct functions may reside in distinct hardware or software.

The invention is also embodied in any computer program comprising program code means for performing a method in accordance with the invention when said program is run on a computer as well as in any computer program product comprising program code means stored on a computer readable medium for performing a method in accordance with the invention when said program is run on a computer, as well as any program product comprising program code means for use in a system or device in accordance with the invention, for performing the action or actions or combinations of actions specific for the invention.

The above and other objects, features and advantages of the present invention will become more apparent from the following detailed description when taken in conjunction with the accompanying drawings in which: Fig.l schematically illustrates an image error after compression of data and after compression and enhancement;

Fig. 2 schematically illustrates a iurther image error after compression of data and after compression and enhancement; Fig. 3 schematically illustrates a device in accordance with an aspect of the invention

Fig. 4 schematically illustrates a device in accordance with a further aspect of the invention.

The present invention will now be described more fully hereinafter with reference to the accompanying drawings. This invention may, however, be embodied in many different forms and should not be construed as limited to the embodiment set forth herein; rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art. Like numbers refer to like elements throughout.

Preferred embodiments of the present invention will be described herein below with reference to the accompanying drawings. In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those skilled in the art that various changes in form and detail may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. The first part of Fig. 1 illustrates a picture or image as it would be displayed when the full data stream would be used. However, often data is compressed, for instance using digitally compressed (MPEG2) streams. Some compression techniques are loss less, however, when a large compression factor is used, some loss of data may be inevitable. Typically, data compression techniques are arranged such that the loss in data is kept relatively small so that no apparent visible effect of the data compression is seen in the displayed image. On close inspection a small effect may be visible. This is illustrated in the second part of Fig. 1 in which compression (comp) provides an image in which, upon close inspection a faint block structure may be visible. However, this structure is not or hardly visible at normal viewing distances by an average viewer. The error is below a visibility threshold.

The inventors have, however, realized that image enhancement techniques such as used in high-end TV/displays (Pixel+) cause picture quality problems with many digital compressed (MPEG2) streams. This is illustrated in the last part of Fig. 1, in which after compression and enhancement (comp+enh) a block structure is clearly visible. This is the more annoying, since this effect is present in high-end TV displays, which the customer has bought in order to have a high quality picture.

Fig. 2 illustrates a further image error, namely a halo near an edge (the right hand edge) also these errors are increased after compression and subsequent enhancement techniques.

Image enhancement techniques may for instance be resolution, contrast or color enhancement techniques.

Data compression techniques typically reduce the data stream by: - a more clever way of arranging the data, for instance by replacing a string of zeros or ones by data indicating the length of the string and the value. expressing the data in a different parameter space, by means of a type of Fourier transformation, and reducing the data by deleting those data that for which the contribution is only minor and only transmitting the 'important" data. However, if only the most important data are transmitted or stored, per definition some data has been lost. The effect of the loss of these data is so small that at the usual viewing distance the average viewer is not aware of a loss in image quality.

However, often image enhancement techniques are used to sharpen the image.

It has been found that such enhancement of the image also magnifies the small effects of the compression to such an extent that they become visible in the image.

In this respect it is remarked that compression of image data is typically done on relatively large blocks of pixels, of for instance 8x8 or 16x16 pixels. Such blocks are large enough to be visible by a viewer. The compression typically introduces a small image error in particular near the edges of the blocks. The enhancement techniques enhance this error, so that a block structure becomes visible in the displayed image. The net effect of compression and enhancement, is a disturbing block pattern in the image, and furthermore edges of object, instead of being more sharply displayed become serrated.

Similar problems are encountered when a 3D-image is displayed. When a 3D image is displayed a further image aspect comes into play, namely the depth of view, the z- position of an object. Each object besides occupying a position or area within the frame of the image, also has a z-position, indicating the location of object or part of the image in respect of other objects or parts of the image. Compression combined with image enhancement, may negatively effect the perceived impression, because the compression artifacts are varying in distance. In these circumstances the adjuster (9) may reduce the depth of the image in order to reduce this effect.

Fig. 3 illustrates a device and method in accordance with the invention. A digital stream 1 is encoded by encoder 2. The encoder 2 can be the professional encoder at the broadcast side or the internal consumer encoder of a digital (DVD or HDD) video recorder. The encoded data is either transmitted (T) or stored on a storage medium (e.g. disk 3). The data is then demultiplexed in de multiplexer 4 , decoded in decoder 6, and via DAC 7 and connection C (e.g. a Scart connector) in analogue format send to TV 11, which comprises an image enhancer 8. Elements 2 to 6 may be part of a DVD or HDD video recorder 10. The additional element within the framework of the invention are on the one hand, within the recorder part an establisher 5 for establishing compression parameters. 'Establishing' within the framework of the invention is any algorithm or method which enables to determine or deduce compression parameters. E.G. via a P50 Scart communication the recorder sends the mode of the recording. Eg. HQ is a high bitrate (~8Mbs) mode where hardly any visible compression artiiacts are produced. LP is a low bitrate (~3Mbs) mode and contains to a certain degree compression artifacts. Alternatively the compression parameters may be established by comparing internal data of the compressor 2 to the original data stream. The central idea is that the information on the compression is used to adjust the settings of the enhancer. A person skilled in the art is well aware of the various techniques of deriving such information and transferring such information. As explained, such information may be obtained and/or established at various stages, and may be transferred to the adjustor at various means.

In the embodiment shown in Fig. 3 the compression data established in establisher 5 are sent via the connector to the TV. The TV comprises an extractor/adjustor 9, which extracts the compression data from the data stream, and adjust the settings of the image enhancer 8. In this example the two functions are combined in a single stage, they could be distinct stages of operation. As explained above, often the encoding artiiacts (ringing, blocking, mosquito-noise, etc.) present at the output of the recording devices 10 are strongly amplified by the TV-set when an enhancer 8 is used. The present invention overcomes or at least reduces this problem. The compression data is now known via extractor/adjustor 9, and the settings of the enhancer may be tuned, resulting in a better image quality.

In this embodiment the recorder establishes data on compression and, via the connector C data on the compression is provided to the TV-set. The TV-set in this preferred embodiment need only comprise a simple means to read the pre-established data. In other embodiments the TV set may comprise more sophisticated means to extract compression data from the incoming signal. The connection C between the recorder and the TV set may be a cable or a wireless connection.

Fig. 4 illustrates another aspect of the invention. In this aspect the TV set comprises a means for providing data on the enhancer, i.e. the settings of the enhancer. These enhancement data is transferred to the recorder, which uses these data to tune the compression. This embodiment is in a sense a mirror image of the embodiment of Fig. 3. The invention in its broadest sense is to tune the enhancement and the compression. In most circumstances, a compressed data stream is received by the TV set, and in such circumstances the settings of enhancer are tuned to the data compression. However, it is also possible to work in an opposite direction, when the enhancement settings are known, the recorder could follow the enhancement settings. Of course, it is also possible that a two-way communication is used, wherein the recorder and the TV set, exchanges data, so that an optimum tuning of compression and enhancement is achieved. Thus the invention also relates to a combination of a recording device, wherein in operation a data stream is compressed, and a display device, for providing an image on the display, wherein in operation the displayed image is enhanced, wherein the combination comprises a connector for providing a compressed data stream from the recorder to the display device, wherein the combination comprises means for tuning the compression factor to the enhancement factor and/or vice versa.

In the embodiment of Fig. 3 the compression data is distinguishable data embedded in the stream, and thus the extracting simply means reading the relevant part of the incoming compressed data stream. The information can be obtained in the encoder in many ways or can even be implicit in the bitstream itself (format and bitrate). A effective way determining the compression of a digital stream is by comparing in the encoder the local decoder of the encoder with the original input and then e.g. calculate a metric like PSNR (MSE, mean square error, in dB) or BIM (measure for level of blockiness) or a given quality metric X. Establishing the compression information in the encoder has the advantage that the original input is still available so there are more possibilities for computing a quality metric. Using the received mode indication, the TV can now influence the 'smart pictures' settings, Depending on the received compression information the picture enhancement can be set gradually from complete off to maximum enhancement strength, it doesn't have to be a ON/OFF decision. The following embodiment offers a simple solution for embedding the compression data in the data stream:

An almost universally supported signaling channel is the Vertical Blanking Interval (VBI) data, which is embedded in the analogue video signal. In Europe Wide-Screen Signaling data is present on line 23, and carries 14 bits of information. It is used to signal aspect ratio, copyright status, sound mode etc. From these 14 bits, currently 13 bits are used. Bit 7 is not used. A simple embodiment of the present invention is to use bit 7 of line 23 of the Vertical Blanking Interval for carrying information about compression.

It will be appreciated by persons skilled in the art that the present invention is not limited by what has been particularly shown and described hereinabove. The invention resides in each and every novel characteristic feature and each and every combination of characteristic features. Reference numerals in the claims do not limit their protective scope. Use of the verb "to comprise" and its conjugations does not exclude the presence of elements other than those stated in the claims. Use of the article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The present invention has been described in terms of specific embodiments, which are illustrative of the invention and not to be construed as limiting. The invention may be implemented in hardware, firmware or software, or in a combination of them. Other embodiments are within the scope of the following claims.

Claims

CLAIMS:

1. A system (11) arranged for receiving a video data stream for displaying an image, the system comprising an enhancer (8) for enhancement of the displayed image, wherein the system comprises an extractor (9) to extract compression parameters from the video data stream, and comprises an adjustor (9) to adjust, in dependence on the extracted compression parameters, settings of the enhancer (8).

2. A system as claimed in claim 1, wherein the extractor is arranged to read information related to compression from a part or parts of the data stream designated for such information.

3. A system as claimed in claim 2, wherein the extractor (9) is arranged to derive compression parameters from an establisher (5), which inserts extra data comprising compression parameters along with the data stream.

4. A system as claimed in claim 1, wherein the enhancer (8) is a resolution enhancer.

5. A system as claimed in claim 1, wherein the enhancer is a contrast or brightness enhancer.

6. A system as claimed in claim 1, wherein the data stream is a digitally compressed data stream.

7. A system as claimed in claim 1, wherein the data stream is an analog video signal.

8. A system as claimed in claim 2 and 7, wherein in operation the extra data is sent via the vertical blanking interval of the analog video signal.

9. Display device comprising a system as claimed in claim 1.

10. Display device as claimed in claim 9, wherein the display device is for displaying 3-D images.

11. Display device as claimed in claim 8, wherein the adjustor (9) is arranged to control the depth of the 3-D image in dependence on the compression parameters.

12. A combination of a recording device (10), wherein in operation a data stream is compressed, and a display device (11), for providing an image on the display, wherein in operation the displayed image is enhanced in an enhancer (8), wherein the combination comprises a connector (C) for providing a compressed data stream from the recorder (10) to the display device (11), wherein the combination comprises means for tuning the enhancement factor to compression parameters and/or vice versa.

13. A method in which a data stream for an image is processed wherein a data stream is compressed, one or more compression parameters are determined and wherein an image enhancement after compression is performed, wherein the image enhancement is performed in dependence of the compression parameters or vice versa.