WO2007020572A1

WO2007020572A1 - Image processor comprising a block artifacts detector

Info

Publication number: WO2007020572A1
Application number: PCT/IB2006/052763
Authority: WO
Inventors: Remco Muijs
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2005-08-18
Filing date: 2006-08-10
Publication date: 2007-02-22

Abstract

Image processor comprising a block artifacts detector. An image processor comprises a block artifacts detector (BDT). The block artifacts detector (BDT) establishes (HGC, HGN) a normalized gradient map (HNGM) that indicates locations within an image where a difference between adjacent image elements along an axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations. Subsequently, the block artifacts detector (BDT) establishes (VAC, HHB, VGD) a grid (VG) that matches the aforementioned locations that the normalized gradient map (HNGM) indicates.

Description

Image processor comprising a block artifacts detector.

FIELD OF THE INVENTION

An aspect of the invention relates to an image processor that comprises a block artifacts detector. The image processor may be, for example, a video display driver that causes a display device to display a video input signal, which the display driver receives. Other aspects of the invention relate to a method of image processing, a computer program product for an image processor, and an image display system.

DESCRIPTION OF PRIOR ART

The international patent application published under number WO2005/015915 describes a block artifacts detection device that detects block artifacts in a video signal. The block artifacts detection device computes a gradient signal on the basis of the video signal. Subsequently, the block artifacts detection device establishes a list of samples corresponding to respective local maximum values of the gradient signal. A histogram of inter-sample distances is built and analyzed so as to produce a block artifacts indicator.

SUMMARY OF THE INVENTION

According to an aspect of the invention, an image processor comprises a block artifacts detector with a normalized gradient calculator and a grid calculator. The normalized gradient calculator establishes a normalized gradient map that indicates locations within an image where a difference between adjacent image elements along an axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations. The grid calculator establishes a grid that matches the locations within the image where a difference between adjacent image elements along the axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations.

The invention takes the following aspects into consideration. More and more images, which are displayed on display devices, result from an original image that has been encoded and subsequently decoded in accordance with an MPEG coding standard (MPEG is an acronym for Moving Pictures Expert Group). An MPEG encoder divides an image into blocks of pixels and codes the image on a block-by-block basis. MPEG coding causes a loss of information, which may result in discontinuities between adjacent blocks of pixels. This may give an image a "blocky" appearance. Such block artifacts are particularly annoying and, therefore, adversely affect subjective image quality to relatively great extent. What makes matters worse is that many video display sets comprise some form of sharpness, contrast, and brightness enhancement, which tends to increase the visibility of block artifacts.

Block artifacts can be attenuated by means of a smoothing function, which transforms a sharp transition into a smooth transition. For example, a spatial low-pass filter provides a smoothing function. Applying a smoothing function to an entire image will result in a loss of resolution. The smoothing function is therefore preferably locally applied, namely at borders between adjacent blocks. This requires knowledge of where block borders are located within an image. Such knowledge may not be readily available and, therefore, may need to be extracted from the image itself.

The aforementioned prior art describes a technique that allows identification of discontinuities at block borders. In accordance with this technique, a horizontal gradient is calculated for each pixel in an image. The horizontal gradient, which reflects a difference with a horizontally adjacent pixel, is compared with a first predetermined threshold θ. The horizontal gradient is considered as relevant if the horizontal gradient is above the first predetermined threshold value θ. A count is made of relevant horizontal gradients that have the same horizontal position. The count is made for each different horizontal position in the image. Accordingly, an array of relevant horizontal gradient counts is obtained. A second predetermined threshold value α is applied to determine the so-called volume of a peak in the array.

The aforementioned prior art has a reliability problem. Discontinuities at block borders are correctly identified only if both the first and second predetermined threshold values have appropriately been set. An appropriate setting for an image depends on the content of the image, in particular on whether the image comprises relatively many sharp details or not. Different images will therefore have different appropriate settings. The first predetermined threshold value and the second predetermined threshold value may allow correct identification of discontinuities at block borders in an image "A", whereas the same values fail to correctly identify such discontinuities in an image "B". Consequently, there is a risk that block artifacts are incorrectly identified. If so, any smoothing function will be applied at the wrong location. This will adversely affect image quality.

In accordance with the aforementioned aspect of the invention, a block artifacts detector establishes a grid on the basis of a normalized gradient map. The normalized gradient map indicates locations within an image where a difference between adjacent image elements along an axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations. A grid is established that matches the aforementioned locations where differences are significant. The grid represents the block borders in the image.

In contradistinction with the prior art, the invention does not require any predetermined threshold values that need to be precisely set. This is because block artifacts detection is based on a normalized gradient map, which indicates locations within an image where a difference between adjacent image elements along an axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations. Such block artifacts detection is inherently less sensitive to edges of objects that form part of the image compared with block artifacts detection based on comparing gradients with one or more thresholds. Moreover, block artifacts detection in accordance with the invention is inherently less sensitive to any upscaling or downscaling that the image may have undergone. Accordingly, the invention allows a more precise and more reliable block artifacts detection, which allows more precise image processing. The grid that is extracted in accordance with the invention reliably indicates block borders with relatively great precision. The grid can be used, for example, to control an image processor so that the image processor applies a smoothing function at block borders. This will attenuate block artifacts more effectively and, moreover, with less risk of error in the sense that a smoothing function is applied where it should not. As another example, the grid can be used to calculate a reliable block artifacts metric, which indicates the degree of visibility of block artifacts. Block artifact metric calculation generally involves making a distinction between pixels that belong to block borders and other pixels, which do not belong to block borders. The grid allows a reliable and precise distinction and, therefore, a reliable and precise block artifact metric. The block artifact metric can be used to adapt an image processing function that enhances sharpness, contrast, brightness, or other image parameters. The block artifact metric can avoid that the image processing function amplifies the block artifacts to an extent that these become disturbing. For example, sharpness enhancement may be reduced if the block artifact metric indicates that an image, whose sharpness is to be enhanced, comprises block artifacts with a relatively high degree of visibility. For those reasons, the invention allows improved image quality.

Another advantage of the invention relates to the following aspects. It has been explained hereinbefore that the prior art technique requires a precise setting of threshold values. Finding the precise setting, which allows correct block border detection for various different images, will generally be time-consuming and therefore costly. An adaptive setting could be devised, which analyzes the image concerned and adjusts threshold values on the basis of this analysis. Such an adaptive setting could improve reliability of block border detection in accordance with the prior art. However, such an improvement would make the prior art technique even more complex and therefore more costly. The invention does not require a precise threshold setting as explained hereinbefore. What is more, block border detection in accordance with the invention inherently has a high degree of reliability, which is due to the fact that this detection is based on normalized gradients. Consequently, there is no need for any special reliability-enhancing measures, which would drive up costs. For those reasons, the invention allows cost-efficient implementations.

Yet another advantage of the invention relates to the following aspects. As explained hereinbefore, it is possible to attenuate block artifacts by means of a smoothing function. This will inevitably introduce some loss of resolution. The smoothing function is preferably adapted in dependence on the degree of visibility of the block artifacts. The less visible the block artifacts are, the weaker the smoothing function should be. The normalized gradient map, which is established in accordance with the invention, allows a block artifact metric, which indicates the degree of visibility of block artifacts, to be established in a relatively simple manner. Consequently, the invention allows effective block artifact attenuation, while preserving image resolution, in a cost-efficient manner. These and other aspects of the invention will be described in greater detail hereinafter with reference to drawings.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram that illustrates a video display set. FIGS. 2A, 2B, and 2C are pictorial diagrams each of which illustrates an image, which the video display set receives, and a coding block grid associated thereto.

FIG. 3 is a functional diagram that illustrates a video processor, which forms part of the video display set.

FIG. 4 is a functional diagram that illustrates a block artifacts detector, which forms part of the video processor.

FIG. 5 is a graph that illustrates an array of accumulated normalized horizontal gradients, which the block artifacts detector establishes.

FIG. 6 is a graph that illustrates a histogram of horizontal interpeak distances, which the block artifacts detector establishes. FIG. 7 is a graph that illustrates a histogram of horizontal peak offsets, which the block artifacts detector establishes.

FIG. 8 is a pictorial diagram that illustrates a vertical coding block grid, which the block artifacts detector establishes. FIG. 9 is a functional diagram that illustrates an alternative video processor, which may replace the video processor in the video display set.

FIG. 10 is a block diagram that illustrates an implementation of the video processor in the form of a suitably programmed processor.

DETAILED DESCRIPTION

FIG. 1 illustrates a video display set VDS. The video display set VDS comprises a video display driver VDD, a display device DPL, and a remote control device RCD. The video display driver VDD comprises an input circuit INP, a video processor VPR, an output circuit OUT, and a controller CTRL. The video display driver VDD receives various input video signals VA, VB, VC from various video sources, which are not shown. The display device DPL may be, for example, a flat panel display of the liquid crystal type.

The video display driver VDD basically operates as follows. Let it be assumed that a user selects a particular video source on his or her remote control device RCD. The remote control device RCD sends a command to the controller CTRL that indicates the particular video source to be selected. In response, the controller CTRL causes the input circuit INP to select that particular video source. The input circuit INP applies an internal video signal VI to the video processor VPR. The internal video signal VI corresponds with the input video signal from the video source that the user has selected.

The video processor VPR processes the internal video signal VI so as to enhance various display characteristics, such as, for example, sharpness, brightness, and contrast. The user may adjust one or more display characteristics by means of his or her remote control device RCD. The video processor VPR applies an enhanced internal video signal VIE to the output circuit OUT. The output circuit OUT provides a display driver signal DDS in response to the enhanced internal video signal VIE. To that end, the output circuit OUT may carry out various signal processing operations, such as, for example, amplification, level shifting, bias voltage generation, and synchronization. The display device DPL, which receives the display driver signal DDS, displays the input video signal that the user has selected. FIGS. 2A, 2B and 2C illustrate respective images that form part of input video signals VA, VB and VC, respectively. Each image can be regarded as a matrix of MxN pixels having M columns and N rows, M and N being integer numbers. Each cell of the matrix, which has a column number "i" and a row number "j" corresponds with a pixel I(i,j). Indices "i", "j" thus denote a horizontal and a vertical location, respectively, of the pixel I(i,j) within the image.

The image, which FIG. 2A illustrates, comprises blocks of pixels. This is because the image is a decoded version of an original image that has been encoded in accordance with an MPEG video coding standard. MPEG is an acronym for "Moving Pictures Expert Group". An MPEG encoder typically divides an image into blocks of pixels and encodes the image on a block-by-block basis. MPEG coding introduces a certain loss of information, which may cause a discontinuity at a border between two adjacent blocks. As a result, a block division may be visible in the decoded version of the original image. The image will have a "blocky" appearance. Such block artifacts are particularly annoying. FIG. 2A illustrates a coding block grid within the image. The MPEG encoder has divided the original image in accordance with the coding block grid for a block-by-block encoding. The coding block grid indicates all borders between all adjacent blocks of pixels. Consequently, the coding block grid indicates portions of the image where relatively strong discontinuities may potentially occur. The image, which FIG. 2B illustrates, results from an original image that has been transmitted in the following fashion. A source entity has downscaled the original image by a factor two in both dimensions, horizontally and vertically, so as to obtain a downscaled version of the original image. Such downscaling can be achieved by sub-sampling the original image. The source entity has subsequently encoded the downscaled version of the original image so as to obtain an encoded downscaled version of the original image. An intermediate entity has decoded the encoded downscaled version of the original image so as to obtain a decoded downscaled version of the downscaled image. The intermediate entity has subsequently upscaled the decoded downscaled version of the original image so as to obtain a decoded copy of the original image of similar size. The intermediate entity has transmitted this copy of the original image to the video display set, which FIG. 1 illustrates.

FIG. 2B illustrates a coding block grid within the image. The source entity has divided the downscaled version of the original image in accordance with the coding block grid for a block-by-block encoding. The coding block grid defines blocks that are two times bigger than the blocks that FIG. 2A illustrates. This is due to the aforementioned upscaling that the intermediate entity has carried out.

The image, which FIG. 2C illustrates, results from an original image that has been transmitted in the following fashion. A source entity has encoded the original image in accordance with an MPEG video coding standard so as to obtain an encoded version of the original image. The source entity has transmitted the encoded version to an intermediate entity. The intermediate entity has decoded the encoded version so as to obtain a decoded version of the original image. The decoded version may be similar to the image that FIG. 2A illustrates. The intermediate entity has selected a portion of the decoded version of the original image, which corresponds to cutting out a portion of the image that FIG. 2 A illustrates. Accordingly, a decoded and cropped version of the original image is obtained. The intermediate entity has upscaled the decoded and cropped version so as to obtain an MxN pixel representation of that version. Accordingly, a decoded, cropped, and upscaled version of the original image is obtained, which corresponds to the image that FIG. 2C illustrates.

The intermediate entity has transmitted this version of the original image to the video display set, which FIG. 1 illustrates.

FIG. 2C illustrates a coding block grid in accordance with which the source entity has divided and encoded the original image. The coding block grid is wider than the coding block grid that FIG. 2A illustrates. This is due to the aforementioned upscaling that the intermediate entity has carried out. Moreover, the coding block grid is not aligned with the image so that the image comprises partial blocks of pixels. This is due to the aforementioned cropping of the decoded version of original image that the intermediate entity has carried out. FIGS. 2A, 2B and 2C illustrate that the video display set VDS receives images, which, in principle, may have any arbitrary coding block grid. The coding block grid of an image received depends on coding parameters, as well as parameters of other video processing steps, if any, which have been applied in a transmission of an original image to the video display set VDS. The aforementioned parameters are not necessarily known at a receiving end of which the video display set VDS forms part. The video display set VDS may have no a-priori knowledge of any block artifacts and, in particular, no knowledge of block artifact parameters, such as, the coding block grid and the degree of visibility of the block artifacts. FIG. 3 illustrates the video processor VPR. The video processor VPR comprises various functional entities: a block artifacts detector BDT, a block artifacts attenuator BAA, a video enhancer ENH, and three buffer memories BUFl, BUF2, BUF3. Any of these functional entities may be implemented by means of software or hardware, or a combination of software and hardware. For example, a suitably programmed processor may carry out operations that will be described hereinafter with reference to the video processor VPR. As another example, each of the aforementioned functional entities may be implemented in the form of a dedicated circuit.

The video processor VPR basically operates as follows. It is assumed that the internal video signal comprises the image that FIG. 2C illustrates. Buffer memory BUFl temporarily stores the image. The block artifacts detector BDT extracts coding grid parameters GRD from the image. The coding grid parameters GRD characterize the coding block grid of the image, which FIG. 2C illustrates. In addition, the block artifacts detector BDT establishes a block artifact metric BM, which indicates the degree of visibility of block artifacts, if any, in the image. The block artifacts detector BDT will be described in greater detail hereinafter.

The block artifacts attenuator BAA processes the image on the basis of the coding grid parameters GRD and the block artifact metric BM. The block artifacts attenuator BAA applies a smoothing function to pixels that form part of the coding block grid and, if necessary, pixels close thereto. The coding grid parameters GRD, which the block artifacts detector BDT provides, tell the block artifacts attenuator BAA, as it were, where to apply the smoothing function. This allows the block artifacts attenuator BAA to effectively attenuate blocking artifacts while preserving image sharpness.

What is more, the block artifacts attenuator BAA adjusts the smoothing function in dependence on the block artifact metric BM, which the block artifacts detector BDT provides. The block artifacts attenuator BAA will locally smooth the image at the coding block grid to a relatively great extent if the block artifact metric BM indicates that there are relatively strong block artifacts. Conversely, the block artifacts attenuator BAA will locally smooth the image to relatively small extent if the block artifact metric BM indicates that there are relatively weak block artifacts. This further contributes to an effective attenuation of blocking artifacts, if any, while preserving image sharpness.

The block artifacts attenuator BAA may comprise, for example, a controllable low pass filter, which establishes a low-pass filtered pixel for a pixel of interest. The low- pass filtered pixel is a weighted average of pixels within a filter window that comprises the pixel of interest and neighboring pixels. Respective weighting coefficients are assigned to respective pixels. Accordingly, the controllable low pass filter provides the aforementioned smoothing function at a location that corresponds with the pixel of interest. The filter window or the weighing coefficients, or both, may be adjusted in dependence on the block artifact metric BM.

The block artifacts attenuator BAA provides an artifact-attenuated internal video signal VID, which is temporarily stored in buffer memory BUF2. The video enhancer ENH processes the artifact-attenuated internal video signal VID so as to subjectively improve image quality. To that end, the video enhancer ENH may enhance various visual parameters such as, for example, sharpness, contrast, and brightness. Such processing generally amplifies block artifacts. There is a risk that this block artifacts amplification outweighs a subjective quality improvement in terms of sharpness, contrast, and brightness. Stated in popular terms, there is a risk that the medicine is worse than the illness. The block artifacts detector BDT, which appropriately controls the block artifacts attenuator BAA, reduces this risk and, therefore, allows the video enhancer ENH to provide the enhanced internal video signal VIE, which has an improved quality.

FIG. 4 illustrates the block artifacts detector BDT. The block artifacts detector BDT comprises various functional entities that constitute a vertical grid detection path VGDP: a horizontal gradient calculator HGC, a horizontal gradient normalizer HGN, a vertical accumulator VAC, a horizontal histogram builder HHB, and a vertical grid detector VGD. The block artifacts detector BDT comprises various functional entities that constitute a horizontal grid detection path HGDP: a vertical gradient calculator VGC, a vertical gradient normalizer VGN, a vertical gradient normalizer HAC, a vertical histogram builder VHB, and a horizontal grid detector HGD. The block artifacts detector BDT further comprises a block artifact metric calculator BMC and a grid composer GCO. The block artifacts detector BDT may comprise buffer memories for temporarily storing data, similar to the three buffer memories BUFl, BUF2, BUF3 in the video processor VPR. FIG. 4 does not show such buffer memories for reasons of conciseness.

Any of the aforementioned functional entities may be implemented by means of software or hardware, or a combination of software and hardware. For example, a suitably programmed processor may carry out operations that will be described hereinafter with reference to the block artifacts detector BDT. As another example, each of the aforementioned functional entities may be implemented in the form of a dedicated circuit. The block artifacts detector BDT, which FIG. 4 illustrates, operates as follows. The horizontal gradient calculator HGC calculates a horizontal gradient for substantially each pixel in the image. The horizontal gradient of a pixel is the absolute difference between the pixel and the closest right adjacent pixel. The horizontal gradient may mathematically be expressed as:

D_H{ij) = \l{i + lj)- l{ij)

Alternatively, the horizontal gradient may be the absolute difference between the pixel and the one-but-closest right adjacent pixel. Such a double step size may be advantageous for detection of block artifacts in images that have been upscaled by factors comprised in a range between 1 and 2.

The horizontal gradient calculator HGC provides a matrix of horizontal gradients HGM, wherein each cell corresponds with a particular pixel and defines the horizontal gradient for that pixel. The horizontal gradient of a pixel indicates a difference between the pixel and the closest right adjacent pixel. There is a strong discontinuity if the difference is large. An edge of an object that forms part of the image may cause a large difference. Blocking artifacts may also cause a large difference.

The horizontal gradient normalizer HGN establishes a normalized horizontal gradient for substantially each pixel in the image. The normalized horizontal gradient of a pixel is equal to the horizontal gradient of the pixel divided by an average of respective horizontal gradients of respective neighboring pixels. The normalized horizontal gradient may mathematically be expressed as:

wherein DH, _av(i,j ) denotes the average of respective horizontal gradients of respective neighboring pixels. This average may mathematically be expressed as:

wherein K is an integer number that denotes the number of neighboring pixels, which are left adjacent or right adjacent, that are taken into account in establishing the average. For example, let it be assumed that K=I. In that case, the horizontal gradient normalizer HGN establishes the average on the basis of the horizontal gradients of the closest left adjacent pixel and the closest right adjacent pixel only.

The horizontal gradient normalizer HGN provides a matrix of normalized horizontal gradients HNGM, wherein each cell corresponds with a particular pixel and defines the normalized horizontal gradient for that pixel. The normalized horizontal gradient of a pixel indicates whether the horizontal gradient of that pixel differs to relatively great extent from respective horizontal gradients of respective neighboring pixel. That is, the normalized horizontal gradient indicates whether the horizontal gradient locally stands out, as it were.

The normalized horizontal gradient takes into account, as it were, the degree of visibility of a discontinuity, which the horizontal gradient represents. Let it be assumed that the horizontal gradient of a pixel is relatively high, whereas neighboring pixels have horizontal gradients that are relatively low. In that case, the pixel represents a discontinuity with a high degree of visibility because there are no discontinuities of comparable strength in the immediate vicinity. The discontinuity stands out.

Conversely, let it now be assumed that the horizontal gradient of the pixel is relatively high, and that neighboring pixels equally have horizontal gradients that are relatively high. In that case, the pixel represents a discontinuity with a low degree of visibility because there are other discontinuities of comparable strength in the immediate vicinity. The discontinuity does not stand out because the other discontinuities, which are relatively close, mask the discontinuity of interest.

The aforementioned examples illustrate that the normalized horizontal gradient of a pixel indicates whether the pixel represents a discontinuity with a high degree of visibility or not. A discontinuity that is a block artifact may occur in a portion of the image that comprises many other discontinuities, which represents one or more objects that form part of the image. In such a case, the block artifact will have a low degree of visibility. There may be another discontinuity that is a block artifact too, but which is smaller and which occurs in another portion of the image that comprises relatively few discontinuities or even none at all. The last mentioned block artifact will have a higher degree of visibility although, in an absolute sense, this block artifact is weaker than the first mentioned block artifact. The normalized horizontal gradient takes this into account.

The vertical accumulator VAC establishes an array of accumulated normalized horizontal gradients HAA on the basis of the matrix of normalized horizontal gradients HNGM. An accumulated normalized horizontal gradient is the sum of normalized horizontal gradients in a particular column of the matrix of normalized horizontal gradients HNGM. The accumulated normalized horizontal gradient may mathematically be expressed as:

J=I N Wherein S_H(O denotes the accumulated normalized horizontal gradient for column "z" in the matrix. Each column comprises N cells as FIG. 2C illustrates.

FIG. 5 illustrates an example of the array of accumulated normalized horizontal gradients HAA in the form of a graph with a horizontal axis and a vertical axis. The horizontal axis represents the column number "i". The highest column number corresponds with the number of pixels on a line in the image that FIG. 2C illustrates, which number is M. The vertical axis represents the sum S_H of normalized horizontal gradients. The array of accumulated normalized horizontal gradients HAA comprises peaks at regular intervals. Discontinuities that are due to block artifacts cause these peaks. More specifically, a peak corresponds with a vertical border between two adjacent blocks of pixels. The regular intervals at which the peaks occur corresponds with the pitch of the coding block grid. Any of the peaks may have a distance from the origin, which is not necessarily an integer number of the regular intervals. There may be a so-called peak offset. The peak offset can also be derived from the array of accumulated normalized horizontal gradients HAA, which FIG. 5 illustrates.

The horizontal histogram builder HHB builds two histograms on the basis of the array of accumulated normalized horizontal gradients HAA: a histogram of horizontal interpeak distances and histogram of horizontal peak offsets HHO. To that end, the horizontal histogram builder HHB may carry out the following operations. In a first step, the horizontal histogram builder HHB identifies all the respective peaks that the array of accumulated normalized horizontal gradients HAA comprises. An accumulated normalized horizontal gradient that constitutes a peak has the following characteristic: the closest left adjacent neighbor and the closest right adjacent neighbor are smaller. This may mathematically be expressed as: S_H (i)- S_H {i -l) > 0 & S_H {i + i)-S_H (i) < 0

In a second step, the horizontal histogram builder HHB removes small local maxima that may adversely affect interpeak distance analysis. To that end, the horizontal histogram builder HHB only retains peaks which satisfy the following condition: the accumulated normalized horizontal gradient that constitutes the peak should be larger than any of a set of X closest left adjacent accumulated normalized horizontal gradients and larger than any of a set of X closest right adjacent accumulated normalized horizontal gradients, whereby X is an integer that is preferably comprised in a range between 2 and 7. The aforementioned condition can mathematically be expressed as: S_H (i) > S_H (i + x\ x≡ [-X,X], x ≠ 0 The first and second step can be regarded as a processing that simplifies the array of accumulated normalized horizontal gradients HAA, which FIG. 5 illustrates. An accumulated normalized horizontal gradient that does not constitute a peak, or that constitutes an insignificant peak, is effectively set to zero (0). Accordingly, a simplified array is obtained, which comprises accumulated normalized horizontal gradients that constitute relevant peaks only.

In a third step, the horizontal histogram builder HHB establishes a histogram of horizontal interpeak distances HHI on the basis of the simplified array, which results from the first and second step. The horizontal histogram builder HHB scans the simplified array, as it were, peak by peak. For each peak, the horizontal histogram builder HHB determines respective absolute distances, in terms of number of pixels, from the peak to respective other peaks, which are comprised in the simplified array. Each absolute distance counts as one histogram value unit.

FIG. 6 illustrates an example of the histogram of horizontal interpeak distances HHI. The histogram of horizontal interpeak distances HHI has a horizontal axis, which represents absolute interpeak distances IPD, and a vertical axis which represents histogram values HV. The histogram of horizontal interpeak distances HHI illustrates that more than 80 times absolute distances of 8, 16, 24, and 32 pixels have been found. Consequently, the regular intervals at which the relevant peaks substantially occur, are 8 pixels wide.

The horizontal histogram builder HHB further establishes the histogram of horizontal peak offsets HHO, while scanning the simplified array peak by peak. For each peak, the horizontal histogram builder HHB determines the distance between the peak and the origin of the simplified array. The distance corresponds with the column number. The horizontal histogram builder HHB expresses the distance as an integer number of the width of the regular intervals and a remaining portion, which is smaller than the aforementioned width. The remaining portion represents the peak offset. For example, let it be assumed that a relevant peak occurs in column number 34. The regular intervals are 8 pixels wide. The distance, which is 34, can be expressed as 4 times 8 pixels, which is the width of the regular intervals, plus a remaining portion of 2 pixels. The remaining portion is the peak offset, which is 2 pixels in this example.

FIG. 7 illustrates the histogram of horizontal peak offsets HHO. The histogram of horizontal peak offsets HHO has a horizontal axis, which represents peak offset PO, and a vertical axis which represents histogram values HV. The histogram of horizontal peak offsets HHO illustrates that the peak offset is 2 pixels for a great majority of the relevant peaks. FIG. 8 illustrates a vertical grid VG, which the vertical grid detector VGD determines on the basis of the histogram of horizontal interpeak distances HHI and the histogram of horizontal peak offsets HHO. Two parameters characterize the vertical grid VG: the vertical grid pitch VGP and the vertical grid offset VGO. The vertical grid detector VGD determines the vertical grid pitch VGP on the basis of the histogram of horizontal interpeak distances HHI, which indicates the width of the regular intervals at which the relevant peaks substantially occur. The vertical grid pitch VGP corresponds with the width of these regular intervals. The vertical grid detector VGD determines the vertical grid offset VGO on the basis of the histogram of horizontal peak offsets HHO. The vertical grid offset VGO corresponds with the peak offset in this histogram that has the highest histogram value.

The horizontal grid detection path HGDP, which FIG. 4 illustrates, establishes a horizontal grid HG in a similar fashion. In effect, the horizontal grid detection path HGDP carries out operations that are substantially identical to those described hereinbefore when replacing the term "horizontal" by the term "vertical" and vice versa.

The vertical gradient calculator VGC calculates a vertical gradient for substantially each pixel in the image. The vertical gradient of a pixel is the absolute difference between the pixel and the closest lower adjacent pixel. The vertical gradient may mathematically be expressed as:

Accordingly, the vertical gradient calculator VGC provides a matrix of vertical gradients VGM, wherein each cell corresponds with a particular pixel and defines the vertical gradient for that pixel. The vertical gradient normalizer VGN establishes a normalized vertical gradient for substantially each pixel in the image. The normalized vertical gradient of a pixel is equal to the vertical gradient of the pixel divided by an average of respective vertical gradients of respective neighboring pixels. The normalized vertical gradient may mathematically be expressed as:

wherein Dv_iav(i,j ) denotes the average of respective horizontal gradients of respective neighboring pixels. This average may mathematically be expressed as:

wherein K is an integer number that denotes the number of neighboring pixels, which are lower adjacent or upper adjacent, that are taken into account in establishing the average. Accordingly, the vertical gradient normalizer VGN provides a matrix of normalized vertical gradients HVGM, wherein each cell corresponds with a particular pixel and defines the normalized vertical gradient for that pixel.

The horizontal accumulator HAC establishes an array of accumulated normalized vertical gradients VAA on the basis of the matrix of vertical normalized gradients HVGM. An accumulated normalized vertical gradient is the sum of normalized vertical gradients in a particular row of the matrix of normalized vertical gradients. The accumulated normalized vertical gradient may mathematically be expressed as:

Wherein Sv(O denotes the accumulated normalized vertical gradient for row "f in the matrix. Each row comprises M cells as FIG. 2C illustrates. The vertical histogram builder VHB builds two histograms on the basis of the array of accumulated normalized vertical gradients: a histogram of vertical interpeak distances VHI and a histogram of vertical peak offsets VHO. The vertical histogram builder VHB operates in a fashion that is similar to the fashion in which the horizontal histogram builder HHB builds the histogram of horizontal interpeak distances HHI and the histogram of horizontal peak offsets HHO, which FIG. 6 and 7, respectively, illustrate. The horizontal grid detector HGD determines the horizontal grid HG on the basis of the histogram of vertical interpeak distances VHI and the histogram of vertical peak offsets VHO.

The grid composer GCO combines the vertical grid VG and the horizontal grid HG so as to obtain the coding grid parameters GRD that characterize the coding block grid. The coding grid parameters GRD comprise the horizontal grid pitch , the vertical grid pitch VGP, the horizontal grid offset, and the vertical grid offset VGO. The block artifacts attenuator BAA receives these parameters, which FIG. 3 illustrates.

The block artifact metric calculator BMC, which forms part of the block artifacts detector BDT that FIG. 4 illustrates, calculates the block artifact metric BM, which the block artifacts attenuator BAA receives. The block artifact metric calculator BMC calculates the block artifact metric BM on the basis of the array of accumulated normalized horizontal gradients HAA, the array of accumulated normalized vertical gradients VAA, the vertical grid VG, and the horizontal grid HG. The aforementioned grids allow the block artifact metric calculator BMC to distinguish between normalized gradients that belong to the coding block grid and other normalized gradients.

The block artifact metric calculator BMC calculates the average of accumulated normalized gradients that belong to the coding block grid and the average of the other accumulated normalized gradients, which do not belong to the coding block grid. Subsequently, the block artifact metric calculator BMC divides the first mentioned average by the last mentioned average. This division provides a gross block artifact metric, which may mathematically be expressed as:

S_{H V}(bl) S_{H V}(nbl) wherein S_{H v} (bl) denotes the average of accumulated normalized gradients that belong to the coding block grid, and wherein S_{H r} (nbl) denotes the average of the other accumulated normalized gradients, which do not belong to the coding block grid.

The higher the value of the gross block artifact metric is, the more visible the block artifacts will be. For example, block artifacts will hardly be visible if the gross block artifact metric has a value that is comprised between 1 and 2. In contradistinction, block artifacts will be quite visible if the gross block artifact metric has a value that is comprised between 4 and 5.

The block artifact metric BM is equal to the gross block artifact metric multiplied by a grid-pitch dependent correction factor. This may mathematically be expressed as:

BM = BM_gross - f(d) wherein/^ represents the grid-pitch dependent correction factor. The grid- pitch dependent factor accounts for the following subjective phenomenon. The larger the pitch of a coding block grid is, the more annoying block artifacts are. The grid-pitch dependent correction factor can empirically be established. It has been found that the following grid-pitch dependent correction factor provided satisfactory results:

wherein d represents the pitch of the coding block grid. FIG. 9 illustrates an alternative video processor VPRa, which may replace the video processor VPR in the video display set that FIG. 1 illustrates. The alternative video processor VPRa comprises an alternative block artifacts detector BDTa, an alternative video enhancer ENHa, and two buffer memories BUF 11, BUF 12. The alternative block artifacts detector BDTa is substantially similar to the block artifacts detector BDT that has been described hereinbefore with reference to FIGS. 4-8. The alternative block artifacts detector BDTa need not comprise the grid composer GCO that FIG. 4 illustrates and need not provide the coding grid parameters GRD. It is sufficient that the alternative block artifacts detector BDTa provides the block artifact metric BM described hereinbefore.

The alternative video enhancer ENHa, which receives the block artifact metric BM, carries out video processing functions that aim to enhance subjective image quality. For example, the alternative video enhancer ENHa may carry out video processing functions that are designated by the term "Pixel Plus", which is a registered trademark assigned to Koninklijke Philips Electronics N. V.. The alternative video enhancer ENHa adapts these video processing functions on the basis of the block artifact metric BM. As explained hereinbefore, such video processing functions may amplify block artifacts. The block artifact metric BM allows the alternative video enhancer ENHa to optimize the video processing functions in the sense that block artifact amplification, if any, remains below a critical level at which block artifacts become visible.

FIG. 10 illustrates a software-enabled implementation of the video processor VPR. The software-enabled implementation comprises an input buffer IBU, a processing circuit CPU, a program memory PMEM, a data memory DMEM, an output buffer OBU, and a bus BS, which couples the aforementioned elements to each other.

The program memory PMEM comprises a set of instructions, i.e. software, which causes the processing circuit CPU to carry out the various different operations that have been described hereinbefore with reference to FIGS. 3 to 8. For example, the set of instructions may comprise various modules, each of which corresponds to a particular functional entity that FIG. 3 or 4 illustrates. The set of instructions may be loaded into the program memory PMEM from a data carrier on which the set of instructions is stored. The data carrier may be, for example, a CD ROM or a similar or carrier. The data carrier may also take the form of a server, which can be accessed via a communication network so as to download the set of instructions that causes the processing unit CPU to carry out the various different operations that have been described hereinbefore.

The data memory DMEM stores intermediate results of video-processing operations. For example, the data memory DMEM may comprise a set of memory locations that forms buffer memory BUFl, which FIG. 3 illustrates. The same applies to buffer memories BUF2, BUF3, which may equally be implemented as respective sets of memory locations in the data memory. The set of instructions may comprise a module that statically or dynamically assign respective sets of memory location to respective type of data, which need to be buffered.

CONCLUDING REMARKS

The detailed description hereinbefore with reference to the drawings illustrates the following characteristics, which are cited in various independent claims. In order to detect block artifacts, a normalized gradient map (HNGM) is established (HGC, HGN) that indicates locations within an image where a difference between adjacent image elements along an axis (i) is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations. Subsequently, a grid (VG) is established (VAC, HHB, VGD) that matches the locations within the image where a difference between adjacent image elements along the axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations.

The detailed description hereinbefore further illustrates various optional characteristics, which are cited in the dependent claims. These characteristics may be applied to advantage in combination with the aforementioned characteristics. Various optional characteristics are highlighted in the following paragraphs. Each paragraph corresponds with a particular dependent claim.

Respective gradients (D_H) are calculated at respective locations within at least a portion of the image. A gradient (D_H) represents a difference between adjacent image elements along the axis at a particular location within the image. Respective normalized gradients (D_H,norm) are established subsequently. A normalized gradient (D_H,norm) represents the magnitude of a gradient (D_H) at a particular location relative to respective magnitudes of respective gradients at neighboring locations. This set of characteristics allows the normalized gradient map to be established in a cost-efficient manner.

An array of accumulated normalized gradients (HAA) is established (VAC) on the basis of the normalized gradient map (HNGM). An accumulated normalized gradient (S_H) represents a sum of respective normalized horizontal gradients (D_H,™™) that have the same location along the axis. The array of accumulated normalized gradients (HAA) comprises respective accumulated normalized gradients (S_H) for respective different locations along the axis. This set of characteristics further contributes to reliable block artifact detection and, therefore, contribute to image quality improvement.

Relevant peaks in the array of accumulated normalized gradients (HAA) are identified (HHB). A relevant peak is an accumulated normalized gradient (S_H) that is greater than any accumulated normalized gradient in a set of neighboring accumulated normalized gradients. A simplified array of normalized gradients represents the relevant peaks. This set of characteristics further contributes to reliable block artifact detection and, therefore, contribute to image quality improvement.

A block artifact metric (BM) is established (BMC). The block artifact metric (BM) varies as a function of a ratio between normalized gradients (DH,_nOrm) at locations that coincide with the grid (VG) and normalized gradients (DH,_nOrm) at other locations, which do not coincide with the grid (VG). This set of characteristics further contributes to image quality improvement. This is because the block artifact metric is a relatively precise indication of the degree of visibility of block artifacts. Consequently, the block artifact metric allows a satisfactory compromise between block artifact attenuation and loss of resolution.

The pitch of the grid (VG) is included in establishing (BMC) the block artifact metric (BM) so that the block artifact metric (BM) varies as a function of the pitch of the grid (VG). This characteristic allows the block artifact metric to be an even more precise indication of the degree of visibility of block artifacts. The aforementioned characteristics can be implemented in numerous different manners. In order to illustrate this, some alternatives are briefly indicated.

The aforementioned characteristics may be applied to advantage in any type of product or method that relates to image processing. A video display driver is merely an example. The aforementioned characteristics may equally be applied in, for example, a digital photo and/or video camera, a photo-editing and/or video-editing software program, a personal computer, a set-top box, a television set, a personal digital assistant, a mobile phone, and so on.

There are numerous different manners to establish a normalized gradient map. The detailed description hereinbefore merely illustrates one possible manner. For example, the horizontal gradient normalizer may be modified as follows. The average of respective horizontal gradients of neighboring pixels is a weighted average; each of these horizontal gradients is multiplied by a weighting coefficient.

It should further be mentioned that the normalized gradient map may relate to only a portion of the image. For example, the block artifact detector that FIG. 4 illustrates may take a portion of 100 by 100 pixels from the image and establish the matrix of normalized horizontal gradients on the basis that portion of the image. This reduces computational power and, in many cases, will still allow reliable grid detection.

There are numerous different manners to establish a grid on the basis of a normalized gradient map. There are many suitable statistical analysis techniques other than the histogram-based approach in the detailed description. All what matters is that a statistical technique, or other technique, extracts a coding block grid on the basis of a normalized gradient map. As mentioned hereinbefore, grid detection may be based on only a portion of the image. It should be noted that a grid need not necessarily have a pitch that is an integer number of pixels. For example, a block artifacts detection in accordance with the invention may extract a grid that is \0²Λ pixels wide. Such a grid may result from a downscaling of an original image by a factor 1.33 before encoding, which is based on an 8 x 8 pixel block division, and a complementary upscaling after decoding.

An image processor in accordance with the invention can be implemented in numerous different manners. FIG. 3 merely illustrates an example, which comprises the block artifacts attenuator BAA and the video enhancer ENH, shown as separate functional entities. The aforementioned functional entities may be combined into a single entity.

The term "image" should be understood in a broad sense. This term includes a frame, a field, and any other entity that may wholly or partially constitute an image or picture.

There are numerous ways of implementing functions by means of items of hardware or software, or both. In this respect, the drawings are very diagrammatic, each representing only one possible embodiment of the invention. Thus, although a drawing shows different functions as different blocks, this by no means excludes that a single item of hardware or software carries out several functions. Nor does it exclude that an assembly of items of hardware or software or both carry out a function.

The remarks made herein before demonstrate that the detailed description with reference to the drawings, illustrate rather than limit the invention. There are numerous alternatives, which fall within the scope of the appended claims. Any reference sign in a claim should not be construed as limiting the claim. The word "comprising" does not exclude the presence of other elements or steps than those listed in a claim. The word "a" or "an" preceding an element or step does not exclude the presence of a plurality of such elements or steps.

Claims

Claims.

1. An image processor (VDD) comprising a block artifacts detector (BDT), the block artifacts detector (BDT) comprising: a normalized gradient calculator (HGC, HGN) arranged to establish a normalized gradient map (HNGM) that indicates locations within an image where a difference between adjacent image elements along an axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations; and a grid calculator (VAC, HHB, VGD) arranged to establish a grid (VG) that matches the locations within the image where a difference between adjacent image elements along the axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations.

2. An image processor (VDD) as claimed in claim 1, the normalized gradient calculator (HGC, HGN) comprising: - a gradient calculator (HGC) arranged to calculate respective gradients

(D_H) at respective locations within at least a portion of the image, a gradient (D_H) representing a difference between adjacent image elements along the axis at a particular location within the image; and a gradient normalizer (HGN) arranged to establish respective normalized gradients (D_H,norm), a normalized gradient (D_H,norm) representing the magnitude of a gradient (D_H) at a particular location relative to respective magnitudes of respective gradients at neighboring locations.

3. An image processor (VDD) as claimed in claim 2, the grid calculator (VAC, HHB, VGD) comprising: a gradient accumulator (VAC) arranged to establish an array of accumulated normalized gradients (HAA) on the basis of the normalized gradient map (HNGM), an accumulated normalized gradient (S_H) representing a sum of respective normalized horizontal gradients (DH,_nOrm) that have the same location along the axis, the array of accumulated normalized gradients (HAA) comprising respective accumulated normalized gradients (S_H) for respective different locations along the axis.

4. An image processor (VDD) as claimed in claim 3, the grid calculator (VAC, HHB, VGD) comprising: an array processor (HHB) arranged to identify relevant peaks in the array of accumulated normalized gradients (HAA), a relevant peak being an accumulated normalized gradient (S_H) that is greater than any accumulated normalized gradient in a set of neighboring accumulated normalized gradients so as to form a simplified array of normalized gradients that represents the relevant peaks.

5. An image processor (VDD) as claimed in claim 1, the block artifacts detector (BDT) comprising: a pair of normalized gradient calculators (HGC, HGN; VGC, VGN), one normalized gradient calculator (HGC, HGN) being arranged to establish a normalized gradient map (HNGM) that relates to one axis, the other normalized gradient calculator (VGC, VGN) being arranged to establish a further normalized gradient map (VNGM) that relates to another axis; a pair of grid calculators (VAC, HHB, VGD; HAC, VHB, HGD), one grid calculator (VAC, HHB, VGD) being arranged to establish a grid (VG) relates to one axis, the other grid calculator (HAC, VHB, HGD) being arranged to establish an other grid (HG) relates to the other axis.

6. An image processor (VDD) as claimed in claim 2, the block artifacts detector (BDT) comprising: a block artifact metric calculator (BMC) arranged to establish a block artifact metric (BM) that varies as a function of a ratio between normalized gradients

(DH,norm) at locations that coincide with the grid (VG) and normalized gradients (D_H,norm) at other locations, which do not coincide with the grid (VG).

7. An image processor (VDD) as claimed in claim 6, the block artifact metric calculator (BMC) being arranged to include the pitch of the grid (VG) in establishing the block artifact metric (BM) so that the block artifact metric (BM) varies as a function of the pitch of the grid (VG).

8. An image processor (VDD) as claimed in claim 1, the image processor (VDD) comprising: a block artifact attenuator (BAA) arranged to smooth image elements at locations that coincide with the grid (VG).

9. An image processor (VDD) as claimed in claim 6, the image processor (VDD) comprising: a block artifact attenuator (BAA) arranged to smooth image elements at locations that coincide the grid (VG) to an extent that depends on the block artifact metric (BM).

10. A method of image processing comprising: a normalized gradient calculation step (HGC, HGN) in which a normalized gradient map (HNGM) is established that indicates locations within an image where a difference between adjacent image elements along an axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations; and a grid calculation step (VAC, HHB, VGD) in which a grid (VG) is established that matches the locations within the image where a difference between adjacent image elements along the axis is significant relative to respective differences between respective adjacent image elements along the axis at neighboring locations.

11. A computer program product that comprises a set of instructions, which when loaded into a processor, causes the processor to carry out the method according to claim 10.

12. An image display system (VDS) comprising an image processor (VDD) (VDD) according to claim 8 for providing an enhanced image signal (DSS) in response to an input image signal (VA, VB, VC), and a display device (DPL) for displaying the enhanced image signal.