WO2010029476A1

WO2010029476A1 - Bilateral filter

Info

Publication number: WO2010029476A1
Application number: PCT/IB2009/053858
Authority: WO
Inventors: Robert-Paul M. Berretty; José P. MAGALHAES; Reinier B. M. Klein Gunnewiek; Bart G. B. Barenbrug; Ling Shao
Original assignee: Koninklijke Philips Electronics N.V.
Priority date: 2008-09-11
Filing date: 2009-09-04
Publication date: 2010-03-18

Abstract

A system for applying a bilateral filter to an image comprising a plurality of elements is provided. Input (101) is arranged for receiving respective local control parameter values (109) in respect of respective center elements of respective windows of an image and indicative of a difference-to-weight function (104). Weight generator (102) is arranged for establishing a plurality of respective weights (112) of a plurality of respective window elements of one of the windows (502) by computing a respective weight of a respective window element (504) by applying the difference-to -weight function (104) to a difference between an element value (110) of the center element (503) of the one of the windows (502) and a respective element value (111) of the respective window element (504), the weight generator (102) comprising means (103) for determining the difference-to-weight function (104) based on the local control parameter value (109) in respect of the center element (503).

Description

Bilateral filter

FIELD OF THE INVENTION

The invention relates to a bilateral filter.

BACKGROUND OF THE INVENTION The bilateral filter is a technique to smooth images while preserving edges. It has been used in various contexts such as denoising, texture editing and relighting, tone management, demosaicing, stylization, and optical-flow estimation. In a bilateral filter, each pixel is replaced by a weighted average of its neighbors. By properly designing the filter, it is possible to determine the size and contrast of the features to preserve. The bilateral filter is described in "Bilateral filtering for gray and color images" by Tomasi, C. and Manduchi, R., In: Proc. of International Conference on Computer Vision, IEEE (1998) 839-846. A fast implementation is described in "A Fast Approximation of the Bilateral Filter Using a Signal Processing Approach" by S. Paris and Fredo Durand, in International Journal of Computer Vision, 2007, Springer Link, DOI 10.1007/sl 1263-007-0110-8. Similarly to the Gaussian convolution, the bilateral filter is also defined as a weighted average of pixels. The difference is that the bilateral filter can take into account the variation of intensities to preserve edges. The rationale of bilateral filtering is that two pixels are close to each other not only if they occupy nearby spatial locations but also if they have some similarity with respect to pixel values. One example of a bilateral filter, denoted by BF[.], is defined by:

V] GV, (||p - q||) G_nr Λ> - I_n ⁾ I κ q÷:<s where W_p is a normalization factor and S a neighborhood of p: τr_p = y" fC 2 I(| I ips-* _-___ rq*||) \ fG 2'_Λ, 1(/ !p .-... _{λ_q) ,

P ^ qe ^■-- ,s p and q denote pixel indices, I_p and I_q denote the pixel values at pixels p and q, respectively, parameters σ_s and σ_r determine the amount of filtering for the image I. BF[I]_P is a normalized weighted average. G_σs is a spatial Gaussian that decreases the influence of distant pixels. G_σr is a range Gaussian that decreases the influence of pixels q with an intensity value I_q different from Ip. Herein, the term "range" refers to quantities related to pixel values, whereas the term "space" refers to quantities related to pixel locations.

The bilateral filter may be controlled by two parameters: σ_s and σ_r. As the range parameter σ_r increases, the bilateral filter becomes closer to Gaussian blur because the range Gaussian is flatter. In the extreme case, the range Gaussian is almost constant over the intensity interval covered by the image. Increasing the spatial parameter σ_s results in smoothing larger features.

The bilateral filter may be used, for example, to smooth an image while preserving its discontinuities, but also to separate structures of different scales. The bilateral filter may also be used to combine data from different sources to obtain a single result. For example, in a 3D video application, a depth map may be obtained in addition to the luminance image (or color image). If the depth map needs to be smoothened, the bilateral filter can be used to take into account the edges in the luminance or color image. This can be done by substituting luminance (or color) pixel values for I_p and I_q when computing

G_σ (I_p -I_q) in the equations for computing BF[I]_P and W_p, while substituting depth pixel values in the rightmost occurrence of I_q in the equation for computing BF[I]_P.

The results obtained with the bilateral filter are not always satisfactory.

SUMMARY OF THE INVENTION

It would be advantageous to have an improved bilateral filter. To better address this concern, in a first aspect of the invention a system is presented for applying a bilateral filter to an image comprising a plurality of elements, each element having at least one element value associated therewith, comprising an input for receiving respective local control parameter values in respect of respective center elements of respective windows of an image and indicative of a transfer characteristic of a difference-to -weight function; and a weight generator for obtaining a plurality of respective weights of a plurality of respective window elements of one of the windows by computing a respective weight of a respective window element by applying the difference-to -weight function to a difference between an element value of the center element of the one of the windows and a respective element value of the respective window element, the transfer characteristic of the difference- to-weight function being controlled by the local control parameter value in respect of the center element. Thanks to the local control parameter value, the effect of the bilateral filter can be controlled locally. This can be used to locally control the amount of edge enhancement, smoothing, and/or sharpening which is provided by the bilateral filter. For example, the elements may be pixels. The inventors have realized, that it is advantageous to be able to locally adapt the way the weights depend on the differences. This adaptation can be based on the local properties of the image data, for example. For example, using the notation introduced above, it is advantageous to adapt the function G_σ (I ~I_q) using a local control parameter reflecting a local image property. For example, the local image property is based on a local reliability measure of the image. This allows improving the filter result by, for example, allowing more smoothing in an image region where the reliability of the image is lower. This way, errors in low-reliability areas may be averaged out by making the difference-to -weight function less steep. On the other hand, where the reliability of the image data is higher, the edges may be preserved more by making the difference-to-weight function steeper.

For example the steepness of the function G_σ is adapted, thereby increasing or decreasing the influence of the difference I_p -I_q on the weight.

Within the context of this document the qualifier "local", e.g. in the expression "local control parameter", is intended to designate a spatial context in respect of a particular image element in an image. The expression local is used to differentiate between a local context being a subset of the image and a global context which relates to the image in its entirety. The local context may be defined spatially by means of a window, such as e.g. by means of an aperture associated with the bilateral filter.

Preferably the local context is defined by means of a distance metric. Preferably such a distance metric refers to a spatial distance metric, i.e. a distance metric wherein a threshold determines what is within the local context and what is not. Alternatively the metric can relate to a distance metric spanning the spatial the range, and/or the temporal domain. The threshold in turn may be chosen to be a predetermine value or can be chosen to be image adaptive. It is further noted that the local context, e.g. when referring to the local control parameter of an image element can correspond to the same local context as used for defining local reliability. However this does not preclude the use of a different local spatial context, such as may be the case determining the local homogenity or local edginess. The local context may correspond to a fixed regular spatial area but can also relate to irregular shaped spatial areas, e.g when segmentation information is available.

In an embodiment, the local control parameter relates to a quantification of edginess of a neighborhood of the center pixel. The bilateral filter is commonly used as an edge preserving smoothing filter. However, these edges are preserved better by adapting the function used to derive the weights in dependence on a quantification of local edginess. This embodiment allows making the dependence of the weights on the difference less pronounced if the local edginess is lower and more pronounced if the local edginess is higher, which improves the output of the bilateral filter by preserving edges better. In an embodiment, the local control parameter is derived from a local image property being a quantification of edginess of a neighborhood of the center element, and the difference of element values is a difference of the element value of the central element in a first image and the element value of the element in the window in a second image, the first image and the second image being different images in a temporal image sequence, and wherein the weights are applied to an element value of the element in the window in a third image to obtain a weighted sum. If the third image comprises a local property of the second image, for example the third image is a depth image corresponding to the second image, then the result of the bilateral filtering process can comprise a fourth image comprising the local image property for the first image. The use of the edginess to derive the local control parameter value helps to improve the accuracy of the fourth image.

In an embodiment, the difference-to -weight function is made steeper where the local image contrast is lower. This helps to preserve edges in low-contrast areas. On the other hand, the difference-to -weight function is made less steep where the local image contrast is higher. This helps to smooth away noise, under the assumption that where the local image contrast is higher, small variations in element value are less probably due to an edge.

BRIEF DESCRIPTION OF THE DRAWINGS

These and other aspects of the invention will be further elucidated and described with reference to the drawing, in which Fig. 1 is a diagram of a system for applying a bilateral filter;

Fig. 2 is a diagram of a method of applying a bilateral filter; Fig. 3 is a graph showing an example of how a difference-to -weight function is controlled by a local control parameter; and

Fig. 4A and Fig. 4B illustrate results of different bilateral filters. Fig. 5 is a diagram of an image comprising elements.

DETAILED DESCRIPTION OF EMBODIMENTS

Fig. 1 illustrates an embodiment of a system for applying a bilateral filter. The bilateral filter may be applied to an image comprising a plurality of elements. For example, the elements are pixels, although other kinds of data elements can also be used. Each element has at least one element value associated therewith. Fig. 5 will be used to illustrate certain aspects relating to an image.

Fig. 5 diagrammatically shows an image 501 having a plurality of elements indicated at for example 503, 504 and 507. In bilateral filtering, the value of a center element 503 of a window 502 is a weighted average of the elements in the window 502, wherein the value of a weight depends on the difference between the element value of the center element and the element value of a window element. The elements in a window 502 may be referred to as window elements. An example window element is indicated at 504. Other window elements are indicated by the small circles within the window 502. Other windows 505, 506 can be created in the image at will. In principle, any element 507 of the image can act as the center element 503 of a window 502. The size and shape of the window 502 depend on the application at hand. Typically the window 502 will lie around the center element 503. However, this is not a limitation. The center element 503 may or may not be in the center of the window 502. The window may consist of data elements having an association with the center element 503 other than their relative position. Also, in case of a center element at the edge of the image, the window may lie on one side of the center element rather than around the center element. It will be understood that one or more element values can be associated with an element. For example, in an image sequence, each image in the image sequence associates different element values with the same elements. Also, in the example of a three- dimensional color image, an element may have element values for hue, saturation, luminance, and depth.

Returning to Fig. 1, a memory 108 is provided for storing images and other quantities and intermediate results 109-114. Several processing blocks are indicated 101-106. These processing blocks may be implemented by computer instructions stored in a memory, for example. Such computer instructions are executed under control of a central processing unit or general purpose processor 115. Alternatively, one or more of the processing blocks 101-106 may be implemented in the form of an electronic circuit such as an integrated circuit (IC). Although not shown in the image, communication means may be provided for retrieval and/or transmission of image data. Such communication means may comprise an Internet connection or a connection to a digital or analog cable television network, for example. Also, non-volatile mass storage media may be provided, such as CD, DVD, hard disk, or flash memory, for storing and retrieval of large amounts of image data. These mass media can take the form of removable media and non-removable media. Also, control means may be provided for controlling the system, and a video output or a display for showing video data to a viewer.

An input 101 is provided for receiving a plurality of local control parameter values 109. These local control parameter values 109 may be stored in memory 108, for example. A local control parameter value generator (not shown) may be provided to compute the local control parameter values, for example based on a local image property. The local control parameter value generator provides the local control parameter values to the input 101. The respective local control parameter values 109 act in respect of respective center elements of respective windows of an image. The local control parameter values 109 are indicative of a transfer characteristic of a difference-to-weight function 104, as will be elaborated hereinafter. The weights in respect of window elements of one window may be computed using one local control parameter value and, consequently, the same transfer characteristic and the same difference-to-weight function.

A weight generator 102 is provided for establishing a plurality of respective weights 112 of a plurality of respective window elements of one of the windows 502. The weight generator 102 comprises a means 103 for determining the transfer characteristic of the difference-to -weight function 104 based on the local control parameter value 109 in respect of the center element 503. Different ways to determine the transfer characteristic are contemplated. This will be elaborated hereinafter. The weight generator 102 is arranged for computing a difference between an element value 110 of the center element 503 of one of the windows 502 and a respective element value 111 of a window element 504. This difference is used in the difference-to- weight function 104 to compute the weight. The difference may be an absolute difference or a signed difference. In most applications, the absolute difference is used. It is also possible to compute the square of the difference or another function of the difference.

The weight generator 102 computes a weight 112 of a window element 504 by applying the difference-to -weight function 104 to the computed difference. The transfer characteristic corresponding to the control parameter value, as determined by the means 103, is used in the difference-to -weight function 104. A summation means 105 may be provided to compute a weighted sum of element values 113 of the window elements 504 of a window 502 according to the weights 112 computed by the weight generator 102. This weighted sum may be assigned to the filtered element value 114 of a center element 503. The weighted sum may be normalized by dividing by the sum of the weights. Alternatively, the weighted sum may be normalized by ensuring that the sum of the weights equals 1. Alternatively, the weighted sum may not be normalized.

A video output 106 may be provided to prepare a video signal representing the filtered element values 114. The filtered element values may be used in the video signal as filtered luminances, filtered depth values, for example, depending on the type of values used as the input values 109, 110, 111, 113.

A display (not shown) may be used to show a rendering of the video signal to a viewer.

Fig. 2 illustrates an example flow of operation of the system of Fig. 1. The process starts at 201 when the filter operation is initiated, for example by an arriving video frame. In step 202, a center element 503 and corresponding window 502 are selected in the image. For example, the upper left element is selected first, later other elements may be selected sequentially. The window 502 may be a set of elements 504 around the center pixel 503, for example in a square window 502 having the center pixel 503 in the middle. However other arrangements are also possible. In step 203, an element value 110 of the center element 503 is identified. In step 204, a local control parameter value 109 corresponding to the selected center element 503 is received via input 101. In step 205, the transfer characteristic of the difference-to-weight function 104 is determined based on the local control parameter value 109.

In step 206, a window element 504 is selected and an element value 111 of the window element 504 is identified. In step 207, the difference is computed between the element value 111 of the window element 504 and the element value 110 of the center element 503. The absolute value of the difference may be computed. In step 208, a weight 112 is computed using the difference-to -weight function 104 and the transfer characteristic corresponding to the local control parameter value 109. In step 209, the summation means 105 identifies an element value 112 of the window element 504. This may be the same element value 111 used in the difference computation. It may also be another element value of the window element 504. This depends on the application. The element value 112 is multiplied by the weight 112 and the result is added to a summation value. In step 210, it is checked whether all window elements 504 of the window 502 have been processed. If not, the flow returns to step 206 to select the next window element.

If yes, the flow proceeds to step 211, wherein the summation value is normalized. This is achieved by dividing the summation value by the sum of all the weights used in respect of this window 502. In step 212, the normalized summation value 114 is stored in memory, on disk, or output to the video output 106. In step 214, it is checked whether all relevant elements have been selected as the center element 503. If no, the flow returns to step 202 to select the next center element 503 and corresponding window 502. If yes, the process terminates in step 215. The transfer characteristic may be indicative of a steepness of the difference- to-weight function. In this case, the means 103 for determining the transfer characteristic comprises means for determining the steepness of the difference-to-weight function. It is a way to determine the tradeoff between the amount of smoothing and the amount of edge preservation. The control parameter value 109 may be a reliability estimation. Such a reliability estimation can be computed by a reliability estimator (not shown). The reliability estimator can be a local noise detector or edge detector. If the reliability is low, more smoothing may be appropriate. If the reliability is high, better edge preservation may be appropriate. Accordingly, the reliability estimator may be arranged for establishing a local control parameter value corresponding to a less steeply decreasing dependence of the weight on the difference in response to a smaller local reliability value.

In an embodiment, the difference-to -weight function 104 defines a decreasing dependence of the weight on the difference. The local control parameter value 109 defines how quickly the weight decreases when the difference increases. This way, the local control parameter value 109 determines the steepness of the difference-to-weight function.

In an embodiment, a local contrast estimator is provided for establishing respective local contrast or edginess values in respect of the respective center elements of the respective windows. The respective local control parameter values may be based on the respective local contrast or edginess values. Depending on the application at hand, a higher local contrast estimation may lead to either increased or decreased steepness of the difference-to -weight function 104. If the steepness is increased, weak edges are smoothed out more in regions with little contrast. If the steepness is decreased, weak edges are preserved better in regions with little contrast. A combination is also possible where for example the steepness for small differences is decreased to provide more smoothing for small noise differences, but the steepness for higher differences is increased, to better sharpen clearer edges. In this case there may be two local control variables: one which relates to the amount of noise in the picture and which controls the steepness for small difference values, and another which for example relates to reliability or edginess, which controls the steepness for larger difference values.

In a concrete example of a difference-to -weight function and local control parameter, the difference-to-weight function comprises a term b^ad , wherein b is a base value, α is the local control parameter, and d is the difference.

In an embodiment, the bilateral filter may be defined by: BF[I,J,K,L]_p = ^~∑w_qK_q , wherein

BF[-]_p denotes an outcome of the bilateral filter at an element p, which may be computed by the summation means 105.

Wq denotes a weight w_q = G_s

- q\\) G₁, yL_p , h_p - J_q J , which value may be computed by the weight generator 102. W_p denotes a normalization factor W_p = V w_g . qeS p and q denote element indices.

/ denotes an element value 110 of element p in a first image.

J _q denotes an element value 111 of element q in a second image. K_q denotes an element value 113 of element q in a third image. Two or all of the first image, second image, and third image may be the same image.

L_p denotes the local control parameter value 109 in respect of a center element p.

G_s denotes a spatial function. G_r denotes a range function whose behavior in respect of

-JJ is controlled by the control parameter value L_p .

It can be said that the bilateral filter system is arranged for filtering the third image. The filtering depends on the first image, second image, and local control parameter values. In an embodiment, the first image and the second image are different images in a temporal image sequence, and the third image is a depth image corresponding to the second image. This way, a second depth image can be created for the second image. To create the second depth image, the depth values of the first image are weighted according to the correspondence between image element values in the first image and the second image. This way, depth values of elements having element values in the second image that are similar to element values of a center element in the first image contribute more to the weighted sum of depth elements. The local control parameter may correspond to a local edginess in one of the images. In case of high edginess, the steepness of the difference-to- weight function 104 may be increased (or decreased). Increasing helps to obtain strong edges at object boundaries. Decreasing the steepness function where the local edginess is higher helps to decrease noise by better smoothing of low-amplitude edges which may be caused by noise. Moreover, this decreasing helps to prevent or reduce leakage of edginess of the image content into the bilateral filtered signal. In an embodiment, G₁, ( L_p , Il _p - J J J = exp (

-OcZ^ ~ J _q 1) , for a given value

CC . The local control parameter value L_p may be an estimation of local edginess.

An embodiment comprises means for determining the spatial function G_s based on a local control parameter value. This local control parameter may be the same local control parameter used in G_r, but it may also be a different local control parameter. For example, the steepness of the spatial function G_s can be adapted based on the local control parameter. Also, the footprint of the bilateral filter may be determined by G_s. In an embodiment, G_s is adapted based on the local reliability of the image data. Where the local reliability is low, the footprint of the bilateral filter may be enlarged to average out a larger region. In view of this text, it will be apparent to the skilled person how to determine the footprint of the bilateral filter by properly adapting G_s.

The system for applying the bilateral filter may be applied in a video processing system. It will be understood that the system may also be applied in an audio processing system or in any other appropriate digital signal processing system. Examples of use are in a set-top box, a television, a digital video recorder, a multimedia player on a personal computer, an audio amplifier.

A method of applying a bilateral filter to an image comprising a plurality of elements, each element having at least one element value associated therewith, comprises receiving 204 respective local control parameter values in respect of respective center elements of respective windows of an image and indicative of a transfer characteristic of a difference-to -weight function; and obtaining 208 a plurality of respective weights of a plurality of respective window elements of one of the windows by computing a respective weight of a respective window element by applying the difference-to -weight function to a difference between an element value of the center element of the one of the windows and a respective element value of the respective window element, the transfer characteristic of the difference-to -weight function being controlled by the local control parameter value in respect of the center element.

In an embodiment, a local depth signal is provided comprising respective local depth values in respect of the respective center elements of the respective windows. The respective local control parameter values may be based on the respective local depth values. In an embodiment, leaking of depth information over multiple objects in the image is reduced by using the edge strength of the image (or the edge strength of the depth map) as a local control parameter for a bilateral filter. This can provide better edge alignment of the depth map. For example, the edge strength is computed as the norm of the gradient of the image. The difference-to -weight function is adapted according to the norm of the gradient, for example:

-ag

^!P -^J _q

WL = 2 wherein α is a constant value and g is a properly scaled value representing the norm of the gradient.

Fig. 4 A illustrates a depth map filtered without using the edge strength (g = 1 in the above equation). Fig. 4B illustrates a depth map filtered using the edge strength g in the above equation. Note the sharper edges when the edge strength is taken into account.

Combining data from different sources to obtain a single result is a ubiquitous problem in computer science. Consider for example the problem of selecting a pixel value

(be it luminance, color and/or depth) out of multiple different candidates (the candidate pixels together forming a window), and assume the existence of a comparison function to rank each candidate (for example a Sum of Absolute or Squared Differences). The selection process typically falls under one of the following two categories: Choose the best candidate: This method may be used to pick the motion vector for a block given all the candidates. The procedure in the case of multiple candidates with the best result is not specified. This method typically suffers from the problem of falling into local minima, which is particularly problematic if the process is being repeated in an iterative fashion.

Weighted averaging: Often, in weighted averaging a negative exponential function is used to determine the weight of each candidate according to its comparison result. The best results are given the most important contribution to the final result, and highly deviating results are ignored or contribute very faintly. Weighted averaging may overcome the problem found in 'choose the best candidate' by averaging the multiple candidates, but it introduces the problem of determining an adequate level of balance between the weight attribution. This problem may be overcome by adaptively determining the weighing function using a local control parameter L, for example according to the formula

This local control parameter L may be based on the quality of the results being compared. Setting the local control parameter to very low values might cause error propagation due to a single poor choice, but high values might introduce blurring which can adversely affect the result. According to an adaptive technique for automatic adjustment, the control parameter relies on the range of the comparison results. The control parameter may be directly derived from the quality of the candidates.

Fig. 3 illustrates how the weight-to-difference function 104 defines a mapping from the difference / -J ^r (indicated by d in the graph of Fig. 3) to the weight value w, under control of the control parameter value L, as defined by the above formula.

Several techniques can be used to provide a good measure of the quality of the candidates, but, given the importance of the best candidate in this embodiment, it is possible to use exclusively the best candidate's result as guideline. Thus if the best candidate's comparison result is particularly good, L would be set low, while if not even the best candidate's result is good then L is would be increased. This can be achieved simply by using the best result itself as value of L, merely adjusting it to fit the adequate expected value range.

The following equation shows a generalization of the above equation supporting different upper (worst) and lower (best) bounds:

-a¹ ^ (worst-best) -best+ w = 2 ¹P ^~J _q Here, C is a constant and "worst-best" is the difference between two image element values, one being graded the worst quality and one being graded the best quality. "Worst" and "best" represent local control parameters.

Alternatively, using β as a sort of reference of good match, the following equation can be used:

-a β ^p-b ^De^es^st^t (.worst-best) -best÷ / -J w_p = 2

The parameter α was introduced to make the function exponential over the mixing factor (which might be desirable since generally the best match can be a small value and also with small variance over multiple selections). It will be appreciated that the invention also extends to computer programs, particularly computer programs on or in a carrier, adapted for putting the invention into practice. The program may be in the form of source code, object code, a code intermediate source and object code such as partially compiled form, or in any other form suitable for use in the implementation of the method according to the invention. It will also be appreciated that such a program may have many different architectural designs. For example, a program code implementing the functionality of the method or system according to the invention may be subdivided into one or more subroutines. Many different ways to distribute the functionality among these subroutines will be apparent to the skilled person. The subroutines may be stored together in one executable file to form a self-contained program. Such an executable file may comprise computer executable instructions, for example processor instructions and/or interpreter instructions (e.g. Java interpreter instructions). Alternatively, one or more or all of the subroutines may be stored in at least one external library file and linked with a main program either statically or dynamically, e.g. at run-time. The main program contains at least one call to at least one of the subroutines. Also, the subroutines may comprise function calls to each other. An embodiment relating to a computer program product comprises computer executable instructions corresponding to each of the processing steps of at least one of the methods set forth. These instructions may be subdivided into subroutines and/or be stored in one or more files that may be linked statically or dynamically. Another embodiment relating to a computer program product comprises computer executable instructions corresponding to each of the means of at least one of the systems and/or products set forth. These instructions may be subdivided into subroutines and/or be stored in one or more files that may be linked statically or dynamically. The carrier of a computer program may be any entity or device capable of carrying the program. For example, the carrier may include a storage medium, such as a ROM, for example a CD ROM or a semiconductor ROM, a flash memory, or a magnetic recording medium, for example a floppy disc or hard disk. Further the carrier may be a transmissible carrier such as an electrical or optical signal, which may be conveyed via electrical or optical cable or by radio or other means. When the program is embodied in such a signal, the carrier may be constituted by such cable or other device or means. Alternatively, the carrier may be an integrated circuit in which the program is embedded, the integrated circuit being adapted for performing, or for use in the performance of, the relevant method. It should be noted that the above-mentioned embodiments illustrate rather than limit the invention, and that those skilled in the art will be able to design many alternative embodiments without departing from the scope of the appended claims. In the claims, any reference signs placed between parentheses shall not be construed as limiting the claim. Use of the verb "comprise" and its conjugations does not exclude the presence of elements or steps other than those stated in a claim. The article "a" or "an" preceding an element does not exclude the presence of a plurality of such elements. The invention may be implemented by means of hardware comprising several distinct elements, and by means of a suitably programmed computer. In the device claim enumerating several means, several of these means may be embodied by one and the same item of hardware. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.

Claims

CLAIMS:

1. A system for applying a bilateral filter to an image comprising a plurality of elements, each element having at least one element value associated therewith, comprising an input (101) for receiving respective local control parameter values (109) in respect of respective center elements of respective windows of an image and indicative of a difference-to -weight function (104); and a weight generator (102) for establishing a plurality of respective weights (112) of a plurality of respective window elements of one of the windows (502) by computing a respective weight of a respective window element (504) by applying the difference-to -weight function (104) to a difference between an element value (110) of the center element (503) of the one of the windows (502) and a respective element value (111) of the respective window element (504), the weight generator (102) comprising means (103) for determining the difference-to-weight function (104) based on the local control parameter value (109) in respect of the center element (503).

2. The system according to claim 1, wherein the local control parameter value

(109) is indicative of a steepness of the difference-to -weight function.

3. The system according to claim 1, further comprising a reliability estimator for establishing respective local reliability values in respect of the respective center elements of the respective windows, and establishing the respective local control parameter values based on the respective local reliability values.

4. The system according to claim 3, wherein the difference-to-weight function defines a decreasing dependence of the weight on the difference, and wherein the reliability estimator is arranged for establishing a local control parameter value corresponding to a less steeply decreasing dependence of the weight on the difference in response to a smaller local reliability value.

5. The system according to claim 1, further comprising a local contrast estimator for establishing respective local contrast or edginess values in respect of the respective center elements of the respective windows, and establishing the respective local control parameter values based on the respective local contrast or edginess values.

6. The system according to claim 1, wherein the difference-to-weight function comprises a term b^ad , wherein b is a base number, α is the local control parameter, and d is the difference.

7. The system according to claim 1, wherein the bilateral filter is defined by:

BF[I,J,K,L]_p = ^~∑w_qK_q , wherein

BF[-]_p denotes an outcome of the bilateral filter at an element p, w_q denotes a weight w_q = G_s (\\p - q\) G₁, [∑_p , \\l_p - J_q |) , Wp denotes a normalization factor W_p = V w_q , qeS p and q denote element indices,

I_p denotes an element value at element p in a first image,

J _q denotes an element value at element q in a second image, K_q denotes an element value at element q in a third image, wherein two or all of these images may be the same image, L_p denotes the local control parameter value in respect of a center element p,

G_s denotes a spatial function, and

G_r denotes a range function whose output value corresponding to an input value / ~JJ is controlled by the control parameter value L .

8. The system according to claim 7, wherein the first image and the second image are different images in a temporal image sequence, and wherein the third image is a depth image corresponding to the first or the second image.

9. The system according to claim 7, wherein

G₁. (L_P , \\l_p

= exp (-aL_p \\l_p , for a given value α .

10. The system according to claim 7, further comprising means for determining the spatial function G_s based on a local control parameter value.

11. A video data processing apparatus comprising the system according to claim 1.

12. A method of applying a bilateral filter to an image comprising a plurality of elements, each element having at least one element value associated therewith, comprising receiving (204) respective local control parameter values in respect of respective center elements of respective windows of an image and indicative of a difference- to -weight function; and establishing (208) a plurality of respective weights of a plurality of respective window elements of one of the windows by computing a respective weight of a respective window element by applying the difference-to -weight function to a difference between an element value of the center element of the one of the windows and a respective element value of the respective window element, the difference-to -weight function being controlled by the local control parameter value in respect of the center element.

13. A computer program product comprising instructions for causing a processor system to perform the method according to claim 12.