WO2007035144A1

WO2007035144A1 - Method and arrangement for digital filtering of images

Info

Publication number: WO2007035144A1
Application number: PCT/SE2005/001392
Authority: WO
Inventors: Mats Elfving
Original assignee: Obigo Ab
Priority date: 2005-09-22
Filing date: 2005-09-22
Publication date: 2007-03-29

Abstract

The present invention relates to a method and an arrangement for processing an input digital image. The image is filtered (42) with an at least two-dimensional filter that corresponds to a differenced version, differenced along a number of directions, of an original filter. The filtered image is then integrated (64) along this number of directions, such that a filtered image results, which correspond to the input digital image when filtered by means of the original filter. Since the differenced filter may be considerably more sparse than the original filter the number of operations required may however be considerably smaller as compared to when the original filter is used.

Description

METHOD AND ARRANGEMENT FOR DIGITAL FILTERING OF IMAGES

Technical field

The present invention relates to a method for computationally efficient processing of an input digital image. More specifically, such processing may be carried out by applying one or more digital filters to the image.

Background of the invention

US, Bl, 6,457,032 describes such a method, where a digital image signal is convolved with a function that corresponds to the difference of a one-dimensional original filter. Then the result is discretely integrated. The output of this procedure may be equivalent to processing the signal with the original filter directly. For many original filters this procedure is however more efficient, in terms of the number of calculation operations needed, than directly processing the signal with the original filter, since the difference of the original filter may contain considerably fewer non-zero elements than the original filter.

Also a two-dimensional original filter (sometimes referred to as a kernel) may be applied in the same way if this filter can be expressed as a product of two vectors, i.e. is separable. The image signal is then proces- sed by separately convolving the information therein with functions corresponding to the two filters, wherein different sets of information elements (pixels in rows or columns) are affected in the different convolving operations . This procedure may typically be used to low-pass filter an image, i.e. a blurring operation.

One problem with this approach is that it can only be carried out for two-dimensional filters that can be expressed as a product of two vectors. Thus it is not applicable at all to non-separable filters. Non-separable filters may have properties that are desired in many image processing contexts.

Summary of the invention

Thus an object of the present invention is to provide a method that allows computationally efficient applications of two- (or more) -dimensional filters that are non-separable. This object is achieved by a method as defined in claim 1. More specifically, the method then comprises the steps of: applying a digital filter to obtain a first filtered image, wherein the digital filter, if integrated along a set of directions, including at least one direc- tion which is oblique in relation to the image dimensions, corresponds to an original filter; and integrating the first filtered image along said set of directions to obtain a second filtered image, such that said second filtered image corresponds to the input digital image when filtered with the original filter. Such a digital filter can be very sparse as compared to the corresponding original filter, which means that the overall filtering process may be very computationally efficient. The set of directions may include both diagonal directions of the image, and may include four or six directions that need not be unique within the set. The set of directions may thus contain at least one duplicate direction, such that the first filtered image is integrated at least twice along this direction. The first filtered image may, before the integrating step, . be summed with one or more other filtered image.

The original filter may be a two-dimensional square filter, e.g. a pyramid filter or a diamond filter. The original filter may typically be a low-pass filter. The object of the invention is also obtained by means of a corresponding arrangement for processing an input digital image comprising a filter device for applying a digital filter to obtain a first filtered image, wherein the digital filter, if integrated along a set of directions, including at least one direction which is oblique in relation to the image dimensions, corre- sponds to an original filter; and an integrating device, integrating the first filtered image along said set of directions to obtain a second filtered image, such that said second filtered image corresponds to the input digital image when filtered with said original filter. This arrangement may be varied in correspondence with the above method.

The object of the invention is also obtained by a method for processing an input digital image, the method having the steps of applying a digital filter to obtain a first filtered image, wherein the digital filter, if integrated along a set of directions, including at least one direction, corresponds to a non-separable, at least two-dimensional original filter; and integrating the first filtered image along said set of directions to obtain a second filtered image, such that said second filtered image corresponds to the input digital image when filtered with said original filter.

Brief description of the drawings Fig 1 illustrates a filtering block scheme according to known art.

Fig 2A shows an image in numerical form. Fig 2B shows a "Pyramid3" filter in numerical form. Fig 2C shows a result in numerical form when the image of fig 2A is filtered with the "Pyramid3" filter of fig 2B.

Fig 3 schematically shows a method for computationally efficient filtering according to a preferred embodiment of the invention. Fig 4A shows, in numerical form, a sparse differenced filter equivalent of the "Pyramid3" filter of fig 2B. The sparse differenced filter is expressed in the "spread" form.

Fig 4B shows, in numerical form, the image of fig 2A after a first filtering operation according to the method of fig 3 with the filter of fig 2B being the original filter.

Fig 4C shows, in numerical form, a sparse differenced filter equivalent of the "Pyramid3" filter of Fig 2B. The sparse differenced filter is expressed in the "multi- ply and add" form.

Figs 5A- 5C illustrate the image of fig 4B after each of three integrating steps.

Fig 6 illustrates an arrangement for processing image data.

Description of preferred embodiments

The general concept of digital filtering of images is first briefly described. A digital linear filtering operation results in that a digital element (typically a pixel) of an output image signal is determined as a linear combination of some of the digital elements of an input image signal. The digital filter determines the actual linear combination.

In many cases, large 'digital filters are useful, wherein an element of the output signal is determined as a linear combination of a large number (e.g. 25 or more) of elements of the input signal. Large filters for averaging and/or blurring of signals in the form of images are useful for graphical processing, e.g. in computer games and animated videos.

In fig 1 it is schematically shown that the input to a linear filtering block 12 consists of an input image 10 and an original filter 14. The output of the linear filtering block 12 is an output image 16. In text books, a linear filtering block is often defined using a mathematical operation called convolution. Using convolution operations, it is possible to produce an explicit definition of how an arbitrary value of the output signal depends on the values of the input signal and on the digital filter used.

There are a number of per se known possible methods that perform the calculations needed in the linear filtering block 12. Generally, these methods produce mutually equivalent results.

One possible method, which will be referred to as "multiply, add and store", will now be described in more detail using a two-dimensional filtering example and referring to figs 2A-2C.

In fig 2A the inner part 20 is a simple digital input image shown in numerical form. Of course, the present invention can be used on more complex images with a plu- rality of grey-scales and color, and the filtering can affect different properties of the image. The image of fig 2A, that will be used throughout this description, is only one simple example used not to obscure the disclosure of the present invention with unnecessary details. ^" In fig 2B an even simpler digital original filter (Pyramid 3) 26 is shown in numerical form. For the filtering, the image 20 may be replaced by a zero-padded image 22, i.e. the input image 20 is provided with a surrounding frame of zeroes. Fig 2C shows the output image that results from the filtering operation. In order to determine element C232 of image 30 in fig 2C (C denoting column and 232 row) which element is the output element corresponding to input element C202 of input image 22, the following opera- tions are performed. The 3 times 3 elements of the sub- image which has the corner elements B201, D201, B203, D203 is element-wise multiplied with the original filter 26. The sum of the products thus obtained constitutes the result. For example, B201 is multiplied with B221, D201 is multiplied with D221 and C203 is multiplied with C223. The results from these 3 times 3 multiplications are then all added, equaling 0, and stored in C232. The padded zeros B201, C201, D201, B202 and B203 are then useful in order to perform the calculations without any exceptions at the border of the image. The underlining of the element C222 of filter 26 (fig 2B) means that element C222 is to be multiplied with the one element of the input image (C202 in the example) that corresponds to the actual element (C232) of the output image.

In a further example, the element G234 is determined using the sub-image which has the corner elements F203, H203, F205 and H205. The calculations/ i.e. the multiplications and the additions are ^■

0*1+0*1+1*1+0*1+1*2+0*1+1*1+0*1+0*1 = 0+0+1+0+2+0+1+0+0 = 4, which is stored in G234. By indexing all the elements of the output image and moving the original filter over the input image accordingly, all the elements of the output image, 30, may be determined using the same principle .

With the "multiply, add and store" method, n times n elements of the input image all have to be multiplied with their corresponding n times n weights of the original filter and then all these products have to be added. In short, n² multiplications and (n²-l) additions, which is approximately 2n² mathematical operations, are needed for each of the elements of the output image. When the output signal has m times m elements the total amount of calculations is 2m²n². If the output signal is intended for an animation with f frames per second, the amount will be 2fm²n² calculations per second. With f=10, m=200, and n=30 the expression 2fm²n² results in approximately 70 Gops (Giga operations per second) . This computational burden is too heavy for most modern PC CPUs (CPU=Central Processing Unit) , but in_. principle possible for modern GPUs (GPU=Graphics Processing Unit). For e.g. a cellular phone having a CPU with approximately 100 MHz of clock frequency and no GPU features to use, the burden is impossible to cope with. Seen from the view of the input image, the value of each element of the input image is "spread" to a number of elements of the output image. For example, the value of element G204 influences the element F234, which corresponds to one step to the left in the input image, using the weight D222, which corresponds to one step to the right, i.e. the element "opposite" to one step to the left with respect to the underlined element, in the original filter. In total, the element G204 may, depending on the weights, influence all the elements within a sub image which has the corners {F233, H233, F235, H235}. Generally, picking the "opposite" filter elements can be replaced by picking filter elements from a flipped version of the original filter. In the flipped version of the original filter, the original filter is flipped, in arbitrary order, around both its vertical and horizontal axis, with the axes crossing at the underlined filter element. For the original filter of fig 2B, the flipped version is identical to the original version. As another example, the filter 72 of fig 4C is the flipped version of filter 70 of fig 4A. Due to the skew symmetry of filter 70, it may seem that the only difference between filter 70 and filter 72 is that the underlined element has moved. However, that is an illusion. For example, the element B251 of filter 72 originates from the element E248 of filter 70.

Another possible method, which will be denoted "multiply, spread and accumulate", may be based on indexing all the elements of the input image. For each element of the input image, all the elements of the flipped version of the original filter are multiplied with the single indexed element resulting in a spread image. The spread image, being centered on the element of the output image that corresponds to the indexed element of the input image, is then added element wise to the results that already may be accumulated in the output image. The computed results form the "multiply, spread and accumulate" method are equivalent to those of the "multiply, add and store" method, but there are other differences between these two methods. For the "multiply, spread and accumulate" method there is no need for zero-padding of the input image. Instead, the output image, 32, will be larger than the input image, 20. More specifically, an outer frame of values is generated, that can be disregarded. The efficiency of the "multiply, spread and accumulate" method increases when the input signal has many zero elements and/or when the filter has many zero elements. Unfortunately, the number of zeros in the images may be low and filters for blurring and averaging have, due to their low-pass characteristics, usually few zero elements .

There is also the possibility of using Fourier transform methods for performing the filtering, but this is not always computationally more efficient. For animations on cellular phones using average and/or blur filtering there is thus a need for computationally more efficient filtering methods.

As will be illustrated, when two-dimensional original filters are processed with combinations of dif- ference operators, differenced filters that are sparse, i.e. have a large percentage of zero elements, may result .

These sparse differenced filters may be really useful in a method for achieving computationally efficient filtering having the behavior of a corresponding original filter, which is not sparse at all. As mentioned, US, Bl, 6,457,032 discloses a method where one-dimensional differenced filters are used and where the result of the filtering is integrated. This method may be applied with two-dimensional original filters, but only when such filters are separable, i.e. can be expressed as a product of two vectors . In contrast, an approach in accordance with an embodiment of the invention can be used on a non-separable two-dimensional filter. A method according to this approach will now, with reference to fig 3, be described on a high level. The input image 40 is one of the two inputs to a linear filtering block 42. Its other input is a sparse differenced filter 54, which is derived from an original filter 44 using difference operations 48, that may depend on a difference scheme 50. The difference scheme comprises a set of directions along which the original filter should be differentiated and, correspondingly, along which the filtered image should be integrated. The differentiation scheme may optionally also contain information regarding the relative order of the differentiation operations.

The output of the linear filtering block 42 is a sparse filtered output 56. Since there are very few nonzero elements in the sparse differenced filter, the "spread step" is efficiently performed and it only has to be carried out for the non-zero elements of the input image. In that way, a lot of multiplications, additions and possibly also memory accesses to the input image may be saved, as compared to known methods.

If a filtering corresponding to the sum of the re-^' suits of several different filters is desired, this is possible to do in an efficient way. The sparse filtered output 56 may, be added 58 to other sparse filtered outputs 50, that are to be included in the same output image 46, and is integrated at 64, using the same inte- gration scheme, 52, i.e. along the same set of directions. Thereby the computations connected with the integration operations are reduced.

The integration operations 64, may be performed in an arbitrary order and may be performed in a coordinated way, which will be described below. The integration operations are based on cumulative additions and will be described in more detail below. The integration operations are used to compensate for the differencing effects that the sparse differenced filter has on the input image. Linear filtering, taking differences and adding all are linear operations and the individual order for these may therefore be altered.

Since the sparse difference filters only depend on the corresponding original filters and the special combination of difference operators, i.e. the difference scheme, and not on the input images, the sparse differ- ence filters do not have to be calculated each time the method is used. Instead, they may be stored in a memory or be a part of a computer program implementing the method.

For large original filters there may occur overflow during integrations performed with integer arithmetic.

This problem would however also occur for the traditional filtering already during the application of the large original filter. In both cases, these problems may be cured by using floating point arithmetic or a larger integer type.

The method according to the invention performs linear filtering with sparse filters and with discrete integration operations that, when performed in an appropriate sequence, have an effect on the filtered image that is equivalent to linear filtering of the same image with the original filter.

The main advantage of the new method is that linear filtering with large filters can be achieved with a significantly reduced total computational burden. The integration operations can be made using addition operations only, i.e. no multiplications are needed for the integration operations.

GENERATING SPARSE FILTERS A first example is described using an original filter called "Pyramid3":

A pyramid filter is generally a two-dimensional filter with an odd number of rows and columns. The center element has the highest value. A quadratic frame of elements, surrounding the center element, have substantially uniform values, lower than the value of the center element. This frame of elements may in turn be surrounded by an outer quadratic frame of elements having even lower values, and so forth. The underlined center element above corresponds to the underlined element C222 of filter 26 in fig 2B.

After taking row-wise differences in the direction from left to right (assuming zeros to the left and right of the original filter) the result is:

E . g . the second row is [ 0 (+1 ) 1 (+1 ) 2 ( -1 ) 1 ( -1 ) 0] , where the differences between adj acent elements , from left to right , are shown within parentheses . The resulting matrix has an extra column to the right .

After taking, in a similar fashion, column-wise differences from up to down the result is :

The resulting matrix has an extra row at the bottom. After differences from upper left to lower right, i.e. in a diagonal direction, the result is: 1 0 0 -1 0

0 -1 0 1

0 0 1 0

-1 0 1 0 0

0 1 0 0 -1

The resulting matrix has one extra column to the right and one extra row at the bottom.

After differences from upper right to lower left, i.e. in the other diagonal direction, the result is:

0 1 0 0 -1 0

-1 0 _g 0 0 1

0 0 0 0 0 0

1 0 0 0 0

0 -1 0 0 1 0

The resulting matrix, which is a sparse differenced filter, has, compared to the original filter, one extra column to the left, two extra columns to the right and three extra rows at the bottom. The used set of directions in this case comprises: (1) horizontally left to right, (2) vertically top to bottom, (3) diagonally upper left to lower right, and (4) diagonally upper right to lower left. The resulting matrix is, due to the positioning of the non-zero elements in a ring-shaped formation, denoted "Ring 6" below. An interesting feature of the "Ring" sparse filter is that all its weights are one or minus one, which means that it can operate on an image using only additions and subtractions, since there is no need for multiplication with the absolute value of the filter weights . For "Ring 6" each row, column and diagonal has a sum of 0. This results from difference operations that have operated along each row, column and diagonal.

As a second example, an original filter "Pyramid 5": 1 1 1 1 1

1 2 2 2 1 1 2 _3 2 1 1 2 2 2 1 1 1 1 1 1

will after taking differences in the direction from left to right (assuming zeros outside the original filter) become:

1 0 0 0 0 -1

1 1 0 0 -1 -1

1 1 1 -1 -1 -1

1 1 0 0 -1 _-_|

1 0 0 0 0

After taking differences from up to down the result will be:

1 0 0 0 0 _ -ι

0 1 0 0 -1 0

0 0 1 -1 0 0

0 0 -1 1 0 0

0 -1 0 0 1 0

-1 0 0 0 0 1

After taking differences diagonally from upper left to lower right the result will be: 1 0 0 0 0 -1 0

0 0 0 0 -1 0 1

0 0 ID -1 0 1 0

0 0 0 1 0 0

0 -1 0 1 0 0 0

1 0 1 0 0 0 0

0 1 0 0 0 0 -1

And, finally, after taking differences diagonally from upper right to lower left the result will be:

0 1 0 0 0 0 -1 0

-1 0 0 0 0 0 0 1

0 0 0 JD 0 0 0 0

0 0 0 0 0 0 0 0

1 0 0 0 0 0 0

0 -1 0 0 0 0 1 0

The final result has the same structure as the "Ring 6" filter at the corners, but contains two additional rows and two additional columns of zeros. Below, it is denoted "Ring 8". The "Ring 8" filter also has weights that are one or minus one as well as rows, columns and diagonals that have a sum of 0. The set of directions in this case is the same as in the previous example.

In fact, all Pyramid filters of the above shown type will, independent of their order, result in Ring filters. The general Ring Filter may be expressed using a Block Matrix: 0 1 0 . . . 0 -1 0

-1 0 0 . . . 0 0 1

0 0

0 . . . 0

1 0 0 . . . 0 0 -1 0 -1 0 . . . 0 1 0 ,

where the corners are the same but where the number of zeros between the corners depend on the size of the original pyramid filter.

FILTERING

"Pyramid 3" will be used as an example even though in this case the total computational burden will not be reduced using the inventive method. Although the original 9 non-zero filter weights of "Pyramid 3" have been replaced with only 8 non-zero filter weights of "Ring 6", the four integration operations that will be needed after the linear filtering with "Ring 6" will add a computational burden that most likely will be larger than the gain during filtering. For "Pyramid 5" however the total computational burden is most likely reduced. The original "Pyramid 5" filter has 25 non-zero weights, and the corresponding "Ring 8" has only 8 non-zeros weights.

For larger filters than "Pyramid 5", the reduction of computational burden is even more significant. A "Pyramid 13" filter has 13x13=169 non-zero filter weights, but the corresponding "Ring 16" filter still has only 8 non-zero filter weights. The filtering operations will thus be reduced with a factor 169/8, which is app- roximately 21, which leaves room for quite a lot of integration operations, still giving a considerable reduction of the total computational burden. In general a "Pyramid n" will result in a "Ring n+3" and the reduction of the filtering operations, except from the integration operations, will be a factor n²/8.

In the following, the invention is described using filters that are applied according to the "multiply, spread and accumulate" method, but this is not necessary for the new method to work. Multiply, add and store would also be functional in this context.

Even though the "Pyramid 3" is an example where the present invention, as mentioned, does not reduce the computational burden to any considerable extent, it will be used as an example in the following for the sake of conciseness. The skilled person is then capable of using a larger filter correspondingly. Fig 4A shows the "Ringβ" filter that was derived above. In fig 4C it is shown in a flipped version, as was explained earlier. Fig 4A and 4C may seem identical, but the centering element is situated differently between the two figures. Therefore, for "Ringβ" flipping has a meaning.

Fig 4B shows the resulting sparse filtered output, 82, from "spreading" the image 20 of fig 2A with the filter "Ringβ" of fig 4A. Element G202 of the input image 20 of fig 2A is multiplied with the filter 70 of fig 4A. The result is identical to the filter 70. The non-zero results are then spread and accumulated to the output 82 of fig 4B. The same procedure is repeated for all the non-zero elements of image 20. For elements E262, F261 and 1261 of the output 82, only one non-zero contribution is given and that is from element G202. Element J262 of the output 82, gets one additional contribution from input image element H203 through weight E243. For element J262 the total accumulated is therefore G202*l+H203* (- 1) =1*1+1* (-1) =0. The sums of each row, column and dia- gonal of the sparse filtered output are also all zero. INTEGRATION

The integration operations according to the integration scheme that corresponds to the difference scheme of "Ringβ" will now be described in more detail. The set of directions is the same as was used when "Ringβ" was obtained from "Pyramid3".

Although the individual order of the integration operations may be arbitrary, that fact will not be proved. The integration operations will be shown in one particular order only. That order is the reversed order in which the "Ring" filter was derived.

In the first step, image 82 of fig 4B is integrated diagonal-wise along the upper right to lower down direc- tion. Element 0287 of fig 5A is determined to be the value of element 0267 of fig 4B. Thus, element 0287 will be 1. The next element, which is N288, is determined as the sum of the previous element, which was 0287, and the N268 element. Thus element N288 will be 1+1 = 2. The whole diagonal from 0287 down to H294 will be determined by integrating, i.e. cumulatively summing, the diagonal from 0257 down to H274. Element G275 is not needed, but may be used for checking purposes, since its value is the last integrated value multiplied by minus one. The latter diagonal is, except from G275, {1,1,0,0,-1,-1,0,0,0} and the cumulative sum is {1,2,2,2,1,1,0,0,0}. The cumulative sum is found in the diagonal from 0287 down to H294 of 92 in fig 5A. When all the diagonals from upper right to lower left have been integrated, 92 of fig 5A is com- pleted.

In the second step, image 92 of fig 5A, is integrated diagonal wise along the upper left to the lower down direction. The result is shown in 102 of fig 5B. In the third step, image 102 of fig 5B, is inte- grated along the up to down direction and the result is shown in 112 of fig 5C. In the fourth, and for this integration scheme final, step, image 112 of fig 5C is integrated along the left to right direction and the result is equivalent to 32 of fig 2C. As mentioned, the digital image processing method according to one aspect of the invention, as well as the corresponding arrangement, use the step of integrating an image along a direction which is oblique in relation to the image dimensions. The meaning of oblique in this context is that the image has rows and column and that the integration procedure creates sums of elements that reside in mutually different columns and mutually different rows. That is, when integration along an oblique direction in an image takes place, each cumulative sum is created not just by adding element values in one column or in one row.

The same applies correspondingly when establishing a sparse filter from an .original filter by differentiating the filter along an oblique direction. In the examples above, the oblique integrating directions are the two diagonals, i.e. the oblique integrating directions correspond to the way a bishop is allowed to move over a chessboard. For a square image and/or filter this is equivalent to the direction between two antipodal corners. However, as will be shown later, other oblique directions are possible.

The method illustrated above may also be used without integrating along oblique directions. I.e. it is possible to differentiate a non-separable filter from top to bottom and from left to right, and achieve a more sparse filter. An image filtered with this filter is then integrated from top to bottom and from left to right.

FURTHER EXAMPLES The above mentioned aspect of the invention, using oblique integration directions may be applied on a wide range of original filters apart from the above mentioned Pyramid-filters .

A first additional example is the original filter "Diamond 5":

0 0 1 0 0

0 1 2 1 0

1 2 !3 2 1

0 1 2 1 0

0 0 1 0 0

Generally, a diamond filter corresponds to a pyramid filter, rotated 45°.

After differentiating from left to right, from up to down, from upper right to lower left, and from upper left to lower right the sparse result is:

0 0 0 1 0 0 0

0 0 0 1 -1 0 0 0

0 0 o^' O- 0 0 0 0

-1 -1 0 0 0 0 1 1

1 1 0 0 0 0 -1 -1

0 0 0 0 0 0 0 0

0 0 0 -1 1 0 0 0

0 0 0 -1 1 0 0 o ,

This filter may be called "Windmill 8" due to the positioning of the non-zero elements. The differencing scheme is the same as used before on Pyramid 5. An example of a non .quadratic original filter is:

1 1 1 1 1 1 1 1 2_ 2 1 1 1 1 1 1 1 1

which corresponds to Pyramid 3 where all columns have been duplicated. This filter will be used as an example where an oblique direction does not imply a diagonal direction corresponding to the movements of a bishop on a chessboard.

Differencing this filter from left to right and top down gives:

1 0 0 0 0 0 -1

0 0 1 0 -1 0 0

0 0 -1 0 1 0 0

-1 0 0 0 0 0 1

This matrix is padded with a double frame of zeroes, and the differences are taken in traces along an oblique direction from top left to lower right, such that each element is compared with an element in the adjacent lower row two columns away. E.g. the bold elements above form a path, whose elements provide a corresponding resulting path in the matrix below, where the corresponding elements are also written with bold characters.

1 0 0 0 0 0 -1 0 0

0 0 0 0 -1 0 0 0 1

0 0 -1 0 0 0 1 0 0

-1 0 0 0 1 0 0 0 0

0 0 1 0 0 0 0 0 -1

A corresponding differentiation from top right to lower left gives:

0 0 1 0 0 0 0 0 -1 0 0

1 0 0 £ 0 0 0 0 0 0 1

0 0 0 0 0 0 0 0 0 0 0

1 0 0 0 0 0 0 0 0 0 -1

0 0 -1 0 0 0 0 0 1 0 0 This matrix corresponds to Ring 6 where the columns have been interleaved with zero-columns.

By performing additional integrating functions further interesting effects may be obtained. A filter called "Quad Diamonds 11" may be designed as:

[ (Diamond 5) 0 -(Diamond 5) 0 0 0 0 0 ^ 0 0 0 0 0 -(Diamond 5) 0 (Diamond 5) ]

or explicitly:

0 0 1 0 0 0 0 0 -1 0 0

0 1 2 1 0 0 0 -1 -2 -1 0

1 2 3 2 1 0 -1 -2 -3 -2 -1

0 1 2 1 0 0 0 -1 -2 -1 0

0 0 1 0 0 0 0 0 -1 0 0

0 0 0 0 0 £ 0 0 0 0 0

0 0 -1 0 0 0 0 0 1 0 0

0 -1 -2 -1 0 0 0 1 2 1 0

1 —9 -3 -2 -1 0 1 2 3 2 1

0 -1 -2 -1 0 0 0 1 2 1 0

0 0 -1 0 0 0 0 0 1 0 0

If this filter is integrated from top to bottom and from the left to the right it becomes:

0 0 1 1 1 1 1 1 0 0

0 1 4 5 5 5 5 4 1 0

1 4 10 13 14 14 13 10 4 1

1 5 13 17 18 18 17 13 5 1

1 5 14 18 19 19 18 14 5 1

1 5 14 18 19 1⁹ 18 14 5 1

1 5 13 17 18 18 17 13 5 1

1 4 10 13 14 14 13 10 4 1

0 1 4 5 5 5 5 4 1 0

0 0 1 1 1 1 1 1 0 0 This filter is a good approximation of a Gauss filter. If "Quad diamond 11" is instead differenced from left to right, from up to down, from upper right to lower left, and from upper left to lower right the resulting filter will be:

0 0 0 1 -1 0 0 0 0 -1 1 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0

-1 -1 0 0 0 0 2 2 0 0 0 0 -1 -1

1 1 0 0 0 0 -2 -2 0 0 0 0 1 1

0 0 0 0 0 0 _0 0 0 0 0 0 0 0

0 0 0 -2 2 0 0 0 0 2 -2 0 0 0

0 0 0 0 0 0 0 0 0 0 0 0 0 0

1 1 0 0 0 0 -2 -2 0 0 0 0 1 1

-1 -1 0 0 0 0 2 2 0 0 0 0 -1 -1

0 0 0 0 0 0 0 0 0 0 0 0 0 0

0 0 0 1 -1 0 0 0 0 -1 1 0 0 0

0 0 0 1 -1 0 0 0 0 -1 1 0 0 ^■ o

By applying this filter, and integrating the resulting image six times: e.g. twice from left to right, twice from up to down, from upper right to lower left, and from upper left to lower right, the result will be roughly equivalent to the application of a much less sparse approximate Gauss filter. Note that the set of directions in this case contains duplicates of two direc- tions, and thus not six unique directions.

FUNCTIONAL BLOCKS

Fig 6 illustrates an arrangement for processing image data. The functional blocks of this arrangement comprise a sparse filter 601a corresponding to a differenced version of an original filter. The sparse filter 601a is applied to an input image signal. It is possible to apply this signal to more than one such filter 601b, etc. and to perform a summing operation using adders 603, etc. The adders then add the values of the filtered outputs pixel-wise. The differentiation schemes resulting in the sparse filters 601a, 601b, etc. should be equivalent.

The, optionally summed, result of the filtering operation (~s) is fed to an integrator block 605, performing the prescribed integrating operations. The image output from the integrator block may be normalized by a normalizer block 607 in order to keep e.g. an average luminance value at the level of the input image.

The functional blocks could be realized by a CPU and software implementation or can be carried out by means of dedicated hardware such as an ASIC or an FPGA, as is realized by the skilled person. Operations may also be performed in parallel, like for example in a Single Instruction Multiple Data, SIMD, capable processor.

If a CPU with small cache is used, and the memory access is slow it might be efficient to coordinate the integration processes. Then the image is run through (looped) a number of times that is smaller than the number of directions in the set of directions. A number of integrations are carried out on e.g. the same row before moving on to the next row, such that the likelihood of having the relevant data for each operation in the CPU cache increases. The number of computational operations does not decrease, except if smart use of parallel pipelines or the like can be used, but the number of cache misses and, correspondingly, the number of CPU wait states per image decreases.

Note that a still substantial reduction of the computational burden can be achieved with a method using a filter corresponding only to one, two, or three levels of differentiation of an original filter. The directions and the order of the individual differences may be varied, as long as the directions of the integration operations are varied correspondingly.

APPLICATIONS

5 In addition to filtering in animations, e.g. blurring of images, the method may be useful for interpolation and filling the holes that arise when images are enlarged.

The present invention is not restricted to the em- 10. bodiments described above. It can be realized in different ways within the scope of the appended claims.

Claims

1. Method for processing an input digital image, the method comprising the following steps:

-applying (42) a digital filter to obtain a first filtered image, wherein the digital filter, if integrated along a set of directions, including at least one direction which is oblique in relation to the image dimen- sions, corresponds to an original filter (44); and

-integrating (64) the first filtered image along said set of directions to obtain a second filtered image, such that said second filtered image corresponds to the input digital image when filtered with said original filter.

2. Method according to claim 1, wherein said set of directions includes both diagonal directions of the image .

3. Method according to claim 1 or 2, wherein, before said integrating step, said first filtered image is added

(58) to one or more other filtered images.

4. Method according to any of the preceding claims, wherein said set of direction include at least four directions .

5. Method according to any of the preceding claims, wherein said set of direction include at least six directions .

6. Method according to any of the preceding claims, wherein said set of directions contains at least one duplicate direction, such that the first filtered image is integrated at least twice along this direction.

7. Method according to any of the preceding claims, wherein said original filter is a two-dimensional square filter.

8. Method according to claim 7, wherein said original filter is a pyramid filter.

9. Method according to claim 7, wherein said original filter is a diamond filter.

10. Method according to any of the preceding claims, wherein said original filter is a low-pass filter.

11. An arrangement for processing an input digital image comprising:

-a filter device (601a) for applying a digital filter to obtain a first filtered image, wherein the digital filter, if integrated along a set of directions, including at least one direction which is oblique in relation to the image dimensions, corresponds to an original filter; and

-an integrator device (605) for integrating the first filtered image along said set of directions to obtain a second filtered image, such that said second filtered image corresponds to the input digital image when filtered with said original filter.

12. A method for processing an input digital image, the method comprising the following steps: -applying a digital filter to obtain a first filtered image, wherein the digital filter, if integrated along a set of directions, including at least one direction, corresponds to a non-separable, at least two- dimensional original filter; and -integrating the first filtered image along said set of directions to obtain a second filtered image, such that said second filtered image corresponds to the input digital image when filtered with said original filter.