WO2005017831A2

WO2005017831A2 - Processing a noisy image

Info

Publication number: WO2005017831A2
Application number: PCT/GB2004/003222
Authority: WO
Inventors: Guy Philip Nason; Piotr Zbigniew Fryzlewicz
Original assignee: University Of Bristol
Priority date: 2003-08-07
Filing date: 2004-07-26
Publication date: 2005-02-24
Also published as: WO2005017831A3; GB0318543D0

Abstract

In order to process an noisy or otherwise unclear image to extract useful information, the image is converted to a digital signal (10) and then two further conversion operations are carried out. The first uses a two dimensional Haar wavelet transform (11) and the second is a transform which takes the Poisson noise to normality, such as a Fisz transform (12). The resulting signal can be used directly, but is preferably further processed using e.g. an inverse Haar wavelet transform (13). It may be processed further by a Haar transform (15) and an inverse Fisz transform (16) at possibly an additional inverse Haar transform (17).

Description

IMAGE PROCESSING

Background of the Invention Field of the Invention The present invention relates to image processing- It is particularly concerned with image processing for processing a noisy or otherwise unclear image to extract from it useful information.

Summary of the Prior Art Many different image processing arrangements are known for processing images, where the information contained within the image is unclear due to e.g. noise. Many different mathematical techniques have been applied to such image processing, and the techniques applied may depend on the aim of the processing, e.g. whether to filter the noise, compress or decompress the image, etc. One known technique for processing images is to make use of the mathematical process known as the Haar transform. The Haar transform is discussed in e.g. the article "A Theory for Multi-Resolution Signal Decomposition: The Wavelet Representation" by S.G. Mallat published in IEEE Transactions on Patent Analysis and Machine Intelligence, Volume 11, Pages 674 to 693 of

1989. Other examples of signal processing arrangements involving the Haar transform appear in EP 0561593, EP 0987882 and EP 1074271. The two-dimensional Haar wavelet transform can be considered to operate in the following way. First, we define two filters: the Haar smoothing filter

and the Haar detail filter. Let {x₍.}"₌.be an input sequence

of even length (sequences of odd length can also be processed with minor changes) . Then the low pass filter H is conventionally defined by c = Ex as follows:

for i = 1, . . . , n/2 . The high pass filter, G, has the same formula as the low pass filter except the + is replaced by a - and conventionally the output sequence is denoted by d_± not c_± . The formula in (1) is a filter with filter coefficients (1/2, 1/2). Notice that the length of the output sequence is half the length of the input sequence. As an example, suppose that the input sequence x_± for i = 1, . . . , 3 is as follows :

4 6 19 16 11 5 3 10 Then the ID smooth coefficients, c_±, after applying H are :

5 17.5 8 6.5

and the ID detail coefficients, d_±, after applying G are:

1 -1.5 -3 3.5

The inverse transform from e and di back to x are of exactly the same form as (1) but with filter coefficients of ( 1 , 1) . For example, x_2j = C_j + d_j (2)

and indeed 5 + 1 = 6, 17.5 + (-1.5) = 16, 8 + (-3) = 5 and 6.5 + 3.5 = 10 which recovers the even members of the original sequence. The odd members are recovered by the formula ₂₇-l =Cj-dj . Formula (2) is merely a filter with coefficients (1,1) .

Other pairs of sets of filter coefficients may be used and are referred to as Haar coefficients but we prefer to use the pair (0.5, 0.5) and (1, 1) here for our purposes described below. The 2D Haar wavelet transform is performed using successive ID Haar wavelet transforms in the following way.

The first step is a Transform Step. The input to the 2D Haar wavelet transform is a matrix (e.g. image, video frame) . First, the H filter is applied to all of the rows of the image and the results are kept in another matrix which we call C. Then the G filter is applied to all of the rows of the image and the results kept in another matrix which we call D. Then both the H and G filters are applied to the columns of both C and D producing another 4 matrices which we call CC, CD, DC and DD. This process is illustrated in Figure 1. The CC matrix is the result of using the smoothing filter H in the horizontal and vertical directions are called the smooth (father) coefficients. The CD, { DC) coefficients are the results of horizontal (vertical) smoothing and vertical (horizontal) detail extraction. The DD coefficients are the result of horizontal and vertical detail extraction which results in "diagonal" detail extraction. The coefficients produced at this first step are "finest scale" coefficients. The new matrices have dimensions which are half those of the input image. Typically, all of the calculations can be carried out in-place" thus although we have described the creation of new matrices C,

D_r CC, CD _r DC and DD they do not have to be physically created. The "in-place" calculation greatly enhances the memory efficiency of the Haar wavelet transform as no new storage needs to be supplied for its operation.

Next, the Transform Step is applied to CC as the new input image producing four new submatrices CC_{l r} CD_\, DC and DO_\ which are the smooth, horizontal, vertical and diagonal detail at the next coarsest scale.

Then, the Transform Step is applied recursively to CCi to produce i₊i, CD_±+i , DC_±+i, DD_i÷1 which are the smooth, horizontal, vertical and diagonal details at scale i+1.

The full transform consists of recursively applying the transform step until the dimension of the produced wavelet coefficients is 1 x 1. The set of all produced wavelet coefficients CD ,DC _f DD\ , CD2 , DC2 _f DD₂ ,

. . . up to the finally produced coefficients is called the discrete two-dimensional Haar wavelet transform of the original image. The coefficients resulting from the full transform for the image given in Figure 1 are shown in Figure 2. In Figure 2, the whole image is composed of a number of subimages separated by dotted lines. The three largest subimages correspond to DC, CD and DD, the next three largest subimages correspond to DCi, CD₂ and DD₂ and so

on. Any coefficient can be indexed as follows: d^* ,_ki _{s k2}

where £ can be any one of horizontal, vertical or diagonal; j corresponds to the scale of the coefficients and [ kχ _f k.₂ ) to the coordinates of the coefficient within the direction-scale matrix.

Summary of the Invention The present invention proposes to make use such a Haar transform, but adds a further transform to its output. At its most general, the present invention proposes that the further transform thus applied to the output of the Haar transform is any one that takes Poisson noise to normality. The transform which is applied to the output of the Haar transform may be the Anscombe transform, but is preferably the Fisz transform. The Anscombe transform is e.g. is disclosed in e.g. the article entitled "The Transformation of Poisson, Binomial and Negative-Binomial Data" by F.J. Anscombe published in Biometrika, Volume 35, pages 246 to 254 of 1948. The Fisz transform is disclosed in e.g. the article entitled "The Limiting Distribution of a Function of Two Independent Random

Variables and its Statistical Application" by M. Fisz published in Colloquium Mathematicum, Volume 3, pages 138 to 146 of 1955. Thus, the present invention may provide a method of image processing comprising: converting the image to a first digital signal; converting said first digital signal to a second digital signal by a two dimensional Harr wavelet transform; and converting said second digital signal to a third digital signal by a transform which takes Poisson noise to normality. The present invention may also provide an apparatus for carrying out the method defined above. The transform in the article by Fisz referred to above shows that if χ, ₂ are Poisson distributed random variables with intensities el and e2 then the quantity

is normally distributed with mean zero and variance one. For our 2D Haar-Fisz transform if we begin with an image that contains Poisson (or near Poisson) distributed pixels then every father wavelet coefficient has the form of the denominator in formula (3) and every horizontal, vertical and diagonal wavelet coefficient has the form of the numerator in formula (3) . Our 2D Haar-Fisz transform uses the Fisz transform by dividing each horizontal, vertical and diagonal wavelet coefficient at location {kl , k2 ) by the square root of the father wavelet coefficient at { kl _f k2 ) at every scale. It is possible for the resulting transformed coefficients to be used directly for e.g. classification or other processes, e.g. compression. However, it is further possible to apply an inverse Haar transform with filter coefficients (1,1) to generate the Haar-Fisz transformed image for further processing, that further processing may be, for example, de-noising but the statistical properties of the image are now more amenable to know de-noising techniques. The distribution of the Haar-Fisz transformed image pixels is brought strongly towards normality, with constant variance and small correlation with the mean of the pixel values reflecting the intensity values of the original Poisson (or near Poisson) image. We prefer to use filters (1/2, 1/2) (forward Haar) and (1, 1) (inverse Haar) are used then the variance of the pixels of the Haar-Fisz transformed image is standardized to 1 and this is constant over the image. Any other Haar filter b [ l , 1) could be used with matching inverse filter, but then the constant variance would not be one and subsequent processing would need to be modified accordingly. It is possible for the resulting process image to be used directly for e.g. classification or other processes that require normalised coefficients at a given scale, e.g. compression. However, it is further possible to apply an inverse Haar transform to generate a final image for further processing. That further processing may be, for example, de-noising but the statistical properties of the image are now more amenable to known de-noising techniques . In a further development of the present invention, the processed signal may be transformed back to the original data domain, e.g. for processing, viewing or for comparability purposes. The Haar transform is applied to the processed signal, an inverse Fisz transform is applied, and an inverse Haar transform is applied, thereby to obtain a final image. The present invention can be applied to any digital image, such as the output of a digital camera or an analogue image which has been converted to digital signals, or any image or images generated by any device sensing at any frequency or combination of frequencies. It may be applied to still or moving images, and in the latter case the invention will be applied to each frame of the moving image. In low light conditions it is sometimes possible (in that existing technologies already do this) to obtain good images by increasing the image capture time of the imaging sensor. Our invention improves the signal to noise ratio and distributional properties of each image and thus does not require such large increases in image capture time to achieve the same improvements in image quality. This feature of our invention will, for the same image quality, permit smaller increases in image capture time and hence improve the frame rate in video capture in low light conditions. Improving the frame rate results in smoother movement and detection of otherwise possibly missed features in the image that could pass during a long capture time when not using our invention.

Brief Description of the Drawings An embodiment of the present invention will now be described in detail, by way of example, with reference to the accompanying drawings in which: Fig. 1 is a flowchart of the operation of the 2D Haar wavelet transform, and has already been discussed. Fig. 2 illustrates the effect of the two-dimensional wavelet coefficients, and has already been discussed. Fig. 3 is a schematic block diagram showing the processing steps accordance with an embodiment of the present invention. Detailed Description Referring to Fig. 3, a camera 10 generates a digital image which may be considered to be N pixels square. Typically, N is a power of two, but this is not essential. The camera 10 may be a still camera, producing a single image, or may be a video camera producing images in a series of frames, each frame then being processed in accordance with the embodiment. In subsequent discussion, a single corresponding to a frame of a video camera will be considered; the process would then be repeated for each frame. The signal from the camera 10 is passed to a first transform unit, 11, which carries out the fast two- dimensional Haar wavelet transform discussed previously using preferred filter coefficients (1/2, 1/2). This results in a set of two-dimensional Haar wavelet coefficients as described earlier. These coefficients are then passed to a second transform unit, 12, which carries out the Fisz transform by dividing every horizontal, vertical and diagonal wavelet coefficients by the square root of the equivalent father wavelet coefficient (as described earlier) unless the father is zero in which case the particular wavelet coefficient is not transformed but set to zero. This transformation is applied to all wavelet coefficients at all scale levels. The effect of the Fisz transformation is to normalize in distribution and stabilize the variance of the wavelet coefficients. The thus normalized and stabilized coefficients may then be passed to a third transform unit, 13, which applies an inverse two-dimensional Haar transform with filter coefficients (1,1) to produce the Haar-Fisz transformed image. The Haar-Fisz transformed image may then be subject to further processing, such as de-noising, 14. The statistical properties of the Haar-Fisz transformed image (output of 13) are better than those of the original image from the point of view of many processing techniques. For example, there are many more techniques known for processing normally distributed noise, and noise with constant variance than there are for Poisson distributed noise and, in general, the former techniques are more accurate and often faster. After processing it is often advantageous to return the processed Haar-Fisz transformed image to its original domain. To do this, the processed Haar- Fisz transformed image is passed to a fourth transform unit, 15, which carries out a two-dimensional discrete Haar wavelet transform with filter coefficients (1/2, 1/2) (equivalent to the action of unit 11 and the inverse of the action of unit 13) . The signal is then passed to a fifth transform unit, 16, which performs the inverse Fisz transformation (the inverse to unit 12) which remultiplies the wavelet coefficients by the square roots of the father coefficients. Finally, a sixth transform unit, 17, carries out an inverse two- dimensional Haar wavelet transform with filter coefficients (1, 1) to obtain the final image. The final image is now back on the original data scale corresponding to the output of camera 10. The point at which the image is extracted from the processing in Fig. 3 depends on the purpose of the image. It is possible to use the output of the sixth transform unit 17, the processed image information may be extracted also from the output of the second transform unit 12, or the third transform unit 13. The interest is in classifying the image, for example, the output of either the second or third transform units 12, 13 may be appropriate, whereas if the images is to be viewed, rather than further analysed, the output of the sixth transform unit 17 may be most suitable. The effect of the combination of the Haar transform and the Fisz transform causes the distribution of pixel values in a processed image to become more normally distributed, and their variants stabilised to be constant. The more normally distributed and stable is an image, the easier and better it is to enable further processing such as de-noising, edge detection, etc. to be carried out. This double transform is particularly advantageous for signals with very low intensity, since the effect of the transforms decreases as the intensity increases. Thus, low intensity features are emphasised. Moreover, the Haar and Fisz transforms are computationally efficient, having a computation order which is linear in dependence on the number of pixels. The present invention can be applies to any image enhancement situation. For example low-light CCTV operations, e.g. for security purposes, night vision, etc. may benefit from the application of the present invention, as may some consumer electronics products (e.g. night vision options on video cameras) astronomical and space-borne sensors, industrial sensors in low-light conditions, etc.

Claims

1. A method of image processing comprising: converting the image to a first digital signal; converting said first digital signal to a second digital signal by a two dimensional Haar wavelet transform; and converting said second digital signal to a third digital signal by a further transform which takes Poisson noise to normality.

2. A method according the claim 1, wherein said further transform is a Fisz transform.

3. A method according to claim 1 or claim 2, wherein said two dimensional Haar wavelet transform has filters g 1/2, 1/2.

4. A method according to any one of claims 1 to 3, wherein said third digital signal is connected to a fourth digital signal by an inverse Haar wavelet transform.

5. A method according to claim 4, wherein said inverse Haar wavelet transform has filters of 1, 1

6. A method according to claim 4 or claim 5, further comprising: converting said fourth digital signal to a fifth digital signal by a two dimensional Haar wavelet transform; converting said fifth digital signal to a sixth digital signal by an inverse Fisz transform; and converting said sixth digital signal to a seventh digital signed by an inverse Haar wavelet transform.

7. An image processing apparatus comprising: a converter for converting an image to a first digital signal; a first converter for converting said first digital signal to a second digital signal by a two dimensional

Haar wavelet transform; and a second converter for converting said second digital signal to a third digital signed by a further transform which takes Poisson noise to normality.