AU766343B2

AU766343B2 - A method and apparatus for detecting focus of a digital image

Info

Publication number: AU766343B2
Application number: AU88382/98A
Authority: AU
Inventors: Igor Kharitonenko; Xing Zhang
Original assignee: Motorola Australia Pty Ltd
Current assignee: Motorola Solutions Australia Pty Ltd
Priority date: 1998-10-09
Filing date: 1998-10-09
Publication date: 2003-10-16
Anticipated expiration: 2018-10-09
Also published as: AU8838298A

Description

I -1- A Method and Apparatus for Detecting Focus of a Digital Image Field of the Invention This invention relates to a method and apparatus for detecting focus of a digital image, especially, though not exclusively, of a digital image in a camera, to enable automatic control of a lens producing the image to improve the focus of the image.

Background of the Invention For digital still and video cameras, autofocusing is a known technique whereby the camera automatically adjusts the position of the lens in order to produce an image having the best sharpness attainable. The use S° of digital image processing techniques for focus detection is economical since So: it avoids the use of specially designed optical, infra-red or ultra-sonic focus r •detectors. However, when lighting/contrast conditions are poor, the image 15 processing techniques often fail to detect focus successfully.

A number of different techniques have been developed to try to detect S.focus using digital image processing. One such technique uses a Squared Gradient (SG) operator to detect the focus, or the degree of sharpness, in digital images. This technique is quite successful when the image data are oooo• 20 highly uncorrelated, that is adjacent or nearby pixels are quite different, due :°to sharp edges between them. However, when the image data are correlated, ~that is, the data comes from blurred or flat images with very little contrast between adjacent or nearby pixels, this SG operator will frequently fail to provide reliable measurements.

The traditional SG operator, given by: WS gj 1 )21 (1) I j uses two adjacent pixels to measure focus, were, gij are pixel values, X is a scaling parameter and M and N are the maximum values of i and j, i.e. the maximum numbers of pixels in the rows and columns of the array of pixels that are used to acquire the image. When an image is very blurred or has low contrast, adjacent pixels are often highly correlated, so that the two -2differences in equation can be quite small in value, thereby not producing a very useful result of the computation.

In this specification, including the claims, the terms "comprises", "comprising" or similar terms are intended to mean a non-exclusive inclusion, such that a method or apparatus that comprises a list of elements does not include those elements solely, but may well include other elements not listed.

Brief Summary of the Invention The present invention therefore seeks to provide a method and apparatus of detecting focus of a digital image which overcomes, or at least reduces the above-mentioned problems of the prior art.

Accordingly, in a first aspect, the invention provides a method of detecting focus of a digital image, the method comprising the steps of: ::acquiring a digital image in the form of a matrix of discrete image elements; comparing at least a first discrete image element with at least a second discrete image element at a predetermined distance from the first discrete "image element in the matrix to provide a correlation factor between the first and second discrete image elements; utilising the correlation factor to provide a focus level indication for the digital image; wherein the predetermined distance is greater than a distance between adjacent discrete image elements.

In a second aspect, the invention provides an apparatus for detecting focus of a digital image, the apparatus comprising: a matrix of discrete image elements on which a digital image can be acquired, each discrete image element providing a value indicating a characteristic of the portion of the image acquired thereby; a comparator coupled to the matrix and having a correlator to correlate at least a first discrete image element with at least a second discrete image element at a predetermined distance from the first discrete image element in the matrix so to provide a correlation factor between the first and second discrete image elements; a circuit coupled to the comparator to provide a focus level indication for the digital image -3dependent on the correlation factor; wherein the predetermined distance is greater than a distance between adjacent discrete image elements.

In a preferred embodiment, the predetermined distance is between 2 and 10 times the distance between adjacent discrete image elements, and most preferably is between 6 and 8 times the distance.

In one embodiment, the matrix is a rectangular matrix, and the second discrete image element is preferably at a predetermined diagonal distance from the first discrete image element.

In a third aspect, the invention provides a method of autofocusing a lens in a digital camera, the method comprising the steps of: detecting the focus of a first digital image according to the method of any preceding claim; detecting the focus of a second digital image according to the above described method; comparing the focus level indications for the first and second digital images; providing a lens control signal dependent on the comparison of the focus level indications; and moving a lens of the camera dependent on the lens control signal.

In a fourth aspect, the invention provides a digital camera, comprising: a lens; apparatus for detecting focus of a digital image as described above; a comparator coupled to the apparatus to provide a lens control signal dependent on a comparison of the focus level indications for a first and a second digital image; and a motor coupled to the lens and to the ~comparator so as to move the lens dependent on the lens control signal.

Brief Description of the Drawings One embodiment of the invention will now be more fully described, by way of example, with reference to the drawings, of which: FIG. 1 shows a schematic diagram of part of known apparatus used for autofocusing a lens in a camera; FIG. 2 shows a flow chart showing the sequence of operations involved in detecting focus of a digital image in one embodiment according to the present invention; FIG. 3 shows a schematic block diagram of part of an apparatus that can be used to carry out the steps illustrated in FIG. 2; and t -4- FIGs. 4 and 5 illustrate graphically a comparison of focus detection using the SG operator and the method described with reference to FIG. 2 for a relatively flat image (FIG. 4) and an image taken under low light conditions (FIG. Detailed Description of the Drawings Thus, as shown in FIG. 1, a conventional auto-focusing system 1 in a camera receives the input images at an input 2 to a focus measuring module 3. The focus measuring module 3 measures the focus of a first input image n to produce a measure W[n] of the focus of that image. This measure W[n] is then passed to a decision making module 4, where it is compared with a measure of the focus W[n-t] of a previous image n-t. This previous focus measure W[n-t] has been stored (delayed) in a delay element 5. Depending 0.0. on the result of that comparison, a signal is provided at an output 6 of the decision making module 4 to control the focus, i.e. the motor driving the lens of the camera to adjust the position of the focal plane with respect to a .o position of the image acquiring element in the camera, e.g. an array of pixels.

S"As mentioned above, in the prior art, the focus measuring module 3 utilised the SG operator to determine the focus level of the acquired image.

The present embodiment of this invention is concerned with providing an alternative operator for the focus measuring module 3 in this type of system.

In this embodiment, the image is acquired by an array of picture elements, or pixels, each of which provides an output signal indicating a value of a characteristic, such as brightness, intensity and/or colour, of the image at that pixel. The pixels in the array have a particular pitch and the position of each pixel in the array is designated by coordinates i, j in the array, with difference of 1 in the coordinate value being equal to adjacent pixels.

Thus, in the present embodiment, the most general form of the new twodimensional Diagonal Difference operator is given by: 1M N 2 _2 W2D I (gi-i-kJ-) 1 j In this form, the operator W2D basically correlates the values of a particular pixel at position j) with the values of two pixels at positions (i k, j 1 and (i x, j where the values of k, 1, x and y are positive or negative integers having a value greater than 1. In a preferred form, x 1 and y k so that the operator W2D becomes: -2D g kJ)(1g gj (3) With this version, it will be apparent that lines joining the particular pixel to each of the two pixels that it is being compared to are orthogonal. In a more preferred form, k 1 so that the lines are at 450 to the matrix of the pixels. It has been found that the best results are obtained when k and 1 are in the Srange 2 to 10 and most preferably are in the range 6 to 8.

10 It is also possible, for low complexity applications, to use a onedimensional version of the operator WrD as follows: "M 2 W1D (gij -gi-kj-) (4) g J 4 where, again, k and I are positive or negative integers having a value greater .e than 1. This one-dimensional version of the operator is less complex than the two-dimensional version and therefore requires less computation. It does still, however, provide better results than the SG operator for very blurred or low contrast images.

Thus, according to this embodiment of the invention, the focus measuring module 3 of FIG. 1 utilises the Diagonal Difference operator to determine the focus level of an acquired image. As before, that focus level is then delayed in the delay element 5 and then compared with the focus level of a subsequent image in the decision making module 4 using known decision making techniques to provide the control signal for controlling the lens motor. An example of a decision making technique is the so-called "Hill climbing" scheme described by E. Krotkov in "Focusing" published in the International Journal of Computer Vision, Vol. 1, pages 223-237 in 1987.

Turning now to FIG. 2, there is shown a flowchart illustrating the process used for determining the W 2 D operator. Thus, the process starts at step 10 by initialising the counters of i and j to zero and by setting an -6accumulator of the normalised squared differences to zero. Step 11 then requires that the value of pixel gi, is obtained and step 12 requires that the value of pixel gi-k,j- is obtained. A difference value dl is determined in step 13, where: dl 8i, gi-k,j-I and in step 14 difference value dl is squared and divided by the normalising factor X to provide value D1. In step 15, the value of pixel j-k is obtained and then difference value d2 is determined in step 16, where: d2 i, j gi+, j-k 10 before difference value d2 is squared and divided by the normalising factor X in step 17 to provide value D2. Values Dl and D2 are accumulated at step 18, together with any previously accumulated values of Dl and D2.

S. In step 19, the value of i is checked whether it has reached the maximum value M, i.e. the maximum number of pixels in a row or column of the array of pixels. If not, then i is incremented in step and the process flow continues from step 11 again. If i has reached the maximum value M, then the value of j is checked in step 21 whether it has reached the maximum value N, i.e. the maximum number of pixels in a column or row of the array of pixels. If not, 20 then j is incremented in step 22 and the value of i is reset to zero and the process flow continues from step I1 again. If j has reached the maximum value N, then the process has been completed and the accumulated values of D1 and D2 provide the focus level W2D for that image, which is passed to the decision making module 4, either directly or via the delay element Although the process of FIG. 2 can be performed either in software or in hardware, FIG. 3 shows an example of hardware 25 that can be used to carry out the process of FIG. 2. In FIG. 3, the hardware 25 has three inputs 26, 27 and 28 at which the values of pixels gi-k, j g, j and gil, j-k respectively, are applied. An adder 29 then subtracts the value at input 26 from that at input -7- 27, and adder 30 subtracts the value at input 28 from that at input 27. The output of adder 29 is provided at two inputs of multiplier 32, where they are multiplied and then passed to shifter 33 where the product is normalised.

The output of the shifter 33 is the normalised squared difference D1, which is passed to accumulator 34. Similarly, the output of adder 30 is provided at two inputs of multiplier 35, where they are multiplied and then passed to shifter 36 where the product is normalised. The output 37 of the shifter 36 is the normalised squared difference D2, which is passed to accumulator 34.

The final output W of accumulator 34 after i and j have both reached their maximum values is the final focus level W 2

D.

When the process of FIG. 2 is implemented using software, a Digital S'Signal Processor IC, such as MPC821 manufactured by Motorola Inc., can be o used. Using MPC821, images are transferred from a CCD array into a system Dynamic Random Access Memory for compression and imaging. For this operation, the 64x64 block of image data is captured during the transfer and loaded into a dual-port Random Access Memory. At the same time, the operator is loaded to operate on the data so that the focus decision can be made.

Two examples are given here to show test results obtained by this technique. The first example is an image of a cloud in the sky taken with normal lighting conditions, providing a relatively flat image. FIG. 4 shows results of the focus level, as determined by the SG operator Wsc and by the Diagonal Difference operator W 2 D for k 1 6 (DD6) for images taken at different lens positions. As it can be seen in FIG. 4, the SG operator does not provide consistent growth of WSG between lens positions 0 and 3 (between 6 and 9 too). As a result focus cannot easily be found. With the DD6 operator, however, there is no problem in finding the correct focus position (at lens position 4) because all the measured values W 2 D are have quite large differences.

The second example is an image with good contrast taken in a dark room. Again, the SG operator fails to allow the camera to easily find the correct focus position, especially if the camera is completely out of focus at -8 the beginning (frames 0, 1 and The present operator (DD6), however, enables the camera to find focus successfully (at lens position 6).

From the above two examples, it can be seen that the performance of the DD6 operator is significantly better than that of the SG operator. The DD6 operator is more robust and provides more reliable measurement both in normal, as well as in poor lighting conditions.

It will be appreciated that although only one particular embodiment of the invention has been described in detail, various modifications and improvements can be made by a person skilled in the art without departing from the scope of the present invention. For example, it will be appreciated that the image need not be acquired in a camera, but could be acquired in a S-microscope, telescope or other optical instrument.

o•

Claims

1. A method of detecting focus of a digital image, the method comprising the steps of: acquiring a digital image formed as a matrix of discrete image elements; comparing at least a first discrete image element with at least a second discrete image element at a predetermined distance from the first discrete image element in the matrix to provide a correlation factor between the first and second discrete image elements; utilising the correlation factor to provide a focus level indication for 4o the digital image; wherein the predetermined distance is greater than a distance between :"adjacent discrete image elements.

A method of detecting focus of a digital image according to claim 1, S'wherein the predetermined distance is between 2 and 10 times the distance between adjacent discrete image elements. 44**

3. A method of detecting focus of a digital image according to claim 2, wherein the predetermined distance is between 6 and 8 times the distance between adjacent discrete image elements.

4. A method of detecting focus of a digital image according to any one of claims 1 to 3, wherein the matrix is a rectangular matrix.

A method of detecting focus of a digital image according to claim 4, wherein the second discrete image element is at a predetermined diagonal distance from the first discrete image element.

6. A method of detecting focus of a digital image according to any preceding claim, wherein the step of comparing comprises: 10 comparing the first discrete image element with at least two second discrete image elements to provide correlation factors between the first discrete image element and each of the second discrete image elements; and summing the correlation factors to provide a total correlation factor for the first discrete image element.

7. A method of detecting focus of a digital image according to claim 6, wherein the at least two second discrete image elements are each on a different diagonal line to the first discrete image element.

8. A method of detecting focus of a digital image according to claim 7, wherein the two diagonal lines are orthogonal.

9. A method of detecting focus of a digital image according to any one of claims 6, 7 or 8, wherein the step of utilising comprises: .o*summing the total correlation factors for each of a plurality of first discrete image elements to provide the focus level indication for the digital image. o:o0•

10. A method of autofocusing a lens in a digital camera, the method comprising the steps of: *detecting the focus of a first digital image according to the method of any preceding claim; detecting the focus of a second digital image according to the method of any preceding claim; comparing the focus level indications for the first and second digital images; providing a lens control signal dependent on the comparison of the focus level indications; and moving a lens of the camera dependent on the lens control signal. -11-

11. Apparatus for detecting focus of a digital image, the apparatus comprising: a matrix of discrete image elements on which a digital image can be acquired, each discrete image element providing a value indicating a characteristic of the portion of the image acquired thereby; a comparator coupled to the matrix and having a correlator to correlate at least a first discrete image element with at least a second discrete image element at a predetermined distance from the first discrete image element in the matrix so as to provide a correlation factor between the first and second discrete image elements; a focus level circuit coupled to the comparator to provide a focus level 0 indication for the digital image dependent on the correlation factor; wherein the predetermined distance is greater than a distance between °adjacent discrete image elements. o

12. Apparatus for detecting focus of a digital image according to claim 11, S•wherein the predetermined distance is between 2 and 10 times the distance between adjacent discrete image elements. 20

13. Apparatus for detecting focus of a digital image according to claim 12, wherein the predetermined distance is between 6 and 8 times the distance *between adjacent discrete image elements.

14. Apparatus for detecting focus of a digital image according to any one of claims 11 to 13, wherein the matrix is a rectangular matrix.

Apparatus for detecting focus of a digital image according to claim 14, wherein the second discrete image element is at a predetermined diagonal distance from the first discrete image element.

16. Apparatus for detecting focus of a digital image according to claim 11, wherein the correlator further comprises a storage element for storing the 12 correlation factors between the first discrete image element and each of at least two second discrete image elements and an adder coupled to the storage element to provide a total correlation factor for the first discrete image element dependent on the sum of the stored correlation factors.

17. Apparatus for detecting focus of a digital image according to claim 16, wherein the at least two second discrete image elements are each on a different diagonal line to the first discrete image element.

18. Apparatus for detecting focus of a digital image according to claim 17, wherein the two diagonal lines are orthogonal. •o o

19. Apparatus for detecting focus of a digital image according to any one of claims 16, 17 or 18, wherein the focus level circuit provides the focus level indication for the digital image based on the total correlation factors for each o.*of a plurality of first discrete image elements.

A digital camera, comprising: a lens; 20 apparatus for detecting focus of a digital image according to any one of claims 11 to 19; a comparator coupled to the apparatus to provide a lens control signal dependent on a comparison of the focus level indications for a first and a second digital image; and a motor coupled to the lens and to the comparator so as to move the lens dependent on the lens control signal.

21. A method of detecting focus of a digital image substantially as described hereinabove with reference to the drawings.

22. A method of autofocusing a lens in a digital camera substantially as described hereinabove with reference to the drawings. 13

23. Apparatus for detecting focus of a digital image substantially as described hereinabove with reference to the drawings.

24. A digital camera substantially as described hereinabove with reference to the drawings. S