WO2010061250A1

WO2010061250A1 - Auto-focus image system

Info

Publication number: WO2010061250A1
Application number: PCT/IB2009/005357
Authority: WO
Inventors: Hiok-Nam Tay
Original assignee: Hiok-Nam Tay
Priority date: 2008-11-26
Filing date: 2009-04-23
Publication date: 2010-06-03

Abstract

An auto focus image system that includes a pixel array coupled to a focus signal generator. The pixel array captures an image that has at least one edge with a width. The focus signal generator may generate a focus signal that is a function of the edge width and/or statistics of edge widths. A processor receives the focus signal and/or the statistics of edge widths and adjust a focus position of a focus lens. The edge width can be determined by various techniques including the use of gradients. A histogram of edge widths may be used to determine whether a particular image is focused or unfocused. A histogram with a large population of thin edge widths is indicative of a focused image.

Description

AUTO-FOCUS IMAGE SYSTEM

Reference To Cross-Related Applications This application claims priority to United States Patent Application 61/118.203 filed on November 26, 2008.

BACKGROUND OF THE INVENTION

1. Field of the Invention

The subject matter disclosed generally relates to auto- focusing electronically captured images.

2. Background Information

Photographic equipment such as digital cameras and digital camcorders may contain electronic image sensors that capture light for processing into still or video images, respectively. Electronic image sensors typically contain millions of light capturing elements such as photodiodes.

Many image capturing devices such as cameras include an auto-focusing system. The process of auto-focusing includes the steps of capturing an image, processing the image to determine whether it is in focus, and if not, generating a feedback signal that is used to vary the focus of the device lens. There are two primary auto-focusing techniques. The first technique involves contrast measurement, the other technique looks at phase differences between pairs of images. In the contrast method the intensity difference between adjacent pixels is analyzed and the focus is adjusted until a maximum contrast is detected. Although acceptable for still pictures the contrast technique is not suitable for motion video.

The phase difference method includes splitting an incoming image into two images that are captured by separate image sensors. The two images are compared to determine a phase difference. The focus is adjusted until the two images match. The phase difference method requires additional parts such as beam splitters and an extra image sensor. Additionally, the phase difference approach analyzes a relatively small band of fixed detection points. Having a small group of detection points is prone to error because noise may be superimposed onto one or more points. This technique is also ineffective if the detection points do not coincide with an image edge. Finally, because the phase difference method splits the light the amount of light that impinges on a light sensor is cut in half or even more. This can be problematic in dim settings where the image light intensity is already low.

BRIEF SUMMARY QF THE INVENTION

An auto focus image system that includes a pixel array coupled to a focus signal generator. The pixel array captures an image that has at least one edge with a width. The generator generates a focus signal that is a function of the edge width and various statistics of edge width.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic of an embodiment of auto-focus image pickup apparatus

FIG. 2 is a schematic of an alternate embodiment of auto- focus image pickup apparatus

FIG. 3 is a block diagram of focus signal generator.

FIG. 4 is an illustration of a horizontal Sobel operator's operation on a grayscale matrix.

FIG. 5 illustrates a calculation of edge width from horizontal gradient.

FIG. 6A, 6B are illustrations of a calculation of edge width of a vertical edge having a slant angle φ.

FIG. 6C, 6D are illustrations of a calculation of edge width of a horizontal edge having a slant angle φ.

FIG. 7 is a flowchart of a process to calculate slant angle φ and correct the edge width for a vertical edge having a slant.

FIG. 8 is an illustration of a vertical concatenated edge.

FIG. 9A is an illustration of a group of closely-packed vertical bars.

FIG. 9B is a graph of a grayscale value across FIG. 9A;

FIG. 9C is a graph of a horizontal Sobel gradient across FIG. 9A.

FIG. 10 is a flowchart of a process to eliminate closely- packed edges having shallow depth of modulation. FIG. 11 is a histogram of edge widths illustrating a range of edge widths for calculating fine focus signal.

FIG. 12 is an illustration of a scene of image capture.

FIG. 13 is a graph illustrating a variation of narrow-edge count during a focus scan of the scene of FIG. 12

FIG. 14 is a graph illustrating a variation of gross focus signal during a focus scan of the scene of FIG. 12.

FIG. 15 is a graph illustrating a variation of fine focus signal across focus positions.

FIG. 16 is an illustration of an apparatus displaying multiple objects in the image and a selection mark over one of the objects.

FIG. 17 is a block diagram of a first embodiment of an exclusion calibrator.

FIG. 18 is a block diagram of a second embodiment of an exclusion calibrator.

FIG. 19 is a flowchart of a process to determine exclusion ranges for one edge characteristic.

FIG. 20 is a flowchart of a process to determine a chain of exclusion ranges for multiple edge characteristics.

FIG. 21 is an illustration of an image having edges at different focus positions each under a different illumination.

FIG. 22 is a first histogram of maximal luminance of edges and a second histogram of sharp-enough edges within a region (wall clock) of FIG. 21. FIG. 23 is a first histogram of hue of edges and a second histogram of sharp-enough edges within a region (wall clock) of FIG. 21.

FIG. 24 is a first histogram of luminance change magnitude across edges and a second histogram of sharp-enough edges within a region (wall clock) of FIG. 21.

FIG. 25 is a flowchart of a process of focus scan.

FIGs. 26, 27, 28 are illustrations of captured images of the scene of FIG. 12 corresponding to different focus positions.

FIG. 29 is a graph illustrating a sharp-edge count versus hue at focus positions FA, FB, and FC of FIG. 13.

FIG. 3OA is a graph showing hue exclusion ranges for focus position FA ("mountains, sun & horizon") .

FIG. 3OB is a graph showing hue exclusion ranges for focus position FB ("car") .

FIG. 3OC is a graph showing hue exclusion ranges for focus position FC ("person") .

FIG. 31 is an illustration showing edges in sharp focus corresponding to focus position FA of FIG. 13 highlighted on a display.

FIG. 32 is an illustration showing edges in sharp focus position FB of FIG. 14 highlighted on a display.

FIG. 33 is an illustration showing buttons on the image capture device for the user to request jumping to the next farther or nearer sharp focus positions.

FIG. 34 is a flowchart for focus control system in seek mode for FIG. 35. FIG. 35 is a graph of narrow-edge count versus focus position during a seek to focus position FB and a switching to tracking mode.

FIG. 36 is a graph showing the gross focus signal during a seek to focus position FB corresponding to FIG. 35.

FIG. 37 is a graph of narrow-edge count versus focus position for an alternate embodiment wherein the focus system switches to tracking mode.

FIG. 38 is a flowchart for focus control system in seek mode for FIG. 37.

FIG. 39 is a graph illustrating the variation of the fine focus signal during tracking of a moving object for an embodiment of a focus control system that maintains the fine focus signal at the minimum.

FIG. 40 is a graph illustrating how fine focus signal varies with time during sharp focus acquisition.

FIG. 41 is a graph illustrating the variation of the fine focus signal during tracking of a moving object for an alternate embodiment of focus control system that maintains the fine focus signal at an offset from the minimum.

FIG. 42A is an illustration showing a first image captured when a selection region is defined to select the bird and the exclusion calibrator calibrates for the bird in sharp focus.

FIG. 42B is an illustration showing a second image captured after the bird has moved to a second location of the image frame .

FIG. 42C is an illustration showing a third image captured after the bird has moved to a third location from the second location. FIG. 43 is a first histogram of length of concatenated edges and a second histogram of only concatenated sharp-enough edges (bird) of FIG. 42A.

FIG. 44 is a first histogram of slant angle from the vertical axis of edges and a second histogram of only sharp- enough edges (bird) of FIG. 42A.

FIG. 45 is a block diagram of an alternate embodiment of a focus signal generator.

FIG. 46 is a schematic of an alternate embodiment of auto- focus image pickup apparatus.

FIG. 47 is a schematic of an embodiment of auto-focus image pickup apparatus having a main pixel array and an auxiliary pixel array.

FIG. 48 is a schematic of an alternate embodiment of auto- focus image pickup apparatus having a main pixel array and an auxiliary pixel array.

FIG. 49 is a schematic of an alternate embodiment of auto- focus image pickup apparatus having a main pixel array and an auxiliary pixel array.

FIG. 50 is a illustration of a variation of an edge width from a main pixel array and a variation of an edge width from an auxiliary pixel array at different focus positions.

DETAILED DESCRIPTION

Disclosed is an auto focus image system that includes a pixel array coupled to a focus signal generator. The pixel array captures an image that has at least one edge with a width. The focus signal generator may generate a focus signal that is a function of the edge width and/or statistics of edge widths. A processor receives the focus signal and/or the statistics of edge widths and adjust a focus position of a focus lens. The edge width can be determined by various techniques including the use of gradients. A histogram of edge widths may be used to determine whether a particular image is focused or ^'* " ^■ unfocused. A histogram with a large population of thin edge widths is indicative of a focused image.

Architecture

Referring to the drawings more particularly by reference numbers, Figure 1 shows an embodiment of an auto- focus image capture system 102. The system 102 may be part of a digital still camera, but it is to be understood that the system can be embodied in any device that requires controlled focusing of an image. The system 102 may include a focus lens 104, a pixel array and circuits 108, an A/D converter 110, a processor 112, a display 114, a memory card 116 and a drive motor/circuit 118. Light from a scene enters through the lens 104. The pixel array and circuits 108 generates an analog signal that is converted to a digital signal by the A/D Converter 110. The pixel array 108 may incorporate a mosaic color pattern, e.g. the Bayer pattern. The digital signal is sent to the processor 112 that performs various processes, e.g. color interpolation, focus position control, color correction, image compression/decompression, user interface control, and display control, and to the focus signal generator 120. Where the focus signal generator 120 and the processor 112 reside within different packages, a color interpolation unit 148 may be implemented to perform color interpolation on the digital signal 130 to estimate the missing color signals on each pixel for the focus signal generator 120. Alternately, where the focus signal generator 120 and the processor 112 reside together within a package 144, the focus signal generator 120 may input fully interpolated color images from the processor 112 on bus 146 as shown in Figure 2.

The focus signal generator 120 receives a group of control signals 132 from the processor 112, in addition, and may output signals 134 to the processor 112. The output signals 134 may comprise one or more of the following: a focus signal 134, a narrow-edge count, and a set of numbers representing a statistics of edge width in the image. The processor 112 may generate a focus control signal 136 that is sent to the drive motor/circuit 118 to control the focus lens 104. A focused image is ultimately provided to the display 114 and/or stored in the memory card 116. The algorithm(s) used to adjust a focus position may be performed by the processor 112.

The pixel array and circuits 108, A/D Converter 110, focus signal generator 120, and processor 112 may all reside within a package. Alternately, the pixel array and circuits 108, A/D Converter 110, and focus signal generator 120 may reside within a package 142 as image sensor 150 shown in Figure 1, separate from the processor 112. Alternately, the focus signal generator 120 and processor 112 may together reside within a package 144 as a camera controller 160 shown in Figure 2, separate from the pixel array 108 and A/D Converter 110.

Focus Signal Generator

Figure 3 shows an embodiment of a focus signal generator 120 receiving image (s) from a image providing unit 202. The image providing unit 202 may be the color interpolator 148 in Figure 1 or the processor 212 in Figure 2. The focus signal generator 120 may comprise an edge detection & width measurement (EDWM) unit 206, a focus signal calculator 210, a length filter 212, a width filter

209, an exclusion filter 207, and an exclusion calibrator 205. It may further comprise a fine switch 220 and an exclusion switch 230, controlled by inputs ^λfine' 222 and 'exclude' 232, respectively. The focus signal generator 120 may provide a narrow-edge count from the width filter 209 and a focus signal from the focus signal calculator

210, the focus signal being configurable between a fine focus signal and a gross focus signal, selectable by input Λfine' 222. Alternately, both fine focus signal and gross focus signal may be calculated and output as part of output signals 134. The edge detection & width measurement unit 206 receives image (s) provided by the image providing unit 202. The exclusion calibrator 205 receives command and control signals (calibrate command, index, and R/W) input to the focus signal generator 120. In the context of Figures 1 and 2, these command and control signals, along with control signal ^Λfine' 222 and 'exclude' 232, may be provided by the processor 112 in signals 132. Also in the context of Figures 1 and 2, the output signals 134 may be provided to the processor 112, which functions as a focus system controller that controls the focus position of the focus lens 104 to bring images of objects into sharp focus on the pixel array 108 by analyzing the output signals 134 to detect a sharp object in the image. Various components of the focus signal generator 120 are described below.

The EDWM unit 206 may transform the input image such that the three signals of the image, red (R) , green (G) and blue (B) are converted to a single signal. This signal can be generated by transforming the image to a grayscale image. Several techniques can be utilized to transform an image to a grayscale. RGB values can be used to calculate a luminance or chrominance value or a specific ratio of RGB values can be taken to form the grayscale image. For example, the luminance value can be calculated with the equation Y=0.2126*R + 0.7152*G + 0.0722*B, where Y is luminance value. The signal from the grayscale image may then be processed by a Gaussian filter or any lowpass filter to smooth out pixel signal values among neighboring pixels to remove noise.

The focus signal generator 120, 120', 120" is not limited to grayscale signal. It may operate on any one image signal to detect one or more edges in the image signal. Or it may operate on any combination of the image signals, for example Y, R-G, or B-G. It may operate one each and every one of the R, G, B image signals separately, or any one or more combinations thereof, to detect edges. It may form statistics of edge widths for each of the R, G, B image signals, or any combination thereof. It may form a focus signal by from statistics of edge widths from one or more image signals. A gradient of the processed image is then calculated. There are various methods available to calculate the gradient, including Laplacian, and Sobel. Gradients across the columns and the rows may be calculated to detect vertical and horizontal edges respectively, using a Sobel-X operator and a Sobel-Y operator, respectively. Sobel X- operator at pixel location [k, g] where k is row number and g is column number, is given by the equation Sx[k, g] = U[k, g+1] - U[k, q-1] . Sobel Y- operator at the same location is given by the equation Sy[k,q] = U[k+l,q] - U[k- l,g], where U is a grayscale signal of the processed image.

Each pixel is tagged either horizontal edge ( ^λH' ) or vertical edge ( ^λV ) if either vertical or horizontal gradient magnitude exceeds a predetermined lower limit, e.g. 5 for an 8-bit image, or no edge if neither is true. This lower limit eliminates spurious edges due to gentle shading or noise. A pixel is tagged as a vertical edge if its horizontal gradient magnitude exceeds its vertical gradient magnitude by a predetermined hysteresis amount or more, e.g. 2 for an 8-bit image, and vice versa. If both gradient magnitudes differ less than the hysteresis amount, the pixel gets a direction tag same as that of its nearest neighbor that has an direction tag already determined. For example, if the image is scanned from left to right in each row and from row to row downwards, the sequence of inspection of neighboring pixels may be the pixel above first, the pixel above left second, and the pixel on the left third, and the pixel above right last. Applying this hysteresis helps to ensure that adjacent pixels get similar tags if each of them has nearly identical horizontal and vertical gradient magnitudes. Figure 4 illustrates the result of such tagging on a 6-by-6 array of horizontal and vertical gradients. In each cell, the horizontal gradient is in the upper-left, vertical gradient is on the right, and direction tag is at the bottom. Only pixels that have either horizontal or vertical gradient magnitude exceeding 5 gualify at this step as edge pixels are printed in bold and gets a direction tag.

The image, gradients and tags may be scanned horizontally for vertical edges, and vertically for horizontal edges. Each group of consecutive pixels in the same row having the same horizontal gradient polarity and all tagged for vertical edge may be designated a vertical edge if no adjacent pixel on left or right of the group satisfies that same. Likewise, each group of consecutive pixels in the same column having the same vertical gradient polarity and all tagged for horizontal edge may be designated a horizontal edge if no adjacent pixel above or below the group satisfies the same. Thus horizontal and vertical edges may be identified.

Each edge may be refined by removing pixels whose gradient magnitudes are less than a given fraction of the peak gradient magnitude within the edge. Figure 5 illustrates this step using a refinement threshold equal to one third of the edge's peak gradient magnitude, refining the edge width down to 3 from the original 9. This edge refinement distinguishes the dominant gradient component that sets the apparent edge width that dominates visual perception of the edge's sharpness despite an image having multiple overlapping shadings that may cause gradients to gently decay over many pixels.

Edge width may be calculated in any one of known methods. One method of calculating edge width is simply counting the number of pixels within an edge. An alternate method of calculating edge width is shown in Figure 5. In Figure 5, a first fractional pixel position (2.4) is found between a first outer pixel (pixel 3) of a refined edge and the adjacent outside pixel (pixel 2) by an interpolation from the refinement threshold 304. Likewise, a second fractional pixel position (5.5) is found between a second outer pixel (pixel 5) and its adjacent outside pixel (pixel 6) . The edge width is found as the difference between these two fractional pixel positions, 5.5 - 2.4 = 3.1.

Each horizontal or vertical edge's edge width may be corrected for its slant from either the horizontal or vertical orientation, respectively. Figure 6A, 6B illustrate a correction calculation for a vertical edge that has a slant from the vertical line. The correction consists of multiplying the edge width by a factor of cos φ , where φ is the angle of slant from the vertical line. Likewise, for a horizontal edge, φ is the angle of slant from the horizontal line. This slant angle φ may be used further in the exclusion filter 207 and exclusion calibrator 205. Figure 7 shows a flowchart of a process to correct edge widths for slant for vertical edges. (For horizontal edges, substitute 'row' for 'column' , and interchange 'vertical' with 'horizontal'.) For each vertical edge, at step 502, locate the column position where the horizontal gradient magnitude peaks, and find the horizontal gradient x. At step 504, find where the vertical gradient magnitude peaks along the column position and within two pixels away, and get the vertical gradient. At step 506, find the slant angle φ = tan^"1(y/x) . At step 508, scale down edge width by multiplying with cos {φ) . Adjacent narrow edges may be eliminated if their peak gradient magnitudes are below a predetermined fraction of the nearest wider edge's peak gradient magnitude. Figure 9A, 9B, and 9C illustrate a problem that is addressed. Figure 9A illustrates three vertical white bars separated by two narrow black spaces each 2 pixels wide. The middle white bar is a narrow bar 2 pixels wide. Figure 9B shows the grayscale plot horizontally across the image in Figure 9A for a sharp image and a blurred image. Figure 9C plots the Sobel-x gradients of Figure 9B for the sharp image and blurred image. In Figure 9C, the first edge (pixels 2-5) for the blurred image is wider than the sharp image, and likewise the last edge (pixels 13-15), but the two narrowest edges (pixels 9 & 10, and pixels 11 & 12) have width of two in both images. In Figure 9B, the corresponding slopes at pixels 9 & 10, and pixels 11 & 12, take two pixels to complete transition. The blurred image, however, has a significant decline, as much as 50%, of peak gradient magnitude from the wider edge to the narrow edge. The sharp image, on the other hand, changes less than 25% between the wider and the narrow edges. The significant decline, e.g. 40% or greater, in peak gradient magnitude for narrow edge adjacent a wider edge of opposite gradient polarity gives the hint that the image is not well focused, and thus the narrow edge should not be taken as an indication that the image is sharp. Likewise, subsequent successive narrow edges of alternating gradient polarity should not be taken as such indication as long as they are in- close proximity to each other, e.g. no more than 1 pixel apart. Furthermore, given that one narrow pulse may have been eliminated due to having a peak gradient less than the elimination threshold, two successive narrow pulses having identical gradient polarity and spaced no more than one plus the narrow width apart and one of them adjacent a wider pulse of significantly larger gradient magnitude should not be taken as such indication either. We thus have the following algorithm for eliminating closely-packed narrow edges based on a screen threshold established as a fraction of a wider pulse adjacent a narrow pulse, and a modulation screen flag that can be turned on and off. For each edge, the screen threshold and screen flag are determined as in Figure 10, then the edge is eliminated unless one of the following conditions is true: (a) the screen flag is off for this edge, (b) the edge's peak gradient magnitude is not below the screen threshold for this edge, or (c) the edge width is not less than sharp_edge_width + 1, a number having been assigned for sharp_edge_width to designate an edge width of a sharp edge. For the example shown in Figures 9A-9C, sharp_edge_width is 2.

Figure 10 is a flowchart to determine the screen threshold and screen flag at each edge. For vertical edges, assume scanning from left to right along a row. (For horizontal edges, assume scanning from top to bottom along a column.) A number is assigned for sharp_edge_width and is 2 for the example shown in Figures 9A-9C. Starting at the first edge at step 702, each edge is queried at step 720 as to whether its edge width is greater than or equal to one plus sharp_edge_width. If yes, the edge is a wider edge, and step 706 follows to set screen threshold to beta times the peak gradient magnitude of the edge, beta being from 0.3 to 0.7, preferably 0.55, then step 708 follows to turn on screen flag, then proceed to the next edge. If no, the edge is not a wider edge, and step 730 follows to check whether the spacing from the prior edge of the same gradient polarity is greater than 0.5 plus sharp_edge_width and the immediate prior edge of opposite polarity is more than 1 pixel away. If yes, step 710 follows to turn off the screen flag, because there is no prior edge that could interfere with the present edge. If no, keep the screen flag and screen threshold for this edge and proceed to determine the screen flag and screen threshold for the next edge .

Below describes a function of length filter 212. Broadly defined, length filter 212 creates a preference for edges that each connects to one or more edges of a similar orientation. A group of edges that are similarly oriented and mutually connected within the group ("concatenated edge") is less likely to be due to noise, compared with an isolated edge that does not touch any other edge of similar orientation. The more edges of similar orientation thus concatenated together, the lesser the chance of them being due to noise. The probability of the group being due to noise drops off exponentially as the number of edges within the group increases, and far faster than linearly. This property can be harnessed to reject noise, especially under dim-lit or short-exposure situations where the signal-to- noise ratio is weak, e.g. less than 10, within the image or within the region of interest. The preference may be implemented in any reasonable method to express such preference. The several ways described below are merely examples .

A first method is to eliminate edges that belong to vertical/horizontal concatenated edges having lengths lesser than a concatenated length threshold. The concatenated length threshold may be larger when the region of interest is dimmer. For example, the concatenated length threshold may start as small as 2, but increases to 8 as a signal-to-noise ratio within the region of interest drops to 5. The concatenated length threshold may be provided by the processor 112, 112', 112", for example through a ^Λlength command' signal, shown in Figure 3, as part of signals 132.

A second method is to provide a length-weight in the length filter 212 for each edge and apply the length-weight to a calculation of focus signal in the focus signal calculator 210. An edge that is part of a longer concatenated edge receives a larger weight than one that is part of a shorter concatenated edge. For example, the length-weight may be the square of the length of the concatenated edge. Thus, a contribution of each edge towards the focus signal may be multiplied by a factor A/B before summing all contributions to form the focus signal, where B is the sum of the length-weights of all edges that enter the focus signal calculation, and A is the length- weight of the edge. Likewise, the edge-width histogram, which may be output as part of signals 134, may have edges that are members of longer concatenated edges contribute more to the bins corresponding to their respective edge widths instead of all edges contribute the same amount, e.g. +1. Thus, for example, each edge may contribute AlC, where C is the average value of A. Similarly, the narrow- edge count may have edges that are members to longer concatenated edges contribute more. Thus, for example, the contribution from each edge may be multiplied by A/D, where D is an average of A among edges that are counted in the narrow-edge count. A group of N vertical (horizontal) edges where, with the exception of the top (leftmost) and the bottom (rightmost) ones, each edge touches two other vertical (horizontal) edges, one above (to the left of) itself, the other below (to the right of) itself is a vertical (horizontal) concatenated edge of length N. The top (leftmost) edge needs only touch one edge below (to the right of) itself. The bottom (rightmost) edge needs only touch one edge above (to the left of) itself.

Figure 8 illustrates a vertical concatenated edge and its length. A vertical (horizontal) concatenated edge of length N is a group of N vertical (horizontal) edges where, with the exception of the top (leftmost) and the bottom (rightmost) ones, each edge touches two other vertical (horizontal) edges, one above (to the left of) itself, the other below (to the right of) itself. The top (leftmost) edge needs only touch one edge below (to the right of) itself. The bottom (rightmost) edge needs only touch one edge above (to the left of) itself. In Figure 8, cells R2C3 and R2C4 form a first vertical edge, cells R3C3, R3C4, and R3C5 together form a second vertical edge, and cells R4C4 and R4C5 together form a third vertical edge. The first and the third vertical edges each touches only one other vertical edge, whereas the second vertical edge touches two other vertical edges. The first, second, and third vertical edges together form a vertical concatenated edge having a length of 3.

In a situation (not shown) where a vertical (horizontal) concatenated edge has two or more branches, i.e. having two edges in a row (column), the length may be defined as the total number of edges within the concatenated edge. Alternately, the length may be defined as the vertical (horizontal) distance from a topmost (leftmost) edge therein to a bottommost (rightmost) edge therein plus one.

Although shown in Figure 3 to immediately follow the Edge Detection & Width Measurement Unit 206, other arrangements are possible. For example, the Length Filter 212 may be inserted before the focus signal calculator 210 and after the exclusion switch 230, wherein the edges processed by the Length Filter 212 are those that pass through either or both of the width filter 209 and exclusion filter 207 depending on the ^λfine' and ^λexclude' command signals.

In an alternate embodiment of a focus signal generator, the fine switch 220 may be removed so that the focus signal calculation unit 210 receives a first set of data not filtered by the width filter 209 and a second set filtered, and for each calculates a different focus signal, gross focus signal for the former, fine focus signal for the latter, and outputs both to the processor 112, 112' .

Refer next to Figure 3 to understand an operation of the Width Filter 209. Figure 11 plots a histogram of edge widths, i.e. a graph of edge counts against edge widths. At edge width of 2, i.e. the aforementioned sharp_edge_width, there is a peak, indicating presence of sharp edges in the image. At edge widths of 4 and 5, however, there are peaks, indicating edges that are blurred, possibly due to the corresponding imaged objects being out of focus, being at a different distance away from the focus lens than those objects that give rise to the sharp edges. For calculating a focus signal, edges whose widths lie outside a predetermined range ("narrow-edge range") may be de-emphasized using the Width Filter 209. The Width Filter 209 may create a lesser weight for edge widths outside the narrow-edge range for use in the focus signal calculation. For example, edge widths may be assigned weight of 1.0, whereas edges widths more than +1 to the right of the upper limit 840 a weight of 0, and edge widths in between weights between 0 and 1.0, falling monotonically with edge width. Alternately, the Width Filter 209 may prevent such edges from entering the focus signal calculation altogether. Appropriate upper and lower limits 830, 840 depend on several factors, including crosstalk in the pixel array 108, the interpolation method use to generate missing colors for the image received by the focus signal generator 120, and the filter coefficients used in the lowpass filter employed in the Edge Detection and Width Measurement Unit 206. Appropriate upper and lower limits 830, 840 and the parameter sharp_edge_width may be determined for the image pickup apparatus 102, 102' by capturing images of various degrees of sharpness and inspecting the edge width histograms. For example, if a sharp image has a peak at edge width of 2, an appropriate lower and upper limit may be 1.5 and 3, respectively, and the sharp_edge_width may be set to 2.0. The lower and upper limits and sharp_edge_width may be provided to the focus signal generator 120, 120', 120" by the processor 112, 112". When ^Λfine command' is ON, the fine focus signal thus calculated de-emphasizes edge widths outside the narrow-edge range.

In addition, the Width Filter 209 may calculate a total count of the edges whose edge widths fall within the narrow-edge range and output as part of output signals 134. Narrow-Edge Count may be input to and used by the focus system controller (processor 112) to detect a presence of sharp image and/or for initiating tracking.

Referring next to the focus signal calculator 210 of Figure 3, the focus signal calculator 210 receives edge widths and outputs a focus signal. The focus signal may be calculated as a weighted average of all the edge widths where the weights are the edge counts for each edge width, viz. focus signal = Ew₁^₁ /EW₁, where e_x are the edge widths, W₁ are the weights, where here W₁=C₁, C₁ being the edge count at edge width e_x, i being a bin number of a histogram of edge widths. Alternately, the weight at each edge width may be the edge count for the edge width multiplied by the edge width itself, i.e. W₁=C₁B₁. In addition, preferences from the Width Filter 209 that are expressed in the form of weights may be further multiplied to each edge width. For example, for weights Q₁ produced by the Width Filter 209, XQ₁=I, focus signal may be calculated as 1O₁W₁G₁ /Ew₁. If control signal ^Λfine' is ON and 'exclude' is OFF, the focus signal would be a value very close to the sharp edge width of 2.0 for the example shown in Figure 11, indicating that among object details within the focus distance range that would produce edge widths between 2.0 and 3.0, most are actually in sharp focus. If control signal 'fine' is OFF and 'exclude' is OFF, the focus signal may be a value close to 5.0, indicating that there are substantial details of the image that are out of focus. Turning ON the fine switch 220 allows the focus signal to respond more to objects slightly blurred while less to those that are completely blurred. When the fine switch 220 is ON, we shall refer to the focus signal as fine focus signal, whereas when the fine switch 220 is OFF, gross focus signal. As aforementioned, the emphasis expressed by the Length Filter 212 may be incorporated into the focus signal in one of several ways, such as eliminating an edge that is de-emphasized from entering the focus signal calculation, or reducing below +1 the edge's contribution towards a count e-_L of a corresponding edge width bin.

Figure 15 sketches a response of the fine focus signal to an adjustment of focus position in the vicinity of where an object is in sharp focus. The fine focus signal reaches a minimum value, approximately at sharp_edge_width, where focus position brings an image into sharp focus, and increases if otherwise. Fine focus signal may be used for tracking objects already in-focus or very nearly so. For moving objects, fine focus signal allows the focus control system to keep the objects in sharp focus even if the focus distance continues to change. Fine focus signal may also be used to acquire sharp focus ("acquisition") of an object that is not yet in sharp focus but close enough such that it gives rise to edges whose widths fall within the narrow- edge range. Since the edge width histogram exhibits a peak at the edge width corresponding to the object away from the sharp_edge_width, resulting in the fine focus signal being larger than the sharp_edge_width, the focus control system may respond by adjusting the focus position to bring the fine focus signal value towards the sharp_edge_width, thus centering the peak of edge width due to the object at the edge width value equal to sharp edge width. Basic Use

Figures 12-16 illustrate how narrow-edge count, gross focus signal, and fine focus signal may be used to perform focus control to achieve sharp images.

Figure 12 illustrates an outdoor scene having 3 groups of objects at different focus distances: "person" in the foreground, "mountain, sun, and horizon" in the background, and "car" in the between.

Figure 13 is an illustration of the narrow-edge count plotted against time when the focus position of the focus lens 104 sweeps from far to near for the scene illustrated in Figure 13. The narrow-edge count peaks when the focus position brings an object into a sharp image on the pixel array 108. Thus the narrow-edge count plot exhibits 3 peaks, one each for "mountain, sun, and horizon", "car", and "person", in this order, during the sweep.

Figure 14 shows the gross focus signal plotted against time. The gross focus signal exhibits a minimum when the focus position is near each of the 3 focus positions where the narrow-edge count peaks. However, at each minimum, the gross focus signal is not at the sharp edge width level, which is 3.0 in this example, due to bigger edge widths contributed by the other objects that are out-of-focus .

Figure 15 illustrates the fine focus signal plotted against focus position in the vicinity of the sharp focus position for "car" in the scene of Figure 12. Fine focus signal achieves essentially the sharp edge width, which is 2 in this example, despite the presence of blurred objects ("person" and "mountains, sun, and horizon") . Referring to Figure 11 again, where two peaks at widths of 4 and 5 are contributed by those two groups of blurred objects, this can be understood as the Width Filter 324 having reduced the weight or eliminated altogether the contributions from the edge widths to the right of upper-limit 840.

A focus control system may use the gross focus signal to search for the nearest sharp focus position in a search mode. It can move the focus position away from the current focus position to determine whether the gross focus signal increases or decreases. For example, if the gross focus signal increases (decreases) when the focus position moves inwards (outwards), there is a sharp focus position farther from the current focus position. The processor 112, 112', 112" can then provide a focus drive signal to move the focus lens 104 in the direction towards the adjacent sharp focus position.

A focus control system may use the fine focus signal to track an object already in sharp focus to maintain the corresponding image sharp (thus a "tracking mode") despite changes in the scene, movement of the object, or movement of the image pickup apparatus. When an object is in sharp focus, fine focus signal level is stable despite such changes. Hence a change in the fine focus signal suggests a change in focus distance of the object from the image pickup apparatus. By "locking" the focus control system to a given fine focus signal level near the minimum, for example between 2.0 to 2.5 in this example, in particular 2.1, any shift in the fine focus signal level immediately informs the processor 112, 112', 112" of a change in the focus distance of the object. The processor 112, 112', 112" can then determine a direction and cause the focus lens 104 to move to bring the fine focus signal level back to the "locked" level. Thus the image pickup apparatus 102, 103, 103', 103" is able to track a moving object.

A focus control system, e.g. as implemented in algorithm in processor 112, 112', 112", may use narrow-edge count to trigger a change from a search mode to a tracking mode. In the tracking mode, the focus control system uses the fine focus signal to "lock" the object. Before the focus position is sufficiently near the sharp focus position for the object, the focus control system may use the gross focus signal to identify the direction to move and regulate the speed of movement of the lens. When a object is coming into sharp focus, narrow-edge count peaks sharply. The processor 112, 112', 112" may switch into tracking mode and use the fine focus signal for focus position control upon detection of a sharp rise in narrow- edge count or a peaking or both. A threshold, which may be different for each different sharp focus position, may be assigned to each group of objects found from a end-to-end focus position "scan", and subsequently when narrow-edge count surpasses this threshold the corresponding group of objects is detected. For a stationary scene, e.g. for still image taking, an end-to-end focus position scan can return a list of maximum counts, one maximum count for each peaking of the narrow-edge count. A list of thresholds may be generated from the list of maximum counts, for example by taking 50% of the maximum counts.

Figure 16 illustrates an image pickup apparatus 102 having a display 114, an input device 107 comprising buttons, and selection marker 1920 highlighted in the display 114. A user can create, shape and maneuver the selection marker 1920 using input device 107. Although shown in this example to comprise buttons, input device 107 may comprise touch-screen overlaying the display 114 to detect positions of touches or strokes on the display 114. Input device 107 and processor 112, 112', 112" or a separate dedicated controller (not shown) for the input device 107 may determine the selection region. The parameters for describing the selection region may be transmitted to the focus signal generator 120, 120' , 120" over bus 132 (or internally within the processor 112 in the case where focus signal generator 120 is part of the processor 112) . In response, the focus signal generator 120 may limit the focus signal calculation or the narrow- edge count^' or both to edges within the selection region described by said parameters or de-emphasize edges outside the selection region. Doing so can de-emphasize unintended objects from the focus signal and then even the gross focus signal will exhibit a single minimum and a minimum level within 1.0 or less of the sharp edge width.

Exclusion Filtering

Below describes exclusion filtering performed by exclusion filters 207, 910.

Broadly defined, given one or more image characteristics and one or more exclusion ranges defined in each characteristic, exclusion filtering creates a preference for edges that do not fall into any one exclusion range in any one characteristic.

The collection of image characteristics may be predetermined. The following is a non-exhaustive list: hue (Figure 23), saturation, maximal luminance (Figure 22), luminance change (Figure 24), mean luminance, slant angle (Figures 6A-6D and 44), length of concatenated edge (Figures 8 and 43), and spacing to nearest edge.

A first embodiment of an exclusion filter is to eliminate edges that belong to any one exclusion range in any one characteristic given to the exclusion filter.

Figure 22 illustrates the use of maximal luminance as a characteristic for exclusion. Figure 22 illustrates an all-edge and sharp-edge histograms for maximal luminance of an image captured of the scene illustrated in Figure 21 with only the wall clock in sharp focus. Maximal luminance is the highest luminance value within an edge. Figure 22 shows a histogram for sharp edges (associated with wall clock in Figure 21) and a histogram for all edges, associated with other objects (lamp shade, bananas, platter, table cloth, light bulb) in the image that are not in sharp focus as well as the wall clock. In Figure 22, the wall clock clearly occupies a lower range of maximal luminance than all other objects, which are brightly illuminated. An exclusion range (shown in Figure 22) covering approximately the upper half of the maximal luminance range would de-emphasize or even exclude altogether most edges that are not in sharp focus.

Figure 23 illustrates the use of hue as a characteristic for exclusion. Figure 23 shows a histogram for sharp edges (associated with wall clock in Figure 21) and a histogram for all edges, associated with other objects in the image that are not in sharp focus as well as the wall clock. For hue histogram, if both sides of an edge has the same hue value, the hue bin that contains the hue value gets a count contribution of +2 from this edge; whereas if each side has a different hue value, then each hue bin that contains one of the hues gets a count contribution of +1. In Figure 23, the wall clock and the platter together occupy a small range of hue in the vicinity of brown hue. Exclusion range 1 and exclusion range 2 (shown in Figure 23) that together cover most of the rest of the hue range would exclude most edges that are not in sharp focus except those arising from the platter, which are not a major part of non-sharp edges. An edge would be de-emphasized or even excluded altogether by an exclusion filter operating by these two exclusion ranges if both hues on its two sides fall within any of these exclusion ranges.

Figure 24 illustrates the use of luminance change as a characteristic for exclusion. Luminance change is the magnitude of change of luminance across an edge. Figure 23 shows a histogram for sharp edges (associated with wall clock in Figure 21) and a histogram for all edges, associated with other objects in the image that are not in sharp focus as well as the wall clock. In Figure 24, the edges from wall clock occupy a small region at the lower range, and an exclusion range (shown in Figure 24) covering the upper range would de-emphasize or even exclude altogether most edges that are not in sharp focus.

Figure 43 shows an example of a histogram of length of concatenated edges. Such a histogram may be used to separate objects having different sizes.

An alternate embodiment of exclusion filter is to create an exclusion-weight in the exclusion filter 212 and/or 910 for each edge and apply the exclusion weight to a calculation of focus signal in the focus signal calculator 210. An edge whose value for a first characteristic falls within an exclusion range of the first characteristic receives a lesser weight for the characteristic ("characteristic weight") than one whose value fall outside all exclusion ranges. For example, the characteristic weight may be 0 in the middle of an exclusion range, rises to 1 at the edges of the exclusion range, and 1 everywhere outside all exclusion ranges. If the exclusion filtering uses only one characteristic, the characteristic weight is the exclusion weight. If two or more characteristics are used, each characteristic has a characteristic weight for each edge, and all characteristic weights may be multiplied together to form the exclusion weight. Thus each edge may have a first, a second, and a third characteristic weight where three different characteristics are used in an exclusion chain, and the exclusion weight would be the product of all three characteristic weights. Thus, a contribution of each edge towards the focus signal may be multiplied by a factor E/G before summing all contributions to form the focus signal, where G is the sum of exclusion weights of all edges that enter the focus signal calculation, and E is the exclusion weight of the edge. Likewise, the edge-width histogram, which may be output as part of signals 134, may have edges that are not members to any exclusion range of any characteristic contribute more to the bins corresponding to their respective edge widths instead of all edges contribute the same amount, e.g. +1. Thus, for example, each edge may contribute E/H instead of +1 towards the bin of the edge-width histogram, where H is an average of E among edges that are counted in the histogram. Similarly, the narrow-edge count may have edges that are not members to any exclusion range of any characteristic contribute more. Thus, for example, the contribution from each edge may be multiplied by E/J, where J is an average of E among edges that are counted in the narrow-edge count.

Exclusion Calibration

Referring next to the exclusion calibrator 205 of Figure 3, the exclusion calibrator 205 receives edges, edge widths, and the input image, and processes them to generate various exclusion control parameters to configure the exclusion filter 207. The exclusion calibrator 205 may further receive control signals ^Λcalibrate command' , λindex' , and ^λR/W . These control signals may come from a focus system controller, e.g. processor 112 of Figures 1 or 2.

The exclusion calibrator 205 is configured to inspect one or more image characteristics of edges in general and separately of sharp edges in particular upon a command on a ^calibrate command' input control signal, and create one or more exclusion ranges in those characteristics to exclude substantial amount of edges that are not sharp enough. Sharp-enough edges are edges whose widths fall within a predetermined narrow range ("sharp-edge range") surrounding sharp_edge_width. Sharp-edge range may be contained within narrow-edge range. For example, sharp-edge range may be +/-0.2 around sharp_edge_width. If sharp_edge_width is 2.0, sharp-edge range may be from 1.8 to 2.2, whereas narrow-edge range may be from 1.5 to 3.0. The exclusion ranges may be memorized under a unique index value provided by the ^xindex' input control signal by providing a write command on the 'R/W input control signal. Multiple sets of exclusion ranges may be memorized, each with a different index value. A set of exclusion ranges may be retrieved by providing its associated index value on the ^λindex' input control signal and a read command on the ^ΛR/W input control signal. The retrieved exclusion ranges may be applied by the exclusion filter 207 in Figure 3 to exclude edges that fall within these exclusion ranges. The exclusion ranges of a first characteristic may be applied before those of a second characteristic.

Figure 17 shows a block diagram of one embodiment of an exclusion calibrator. Edge Statistic unit 902 receives an image, a set of edges, and edge widths. It calculates all- edge statistics, which are histograms for one or more characteristics associated with these edges. It also calculates sharp-edge statistics, which are histograms for the same characteristics but only on edges whose edge widths fall within the sharp-edge range. Both all-edge statistics and sharp-edge statistics may be stored in memory 1 904. The exclusion calculator 906 receives both sets of statistics, performs calculation to form exclusion ranges in one or more characteristics based on histograms therein, and outputs to memory 2 908, which may store these exclusion ranges under an index value provided on input control signal ^λ index' upon a write command on the input control signal ^ΛR/W .

Figure 18 is a block diagram of a alternate embodiment of an exclusion calibrator. This embodiment adds an exclusion filter 910 to the structure of Figure 17. The exclusion filter 910 filters by a chain of sets of exclusion ranges, each set corresponding to a different characteristic. Edges are filtered by one set after another until all sets in the chain are done. At each iteration, the set of exclusion ranges calculated for the characteristic analyzed at the prior iteration is appended to the chain of sets of exclusion ranges. The resultant edges and edge widths (and exclusion weights, if the exclusion filter 910 follows the aforementioned alternate embodiment of exclusion filter) output by the exclusion filter 910 and further filtered by the newly added set are analyzed by Edge Statistics unit 902 to generate a new set of all-edge statistics and sharp-edge statistics (and exclusion weights, ditto) . The exclusion calculator 906 analyzes the new set of all-edge statistics and sharp-edge statistics (and exclusion weights, ditto) to find a set of exclusion ranges for the current iteration' s characteristic. A different characteristic not used in prior iterations is chosen to be the characteristic for this iteration.

The flowchart in Figure 20 illustrates this process. At step 1010, the set of exclusion ranges is emptied. At step 1012, edge statistics unit 902 generates all-edge statistics and sharp-edge statistics for edges and associated edge widths received from exclusion filter 910. At step 1014, exclusion calculator 906 calculates exclusion ranges for all characteristics not used in prior iterations. At step 916, a characteristic is chosen from all characteristics not used in prior iterations and its associated exclusion range calculated at step 914 is appended to the chain of exclusion range sets. At step 918, a query on "good enough" (described below) is made. If answer is yes, the iterative process is done, and the chain of sets of exclusion ranges may be stored to memory 2 908. If not, step 920, exclusion filter re-filters those edges and their associated edge width previously input to step 912 with the latest added exclusion range, and subsequently a new iteration starts at step 912 to capture the edge statistics after this re-filtering.

Each iteration in Figure 20 may choose a different characteristic that is not used in prior iterations in one of several ways. In a first way, the sequence of all characteristics may be predetermined, such that each next iteration uses the next characteristic down the sequence. In a second way, the characteristic that generates the exclusion ranges that would de-emphasize or even exclude the most edges having passed through the exclusion filter in the current iteration without de-emphasizing or removing more than a certain fraction, e.g. 10%~50%, preferably 20%, of sharp edges, is chosen as the current characteristic. This selection procedure for characteristic is applicable to the embodiment of Figure 17 as well as the embodiment of Figure 18.

Exclusion ranges corresponding to a number of different indices may be combined to form new exclusion ranges that are stored under a new index. For each characteristic present under one or more of said different indices, the exclusion ranges of said characteristic under said different indices may be logically ANDed together followed by inversion to form new exclusion ranges such that a value is within such new exclusion ranges if and only if said value is not within an exclusion range for said characteristic for at least an index that has said characteristic in its exclusion chain. Effectively, such new exclusion ranges for said characteristic tries to exclude sharp-enough edges from all of said different indices. One or more characteristics may be operated upon in this manner to generate new exclusion ranges. Such characteristics may form an exclusion chain. An exclusion calibrator may receive a command input that specifies a set of different indices along with the new index to store the resultant exclusion chain under. A new exclusion chain thus formed under a new index may be retrieved to apply on the exclusion filter 207 to exclude edge contributions from a number of objects already calibrated for to discover remaining objects in the scene.

The exclusion calibration works in iteration from the start of the calibrate command until either a certain "good enough" condition is met or no more characteristics remain. Then the entire chain of set of exclusion ranges may be stored in memory 2 808 along with the index value from the Λindex' input control signal or transmitted to the processor 112, 112', 112" or both. The "good enough" condition can be met in one of several ways. In one, the condition is met if the chain of exclusion ranges has three characteristics. In a second, the condition is met if, for example, 75% of edges that are not sharp-enough edges are eliminated after exclusion filtering. In a third, the condition is met if the number of sharp-enough edges outnumber the rest by three to one after exclusion filtering. "Good enough" condition may be anyone of the above, or combination thereof, or other reasonable condition that indicates that what remains after exclusion filtering are not dominated by edges other than sharp- enough edges.

Figure 19 is a flowchart for exclusion calculator 906 shown in Figures 17 and 18 to generate exclusion ranges for each characteristic. The characteristic's range is divided into bins, each bin covering a fraction of the range, and all bins together cover the entire range, and adjacent bins do not overlap except sharing the common boundary. Starting with the first bin at step 1002, the exclusion range is emptied. At step 1004, the count of sharp-enough edges is compared with the count of all edges. If the count of sharp-enough edges in this bin is either zero or less than a given fraction, e.g. three quarters, of the count of all edges in this bin, include this bin in the exclusion range. Repeat steps 1004 and 1006 until all bins are done.

An exclusion calibrator may create a separate set of exclusion ranges ("pointed exclusion ranges") by further limiting sharp-enough edges to be within a given region in the image. To do so, sharp-edge statistics is further tightened to exclude edges that lie outside the given region. The given region may be formed, for example, in Figure 16, from a selection region 1920 shown overlapping the displayed image in the display 114. The selection region 1920 indicates an object desired to be in sharp focus. Thus if there were a different object outside the selection region 1920 but at the same focus distance as the object within the selection region 1920, the different object does not interfere with the formation of exclusion ranges to reject edges not pertaining to the desired object. The exclusion calibrator may create and memorize a separate set of exclusion ranges corresponding to further limiting sharp-enough edges within the selection region. The given region may be formed, alternately, as a predetermined focusing region (not shown) in the center of the display of digital cameras and the viewfinder of cameras in general, often in rectangular shape or circular shape. The given region may be formed, alternately, by a touch-screen sensor overlapping the display 114. The touch-screen sensor may sense one or more strokes of touching by one or more of a stylus and fingers, for example, and provide such information as the positions of the touching to a processor to determine the corresponding selection region.

Although shown to reside together in the focus signal generator 120, one or more components of the exclusion calibrator 205 may reside instead in the processor 112, 112', 112". For example, the exclusion calculator 906 and the memory 908 may reside in the processor 112, 112', 112". In this configuration, the processor 112, 112', 112" receives all-edge and sharp-edge statistics (shown in Figure 17 and 18) via signals 134 for use in the exclusion calculator 906. The processor 112, 112', 112" may perform the process of the flowchart in Figure 19 and/or 20 entirely within itself or in conjunction with the focus signal generator 120, 120' .

Use with Exclusion

Exclusion calibrator 205 and exclusion filter 207 together may be used to form a chain of exclusion ranges to eliminate edges that do not belong to the object in sharp focus in one image captured at one moment in time. The same chain of exclusion ranges may subsequently be used to eliminate those other edges even though the object has moved relative to the background objects. Doing so allows the edges of the object to continue to dominate the focus signal. Exclusion filtering may be operated whether or not width filter 209 is bypassed by fine switch 220 in Figure 3. If width filter 209 is bypassed, i.e. focus signal is gross focus signal, those two outlier peaks in Figure 11 at edge width of 4.0 and 5.0 may be eliminated so that even if the previously in-focus object has moved out of focus so much that the associated peak 720 moves outside of the narrow-edge range, the gross focus signal can provide a good feedback signal to the focus control system to estimate how much to adjust focus position to bring the object of interest back into the narrow-edge range where fine focus signal can provide a more accurate feedback control signal to finally bring the object into sharp focus .

Re-calibration may be performed at any time, for example during tracking of an object of interest, by issuing a "calibrate" command to the exclusion calibrator 205. The processor 112, 112', 112" may provide the same index to the exclusion calibrator 205 as the index under which the existing calibrated exclusion ranges for the object of interest is stored. The index may be the index used to retrieved the set of exclusion ranges and chain currently in use in the calibration filter 207. Issuing the same index as an existing index will replace the previous calibrated exclusion ranges and chain that correspond to the existing index with a new set of exclusion ranges and chain.

A re-calibration may be performed when a selection region is created on and/or around the object of interest to limit sharp-enough edges to within the selection region only, as discussed above. The "pointed exclusion ranges" or "pointed exclusion chain" may be sent to and be used at the exclusion filter 207 instead of non-pointed exclusion ranges or chain, e.g. when the processor 112 receives a user command providing a given region in one of the abovementioned manners.

Re-calibration may be performed during movement of the object of interest within the scene causing one or more image characteristics pertaining to the object to change. For example, the user may command a given region on and/or around the object of interest as soon as the object of interest is no longer highlighted or is losing more and more of the highlight that indicates sharp edges.

For re-calibration, instead of providing the same index, the processor 112 may provide a new index to store a new set of exclusion ranges and chains. The re-calibrated exclusion ranges and chain may be stored under a new index without over-writing the existing index and its associated exclusion ranges and chain. The exclusion ranges and chains from the new and the existing indices may be compared to identify which characteristic (s) change, and which do not. The characteristics that do not change from the existing index to the new index may be promoted in the new exclusion chain. The characteristics that change may be eliminated from the chain. Thus characteristics that remain unchanged over time are retained for tracking the same object over time.

For an object of interest nearly in sharp focus such that it produces edges within the sharp-edge range, exclusion calibrator 205 and exclusion filter 207 help to eliminate edges contributed by out-of-focus objects so that contribution to the fine focus signal (for which width filter is not bypassed) is dominated by edges contributed by the object of interest to give an improvement in the accuracy of fine focus signal as an indication of focus sharpness .

When a selection region 1920 is created on and/or around the object of interest, the focus signal generator 120 may give preference in the calculation of the focus signal to edges that correspond to the selection region 1920 ("preferred edges") only under a control (not shown) of the processor 112. Preference to preferred edges may be given in one of several ways. The image provided to the focus signal generator 120 on bus 130 may be limited to within the selection region 1920. Or a weight may be applied on edges that enter the focus signal calculation such that edges outside and/or farther from the selection region 1920 have lesser weight than preferred edges, or even zero (i.e. equivalent to being excluded from focus signal calculation altogether) . Or edges that are not preferred edges may be eliminated at any one of the units within the focus signal generator 120, such as the edge detection & width measurement unit 206, the width filter 209, and the exclusion filter 207. Ways to give preferences to intra-region edges are not limited to the above. Nor is there a need for a rigid or sharp distinction between inside and outside of the selection region 1920. One skilled in the art can readily appreciate that it suffices as long as a plurality of edges are selected to correspond to the selection region and to receive such preference against a plurality of edges outside the selection region.

When a selection region 1920 is created on or around an object of interest, the processor 112 may control the focus signal generator to generate a gross focus signal that applies a preference to preferred edges. Doing so can eliminate edges not pertaining to the object of interest. For example, in Figure 14 is shown a gross focus signal across a sweep of focus position from far to near without applying such preference. The gross focus signal exhibits multiple minima but only one minimum, at position FB, is due to the car. By applying such preference for the car, such as by using the selection region 1920 shown in Figure 16, the minima corresponding to the foreground "person", at position FC, and the background "mountains, sun & horizon", at position FA, can be eliminated. In addition, the gross focus signal level at position FB, corresponding to the "car", will have a value closer to the sharp-edge width. The focus control system 102 may control the focus position of the focus lens 104 to reduce the difference between the difference between the focus position FB to the current focus position by evaluating the difference between the current gross focus signal level and the sharp_edge_width by applying one of the known control methods, PI- controller. The focus control system 102 may switch to use fine focus signal when the current focus signal level falls within a range close enough to the sharp_edge_width, such as the narrow-edge range.

Control - Scanning

Figure 25 is a flowchart for a focus system controller such as processor 112 of Figures 1 and 2 to adjust focus position and measure focus signal when moving the focus position across its range from one end to the other in search of focus positions that render sharp images of objects in the scene. At step 1400, the focus position is set to the far end. All indices for the exclusion calibrator 205 of Figure 3 may be cleared, along with the exclusion range data in memory 2 808 of Figures 17 and 18. At step 1402, the focus position is moved one step closer. At step 1404, if the focus position is the near end, scanning is completed. If not, at step 1406, check to see if narrow-edge count (from focus signal generator's width filter 209, Figure 3) peaks. If not, go to the next iteration, starting back at step 1402. If yes, create a new index for this focus position, record the focus position, send calibrate command to the exclusion calibrator 205 of Figure 3 to gather edge statistics, form exclusion ranges and chains, and send the index and a write command to store the chain of exclusion ranges, then proceed to the next iteration, starting back at step 1402.

Distinct characteristics and their associated exclusion ranges may be formed during the scanning process for each focus position where sharp images are found. In a subsequent search for any of these objects, the exclusion ranges may be applied to the focus signal generator to help "tune out" edges contributed by the other objects so that the gross focus signal may exhibit a single trough and further the trough has a minimum value that is close to the sharp_edge_width, and the fine focus signal is also free of interference from out-of-focus objects enough that its departure from the sharp_edge_width gives an accurate estimate of how much to adjust the focus position to bring the object into sharp focus. Both remain useful and reliable even if the object of interest moves relative to the other objects.

As an illustration, Figure 13 is a graph of narrow-edge count plotted against time when scanning the scene illustrated in Figure 12. The narrow-edge count peaks whenever focus position is such that an object projects a sharp image on the pixel array 108 in Figures 1 or 2, thus its peaking signifies presence of sharp-enough edges. Figures 26, 27, and 28 illustrate the images captured when the focus lens is at each of those focus position where narrow-edge count peaks.

Figure 14 shows the gross focus signal output by the focus signal generator 120 during the scan when the exclusion filter 207 is bypassed by the 'exclude' signal being OFF. The gross focus signal correctly exhibits a minimum corresponding to the times when the focus position is near each of the three focus positions that brings one object into sharp focus. However, the gross focus signal's minimum is not at the sharp_edge_width level, which is 2.0 in this example, due to the interference from blurred edges contributed by the other objects that are out-of-focus . As aforementioned, turning on 'exclude' signal after having calibrated the exclusion calibrator 205 will subsequently allow the gross focus signal to exhibit a single global minimum for each of the 3 groups of objects by retrieving exclusion ranges using the index for each group of object, even if they have moved from where they were during scanning. Figures 29, and 30A-30C illustrate how exclusion ranges for hue for each of those three groups of objects may be formed.

Search — Display & User Direction

Figure 31-33 illustrate how edges may be highlighted on the display 114 of Figure 1 or 2 when the focus position bring them into sharp focus. Highlighting sharp edges or sharp enough edges is especially helpful to the user when the image capture resolution is far higher than the display resolution such that an insufficiently sharply focus edge still appears to be sharp on the display. It also solves a common problem wherein the user is not able to tell which object among several objects in the scene is in sharp focus. Having been thus informed of which objects are in sharp focus, the user can then press either the outward button 1912 of the image capture apparatus 102 of Figure 16 to focus on a farther object, or inward button 1910 to focus on a nearer object. The image capture apparatus 102 can display a blinking outline 2100 overlapping the object whose edges are to be focused on, as shown in Figure 33.

Search Mode

Figure 34 is a flowchart for seeking to the Nth object in a given direction. At step 2200, the focus lens 104 moves until N-I peaks are skipped. Peak skipping can be detected using one of several signals for peak detection ("peak-detection signal") . One candidate peak-detection signal is gross focus signal, which as Figure 36 shows attains a local minimum at each position where there are objects producing sharp edges in the captured image. For the scene of Figure 12, to bring the car into sharp focus, the focus lens will move outwards from the innermost position to skip focus position FC where the person is in sharp focus, past the focus position where the gross focus signal reaches a local minimum (that nearly reaches the sharp_edge_width value of 2.0) . Anpther candidate peak- detection signal is the narrow-edge count. Figure 35 shows narrow-edge count as focus position moves outward from the innermost position past the position where the person produces a sharp image in the captured image. At step 2202 of the flowchart in Figure 34, unless having found the Nth peak, continue to move focus position in the same direction, at step 2206, and return to the same query at 2202. If found, move to step 2204 to turn on tracking mode. Detection of the Nth peak can use one of several peak-detection signals. One candidate peak- detection signal is the gross focus signal. Another candidate is the narrow-edge count. A different peak- detection signal may be used for step 2202 than step 2200. Tracking may be turned on after the peak is detected, as shown in Figure 35 and 36 following the flowchart of Figure 34, or when approaching and near enough to the peak value, as shown in Figure 37, following flowchart of Figure 38. ("Near enough" may be exceeding a predetermined fraction of the prior detected peak value, e.g. from a prior scan.)

User Control on Search

The number N of the focus position to seek to may be established in one of several ways.

In a first method, the apparatus 102 may provide a user control, such as buttons 1912 and 1910, shown in Figure 16, to navigate outward or inward among sharp focus positions. For example, each pressing of the button 1912 may select the next farther sharp focus position, whereas each pressing of the button 1910 may select the next nearer sharp focus position. The apparatus 102 may display a highlight such as highlight 2000 shown in Figure 31 on a sharp edge or a sharp object. The sharp focus positions may be memorized during a scan. The highlight 2000, or the corresponding sharp edges, or both, at each sharp focus position found during the scan may be formed and memorized in a memory, such as within the processor 112, 112', 112", during the scan. The displayed image at each sharp focus position may be memorized along. During a navigation to a target focus position, such as by using buttons 1912 and 1910 to jump farther and nearer among sharp focus positions, the highlight that corresponds to the selected sharp focus position may be retrieved from the memory and displayed on the display 1902. Along with the highlight 2000, the previously memorized displayed image that corresponds to the selected sharp focus position may be displayed as well to show what object in the memorized display image gave rise to the sharp edges at the selected sharp focus position. With this aid, the user may decide whether the selected sharp focus position will render his or her intended ^■ target sharp, or he or she needs to navigate farther or nearer. Thus, even with several different objects at different focus distances giving rise to edges overlapping within the image in the display 1902 in Figure 16, for example the head of the person and the left end of the car, the user is able to select the target object. The processor 112 thus determines the user's selection of the target sharp focus position and its associated memorized sharp focus position or exclusion chain or index or any combination thereof. If the target sharp focus position is the nearest, found from the scan, N is 1. If the one immediately after, N is 2, and so on.

In a second method, a prior scan is not assumed. Using the same user interface as described in the previous paragraph, the user may navigate farther or nearer by N number of steps. For example, the user may press button 1912 three times to demand an outward search from the current focus position past a first sharp focus position followed by a second sharp focus position to stop at a third sharp focus position. The focus system controller 112 commands the focus lens 104 to move outwards to the third sharp focus position by following the flowchart in Figure 34, having N initialized to 3. If the user subsequently demand to change the target sharp focus position to the second sharp focus position by pressing button 1910 once, the focus system controller 112 commands the focus lens 104 to move to the second sharp focus position by following the flowchart in Figure 34, but moving the focus position in the inwards direction this time, having N initialized to 1. The apparatus 102 may display highlight 2000 on sharp edges each time a sharp focus position is found at the final destination, or including in passing.

Either of the above methods is useful to deal with difficult focusing situation in which edges from objects at different focus distances overlap. One example of such situation is chicken wire and the chicken behind the chicken wire. If the focus position has the chicken wire in sharp focus whereas the chicken is the desired object in focus, the user can navigate outwards one step, for example by pressing button 1912 once. In response, the focus system controller 112 commands the focus lens 104 to move outwards until the next sharp focus is detected, which is the sharp focus position for the chicken. On the other hand, if the focus position has the chicken in sharp focus whereas the chicken wire is the desired object in focus, the user can navigate inwards one step, for example by pressing button 1910 once. In response, the focus system controller 112 commands the focus lens 104 to move inwards until the next sharp focus is detected, which is the sharp focus position for the chicken wire. In either case, the sharp object or its edges may be highlighted on the display 1902 each time an object is brought into sharp focus to help the user determine which object is in sharp focus.

Highlighting on a display an object or its edges in sharp focus, providing a user interface to initiate and/or give a direction to seek to a next focus position to bring a farther or nearer object into sharp focus, seeking to said next sharp focus position upon a user entering a corresponding command via said user interface, and highlighting objects or their edges brought into sharp focus during transiting to the final focus position are steps useful to the user of the apparatus 102 regardless of how the focus signal is formed or how the object or its edges in sharp focus is determined. The focus signal may be generated by measuring widths of edges in an image as in this application. Alternately, the apparatus 102 may measure a contrast across an edge and detecting sharp focus of the edge by detecting a peaking in said contrast when focus position of the focus lens 104 is changed. Alternately, the apparatus 102 may generate a focus signal by dividing the image into groups of adjacent pixels, e.g. each group having 16-by-16 pixels, a contrast measured within each group as a difference between a maximum value and a minimum value of a signal that represents visible light reflecting from the scene, e.g. luminance, and detecting sharp focus within said group of pixels by detecting a peaking in said contrast when focus position of the focus lens 104 is changed, and highlighting a portion of the image displayed in display 1902 corresponding to said group of pixels. Alternately, the apparatus 102 may generate a focus signal using any conventional method. Tracking Mode

The focus system controller, such as processor 112 in Figure 1 or 2, may turn on the ^Λfine' control signal to the focus signal generator 120 to receive a fine focus signal when tracking mode starts. It may also turn on the Λexclude' control signal and provide the corresponding index to an exclusion calibration 205 within the focus signal generator 120 to retrieve the corresponding exclusion ranges, if the object nearly in-focus has been previously calibrated. If not, it can issue a calibrate command to the exclusion calibration 205 to perform a calibration and provide a new index to store the calibrated exclusion ranges, and subseguently retrieve them for use by the exclusion filter 207. Doing so improves the accuracy of the fine focus signal by greatly attenuating the interference from unintended objects.

The focus system controller may adjust the focus position to keep the fine focus signal nearest possible to the minimum, as shown in Figure 39, or it can pursue an alternate strategy of controlling the focus position to keep the fine focus signal an offset away from the sharp_edge_width, as shown in Figure 40 and 41. (sharp_edge_width is labeled as h_FB in Figures 40-41.) Maintaining fine focus signal at an offset above sharp_edge_width has a benefit in that a slight change in the fine focus signal due to any movement in the object gives off clue as to whether the object is moving closer or farther. If the focus position is farther than the ideal focus position that would have brought the fine focus signal to the minimum (hence the fine focus signal level is higher than the minimum), a movement in the object that reduces the fine focus signal must be directed outwards, i.e. closer to the scene location where the current focus position would make sharp image. On the other hand, if a movement in the object increases the fine focus signal, the object must be moving inwards, i.e. away from that scene location. The focus system controller then can immediately move the focus position in the direction thus determined. This is shown in Figure 41 where the fine focus signal is shown to always return to the same level after a small departure, whereas in Figure 39 where the focus system controller targets the fine focus signal to the minimum is unable to immediately determine the direction of movement of the object and may make a mistake when the object starts moving, causing the initial increase in fine focus signal to be more rapid, and only after that to realize the mistaken guess of direction and to correct itself. In Figure 39, the subsequent ripples are maintained as small as Figure 41 on the assumption that the object continues to move in the same direction, hence the focus system controller does not make a second mistake in the direction. But if the object changes direction, the focus system controller that targets the minimum focus signal level can make a mistake again. In general, the focus system controller that pursues minimizing the fine focus signal cannot cope with movement of an object that varies in velocity, whereas the focus system controller that targets the fine focus signal to an offset away from the minimum can do well.

Use with Exclusion Illustrated

Figures 42A-42C illustrate an ability of the focus control system to track an object (the bird) that moves relative to background objects. In Figure 42A, the bird is in sharp focus, and the focus system controller issues a calibrate command to the exclusion calibrator 205 and turns ^Λfine' control signal ON to generate fine focus signal and 'exclude' control signal ON to exclude the mailbox, which is not sharp in Figure 42A. The exclusion calibrator 205 finds two characteristics, namely the length of concatenated edges and slant angles, whose exclusion ranges are shown in Figures 43 and 44, respectively, to exclude the mailbox from the bird.

Next, in Figure 42B, the bird flies near and rests on the mailbox. Between Figure 42A and 42B, the focus control system continues to track the bird to keep it in sharp focus. When the bird comes to rest on the mailbox, edges contributed by the mailbox are also sharp-enough edges, but they are excluded by the exclusion filter 207 by using two characteristics, namely the length of concatenated edges and slant angles, whose exclusion ranges are shown in Figures 43 and 44, respectively. The fine focus signal continues to indicate the sharpness of edges contributed by only the bird, and not the mailbox. Finally, when the bird flies away, the focus control system continues to track the bird to maintain its image in sharp focus and not getting falsely-locked onto the mailbox, which by the sheer larger number of edges would have otherwise overwhelmed the fine focus signal and falsely tell the focus control signal that the object of interest remains at the same focus distance.

One advantage of the focus signal generator is that the minimum focus signal of different images are at approximately the same values. This ensures that the lens will remain in the same position even if the camera is shaking but the image still remains sharp. Another advantage is that the range between the largest and smallest focus signal for a scene with different focal distances are wide enough to ensure that the optimum focus can be obtained.

A further advantage is that the focus signal value is not dependent on illumination nor on color or reflectivity of objects in the scene. Conventional contrast method of generating a focus signal produces contrast signal which is strongly dependent on all said factors, such that when a new object enters the scene even though at the sharp focus distance, the focus signal produced by the contrast method usually changes.

Figure 45 shows an alternate embodiment of a focus signal generator 120'. Focus signal generator 120' outputs statistics of edges and edge widths. Among the edge-width statistics that controller 120' outputs may be one or more of the following: an edge-width histogram comprising edge counts at different edge widths; an edge width where edge width count reaches maximum; a set of coefficients representing a spline function that approximates edge counts at different edge widths; and any data that can represent a function of edge width. Census Unit 240 may receive data computed in one or more of the other units with the focus signal generator 120' to calculate statistics of edge widths.

Referring to Figure 46, the edge-width statistics thus provided in signals 134 to an alternate embodiment of processor 112' in an alternate auto-focus image pickup apparatus 102' may be used by the processor 112' to compute a gross and/or fine focus signal and a narrow-edge count in accordance with methods discussed above or equivalent thereof. In addition, any data computed in the focus signal generator 120' may be output to the processor 112' as part of the output signals 134. For example, the exclusion ranges and chains, along with statistics of edges, such as statistics of edge characteristics used for calculating the exclusion ranges and chains according to the flowcharts of Figure 19 or Figure 20, may be sent to the processor 112' .

The processor 112' may internally generate a focus signal and/or a narrow-edge count in addition to the functions included in the processor 112 of Figure 1.

The pixel array 108, A/D Converter 110, color interpolator 148, and generator 120' may reside within a package 142, together comprising an image sensor 150' , separate from the processor 112' .

Yet another embodiment of a focus signal generator may add a census unit 240 to the generator 102 of Figure 1 and output one or more statistics calculated in such a generator to the processor 112.

Auxiliary Pixel Array

Figure 47 shows an alternate embodiment of an auto- focus image pickup system 103. In addition to elements included in a system 102, the system 103 may include a partial mirror 2850, a full mirror 2852, an optical lowpass filter 2840, a main pixel array 2808, and a main A/D Converter 2810. The partial mirror 2850 may split the incoming light beam into a first split beam and a second split beam, one transmitted, the other reflected. The first split beam may further pass through the optical lowpass filter 2840 before finally reaching the main pixel array 2808, which detects the first split beam and converts to analog signals. The second split beam may be reflected by the full mirror 2852 before finally being reaching the auxiliary pixel array 108", which corresponds to the pixel array 108 in system 102 shown in Figure 1. The ratio of light intensity of the first beam to the second beam may be 1-to-l or greater than 1-to-l. For example, the ratio may be 4-to-l.

The main pixel array 2808 may be covered by a color filter array of a color mosaic pattern, e.g. the Bayer pattern. The optical lowpass filter 2808 prevents the smallest light spot focused on the pixel array 2808 from being too small as to cause aliasing. Where a color filter of a mosaic pattern covers the pixel array 2808, aliasing can give rise to color moire artifacts after a color interpolation, . For example, the smallest diameter of a circle encircling 84% of the visible light power of a light spot on the main pixel array 2808 ("smallest main diameter") may be kept larger than one and a half pixel width but less than two pixel widths by use of the optical lowpass filter. For example, if the main pixel array 2808 has a pixel width of 4.5um, whereas the smallest diameter is 2. Oum without optical lowpass filtering, the optical lowpass filter 2840 may be selected to make the light spot 6.7um or larger in diameter.

The auxiliary pixel array 108" may comprise one or more arrays of photodetectors . Each of the arrays may or may not be covered by a color filter array of a color mosaic pattern. The array(s) in auxiliary pixel array 108" outputs image (s) in analog signals that are converted to digital signals 130 by A/D Converter 110. The images are sent to the focus signal generator 120. A color interpolator 148 may generate the missing colors for images generated from pixels covered by color filters. If auxiliary pixel array 108" comprises multiple arrays of photodetectors, each array may capture a sub-image that corresponds to a portion of the image captured by the main pixel array 2808. The multiple arrays may be physically apart by more than a hundred pixel widths, and may or may not share a semiconductor substrate. Where the pixel arrays within auxiliary pixel array 108" _do not share a semiconductor substrate, they may be housed together in a package (not shown) .

Main A/D Converter 2810 converts analog signals from the Main. Pixel Array 2808 into digital main image data signal 2830, which is sent to the processor 112, where the image captured on the Main Pixel Array 2808 may receive image processing such as color interpolation, color correction, and image compression/decompression and finally be stored in memory card 116.

An array of photodetectors in the auxiliary pixel array 108" may have a pixel width ("auxiliary pixel width") that is smaller than a pixel width of the main pixel array 2808 ("main pixel width") . The auxiliary pixel width may be as small as half of the main pixel width. If an auxiliary pixel is covered by a color filter and the auxiliary pixel width is less than 1.3 times the smallest spot of visible light without optical lowpass filtering, a second optical lowpass filter may be inserted in front of the auxiliary array 108" to increase the smallest diameter on the auxiliary pixel array 108" ("smallest auxiliary diameter") to between 1.3 to 2 times as large but still smaller than the smallest main diameter, preferably 1.5. The slight moire in the auxiliary image is not an issue as the auxiliary image is not presented to the user as the final \ captured image.

Figure 50 illustrates how edge widths may vary about a sharp focus position for main images from the main pixel array 2808 (solid curve) and auxiliary images from the auxiliary pixel array 108" (dashed curve) . The auxiliary images give sharper slopes even as the main images reach the targeted sharp edge width of 2. The auxiliary image is permitted to reach below the targeted sharp edge width, since moire due to aliasing is not as critical in the auxiliary image, as it is not presented to the user as a final image. This helps to sharpen the slope below and above the sharp edge width. The sharper slope is also helped by the auxiliary pixel width being smaller than the main pixel width.

The shaded region in Figure 50 indicates a good region within which to control the focus position to keep the main image in sharp focus. A change in focus position outwards will cause the edge width to increase in the auxiliary image, whereas a change inwards will cause the it to decrease. To maintain the main image's edge widths near the sharp edge width, a linear feedback control system may be employed to target the middle auxiliary edge width value within the shade region and to use as feedback signal the edge widths generated from the auxiliary images.

The auxiliary pixel array 108", A/D Converter 110, focus signal generator 120 together may be housed in a package 142 and constitute an auxiliary sensor 150. . The auxiliary sensor 150 may further comprise a color interpolator 148.

Figure 48 shows an alternate embodiment of auto-focus image pickup apparatus 103' similar to apparatus 103 except focus signal generator 120' replaces focus signal generator 120. The auxiliary pixel array 108", A/D Converter 110, focus signal generator 120' together may be housed in a package 142 and constitute an auxiliary sensor 150' . The auxiliary sensor 150 may further comprise a color interpolator 148.

Figure 49 shows an alternate embodiment of auto-focus image pickup apparatus 103". The focus signal generator 120 and the processor 112" may be housed in a package 144 as a camera controller, separate from the auxiliary pixel array 108". The processor 112" is similar to processor 112 except that processor 112" receives images from the main pixel array 2808 as well as the auxiliary pixel array 108". The processor 112" may perform a color interpolation, a color correction, a compression/decompression, and a storing to memory card 116 for the images received on signal 2830 similar to the processing that the processor 112 may perform on signal 130 in Figure 2. Unlike in Figure 2, here the images received on signal 130 need not receive compression/decompression and storing to memory card 116. The processor 112" may perform color interpolation on images received on signal 130 for pixels that are covered by color filters in the auxiliary pixel array 108" and send the color interpolated images to the focus signal generator 120 on signal 146.

The auto-focus image pickup system 102, 102', 103, 103' , 103" may include a computer program storage medium (not shown) that comprises instructions that causes the processor 112, 112', 112" respectively, and/or the focus signal generator 120, 120' to perform one or more of the functions described herein for detecting sharp focus and/or controlling a focus position. By way of example, the instructions may cause the processor 112 or the generator 120' to perform a slant correction for an edge width in accordance with the flowchart of Figure 7. As another example, the instructions may cause the processor 112' or the generator 120 to perform an edge width filtering in accordance with the above description for Width Filter 209. Alternately, the processor 112, 112' or the generator 120, 120' may be configured to have a combination of firmware and hardware, or a pure hardware implementation for one or more of the functions contained therein. For example, in generator 120, a slant correction may be performed in pure hardware and a length filter 212 performed according to instructions in a firmware.

While a memory card 116 is shown as part of system 102, any nonvolatile storage medium may be used instead, e.g. hard disk drive, wherein images stored therein are accessible by a user and may be copied to a different location outside and away from the system 102.

While certain exemplary embodiments have been described and shown in the accompanying drawings, it is to be understood that such embodiments are merely illustrative of and not restrictive on the broad invention, and that this invention not be limited to the specific constructions and arrangements shown and described, since various other modifications may occur to those ordinarily skilled in the art .

Claims

What is claimed is:

1. An auto focus image system, comprising: an pixel array that captures an image that has a plurality of edges each having an edge width; at least one filter that discriminates a filtered edge based on a characteristic of the filtered edge; and, a generator coupled to said pixel, array, said generator generates a focus signal that is a function of said edge widths.

2. The system of claim 1, wherein said generator includes an edge detection and width measurement unit.

3. The system of claim 2, wherein said generator includes a length filter.

4. The system of claim 1, further comprising a lens that adjusts a focus in response to said focus signal.

5-19. <INTENTIONALLY LEFT BLANK>

20. A method for generating a focus signal, comprising: capturing an image that has a plurality of edges each having an edge width, the image captured using a first pixel array; providing at least one filter that discriminates a filtered edge based on a characteristic of the filtered edge; generating a focus signal that is a function of the edge widths and the discrimination.

21. The method of claim 20, further comprising varying a focus lens position in response to the focus signal.

22. The method of claim 20, wherein the generation of the focus signal includes calculating a plurality of gradients of a plurality of image pixels.

23. The method of claim 22, further comprising refining the edge width by removing gradients that are below a threshold that is a fraction of a peak gradient.

24. The method of claim 23, further comprising finding a fractional pixel position by interpolating between a first gradient above said threshold and a second gradient below said threshold.

25. The method of claim 24, further comprising subtracting between two consecutive fractional pixel positions.

26. The method of claim 22, further comprising correcting the edge width for a slant from the vertical or horizontal line.

27. The method of claim 26, wherein the slant is calculated from a first maximal gradient and a second maximal gradient in a direction perpendicular to the first maximal gradient .

28. The method of claim 20, wherein at least one of the edge widths is weighted by a weight from the filter.

29. The method of claim 20, wherein the characteristic is a length of a concatenated edge, wherein a filtered edge that belongs to a shorter concatenated edge is discriminated against.

30. The method of claim 20, wherein the characteristic is an edge width, wherein edge widths outside a given region about a sharp edge width are discriminated against.

31. The method of claim 20, wherein the characteristic is a slant of the filtered edge.

32. The method of claim 20, wherein the characteristic is an average brightness or a maximum brightness or a difference of brightness across the filtered edge.

33. The method of claim 20, wherein the characteristic is a hue on at least one side of the filtered edge.

34. The method of claim 20, wherein the characteristic is a spacing of the filtered edge from an adjacent edge.

35. The method of claim 21, further comprising tracking by a feedback control to maintain the focus signal at an offset above a minimum of the focus signal.

36. The method of claim 35, further comprising determining a direction of movement of an imaged object from an increase or decrease of the focus signal.

37. The method of claim 20, further comprising generating a second image using a second pixel array different from the first pixel array, the second image being stored in a nonvolatile storage medium and accessible by a user for copying to a different location away from the system housing the first and second pixel arrays.

38. The method of claim 37, wherein the second pixel array has a larger pixel width.

39. The method of claim 37, wherein the second pixel array receives light simultaneously as the first pixel array, the second pixel array receiving less light.

40. The method of claim 35, further comprising entering a tracking mode that operates the tracking upon detecting a peak in a peak detection signal.

41. The method of claim 20, further comprising creating a exclusion range to discriminate against edge widths falling within the exclusion range.

42. The method of claim 21, wherein the exclusion range is created by comparing between sharp enough edges versus other edges in a prior image.

43. The method of claim 42, where the prior image is captured upon detection of a peaking in a peak detection signal.

41. The method of claim 40 or 43, wherein the peak detection signal is generated from the plurality of edge widths.

42. The method of claim 40 or 43, wherein the peak detection signal is generated from a count of edges within a range of edge widths about a sharp edge width.