CN108337425B - Image capturing device and focusing method thereof - Google Patents

Abstract

The present invention provides an image capture device and a focusing method thereof, which is applicable to an image capture device having an image sensor and a touch screen and includes the following steps. First, a real-time image is captured by using an image sensor, and the real-time image is displayed on the touch screen. Next, a touch operation on the touch screen is detected to select a region of interest associated with the touch operation from the real-time image. A first focus value of the region of interest is calculated based on the similarity of each pixel of the region of interest relative to at least one central pixel, and it is determined whether the first focus value meets the focus condition. When the first focus value meets the focus condition, the lens of the image sensor is controlled to move to a focus position corresponding to the first focus value.

Description

Image capture device and focusing method thereof

technical field

The present invention relates to an image capturing device, in particular to an image capturing device and a focusing method thereof.

Background technique

With the development of technology, all kinds of intelligent image capture devices, such as tablet computers, personal digital assistants and smart phones, have become indispensable tools for modern people. Among them, the camera lens of the high-end smart image capture device is already comparable to that of traditional consumer cameras, and can even be replaced. A few high-end models have pixels and image quality close to that of a digital monocular.

Generally speaking, the focusing operation behavior of the image capture device is to calculate the sharpness of the image image while moving the lens, so as to obtain the best focusing image. However, in a scene with rich background information, this focusing function often produces wrong focusing, so that a focusing image that is not expected by the user is captured.

SUMMARY OF THE INVENTION

In view of this, the present invention provides an image capture device and a focusing method thereof, which can reduce the failure of focusing and greatly improve user experience.

In an embodiment of the present invention, the above-mentioned focusing method of an image capture device is applicable to an image capture device having an image sensor and a touch screen, and includes the following steps. First, an image sensor is used to capture a real-time image, and the real-time image is displayed on the touch screen. Next, a touch operation on the touch screen is detected to select a region of interest associated with the touch operation from the real-time image. According to the similarity of each pixel of the region of interest with respect to at least one central pixel, a first focus value of the region of interest is calculated, and it is determined whether the first focus value meets the focus condition. When the first focus value meets the focus condition, the lens of the image sensor is controlled to move to a focus position corresponding to the first focus value.

In an embodiment of the present invention, the above-mentioned image capture device includes an image sensor, a touch screen, a memory, and a processor, wherein the image sensor includes a lens, and the processor is coupled to the image sensor, touch screen screen and memory. The processor uses the image sensor to capture the real-time image, displays the real-time image on the touch screen, and detects the touch operation on the touch screen, so as to select the region of interest related to the touch operation from the real-time image , and then calculate the first focus value of the region of interest according to the similarity of each pixel of the region of interest with respect to at least one central pixel, and determine whether the first focus value meets the focus condition. When the first focus value meets the focus condition, the processor will control the lens to move to a focus position corresponding to the first focus value.

Description of drawings

FIG. 1 is a block diagram of an image capturing apparatus according to an embodiment of the present invention.

FIG. 2 is a flowchart of a focusing method of an image capture device according to an embodiment of the present invention.

FIG. 3 is a schematic flowchart of a focusing method of an image capture device according to an embodiment of the present invention.

FIG. 4 is a graph showing a relationship between a focus value and a lens position according to an embodiment of the present invention.

FIG. 5 is a schematic flowchart of a focusing method of an image capture device according to an embodiment of the present invention.

Description of reference numbers:

100: image capture device;

10: image sensor;

15: lens;

20: touch screen;

30: memory;

40: processor;

S202～S214: the flow of the focusing method;

302, 502: instant image;

ROIa, ROIb: region of interest;

Ca, Cb: touch position;

304, 504: all pixels of the region of interest;

306a, 306b, 306, 506a, 506b, 506: weight distribution map;

C1, C2: the relationship between the focus value and the lens position;

Pf: the focus position of the foreground object;

Pb: The focus position of the background area.

Detailed ways

In order to make the above-mentioned features and advantages of the present invention more obvious and easy to understand, the following embodiments are given and described in detail with the accompanying drawings as follows.

Some embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Element symbols quoted in the following description will be regarded as the same or similar elements when the same element symbols appear in different drawings. These examples are only a part of the invention and do not disclose all possible embodiments of the invention. Rather, these embodiments are merely exemplary of the methods and apparatus claimed in the present invention.

FIG. 1 is a block diagram of an image capturing apparatus according to an embodiment of the present invention, which is only for convenience of description and is not intended to limit the present invention. First of all, FIG. 1 introduces all the components and the configuration relationship of the image capture device. The detailed functions will be disclosed together with FIG. 2 .

Referring to FIG. 1 , the image capture device 100 includes an image sensor 10 , a touch screen 20 , a memory 30 and a processor 40 . In this embodiment, the image capture device 100 is, for example, a digital camera, a single-lens camera, a digital video camera, or other devices with an image capture function, such as a smart phone, a tablet computer, a personal digital assistant, a tablet computer, a head-mounted display, and the like. Inventions are not limited to this.

The image sensor 10 includes a lens 15 , an actuator (not shown) and a photosensitive element (not shown), wherein the lens 15 includes a lens. The actuator can be, for example, a stepping motor (stepping motor), a voice coil motor (VCM), a piezoelectric actuator (piezoelectric actuator) or other actuators that can mechanically move the lens, which is not covered in the present invention this limit. The photosensitive elements are used to respectively sense the intensity of the light entering the lens, and then generate images respectively. The photosensitive element may be, for example, a charge coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) element or other elements, which are not limited in the present invention.

The touch screen 20 is a display device integrating touch detection elements, and can provide display and input functions at the same time. The display device may be, for example, a liquid crystal display (LCD), a light-emitting diode (LED) display, a field emission display (FED) or other types of displays, but not limited thereto. The touch detection elements are arranged on the display device in series and in rows to detect the touch of the user's finger, palm or other objects on the touch screen 20 . The touch detection element may be, for example, a capacitive touch detection element, a surface acoustic wave touch detection element, an electromagnetic touch detection element, or a near-field imaging touch detection element, but is not limited thereto.

The memory 30 is used to store images and data, and can be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory ), hard disk or other similar device or a combination of these devices.

The processor 40 can be, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessors (microprocessors), digital signal processors (DSPs), programmable control devices, application specific integrated circuits (ASICs), programmable logic devices (PLDs), or other similar devices or combinations of these devices. The processor 40 is coupled to the image sensor 10 , the touch screen 20 and the memory 30 , and is used for performing the focusing process, so as to improve the quality of the image captured by the image capturing device 100 .

FIG. 2 is a flowchart of a focusing method of an image capture device according to an embodiment of the present invention, and the focusing method of the image capture device of FIG. 2 can be implemented by various elements of the image capture device 100 of FIG. 1 .

Please refer to FIG. 1 and FIG. 2 at the same time. First, the processor 40 of the image capture device 100 captures a real-time image using the image sensor 10 (step S202 ), and displays the real-time image on the touch screen 20 (step S204 ) . Specifically, the processor 40 can use the image sensor 10 to continuously capture the real-time image in front of the image capturing device 100, and at the same time display the captured real-time image on the touch screen 20 for the user to watch . In one embodiment, the processor 40 can control the lens 15 to move to the focus position by using a focusing technique such as hill climbing, so as to obtain a real-time image with maximum sharpness. Since the real-time image is only for the user to check the composition, the lens 15 can speed up the focus with a larger step distance. However, in another embodiment, the processor 40 may synchronously execute the next step S206 when the lens 15 has not yet completed focusing, which is not limited in the present invention.

Next, the processor 40 detects a touch operation on the touch screen 20 to select a region of interest (ROI) associated with the touch operation from the real-time image (step S206 ). Here, the image capturing device 100 can display the real-time image on the touch screen 20 to provide the user to intuitively select an area of interest for focusing. When the user performs a touch operation such as clicking, pressing, etc. on the touch screen 20, the processor 40 will detect the position of the touch operation, and center the position corresponding to the touch operation in the real-time image to set an area with a predetermined size as a region of interest. The predetermined size can be a system default value stored in the memory 30, and the user can also manually adjust it at any time. For example, the processor 40 can display the region of interest in the real-time image in the form of a frame and the like, and the user can use the touch screen 20 to perform a drag operation such as a drag operation to adjust the size of the region of interest.

Compared with the global search in the known focusing procedure, since the focusing area has been locked in the area of interest, the processor 40 can only perform the focusing procedure for the area of interest, so as to shorten the focusing time. Here, it can be reasonably assumed that the object corresponding to the position of the touch operation is the target that the user wants to focus on. Based on the region of interest is defined with the location of the touch operation as the center, so the center of the region of interest is the location of the target. Here, the processor 40 will calculate the first focus value of the region of interest according to the similarity of each pixel of the region of interest with respect to at least one central pixel (step S208 ), wherein the central pixel is the pixel located in the center of the region of interest , and the number of central pixels is not limited to a single one to avoid the possibility of noise or artifacts.

In this embodiment, the processor 40 will first calculate the similarity of each pixel of the region of interest with respect to the central pixel. From another viewpoint, the processor 40 calculates the similarity of each pixel of the region of interest with respect to the pixel corresponding to the user's touch operation. The similarity here can be represented by at least one of the image coordinate distance and the chromaticity coordinate distance between each pixel and the central pixel, wherein the chromaticity coordinate can be based on color spaces such as RGB, CMYK, etc., which is not limited in the present invention. Next, the processor 40 will assign a corresponding weight value to each pixel according to the similarity of each pixel relative to the central pixel, wherein the greater the similarity, the greater the assigned weight value. In other words, pixels that are adjacent to the target corresponding to the user's touch operation and have similar color characteristics will be given a larger weight value.

Specifically, a generally known focusing algorithm only sums up the contrast for all pixels in the focusing range, for example, the following formula (1):

where (i,j) is the image coordinate of the pixel in the region of interest, x is the position of the lens, FV _fg (x) is the sum of the contrast of the foreground area, FV _bg (x) is the sum of the contrast of the background area, and FV(x) is the contrast sum (ie, focus value) of the region of interest. On the other hand, the processor 40 of this embodiment simultaneously obtains the focus value of the region of interest (that is, the aforementioned "first focus value") according to the contrast and weight values of each pixel in the region of interest, for example, the following Formula (2) of:

where (i, j) is the image coordinate of the pixel of the region of interest, (i ₀ , j ₀ ) is the image coordinate of the central pixel, c is the chromaticity coordinate of the pixel whose image coordinate is (i, j), and c ₀ is The chromaticity coordinates of the central pixel, x is the position of the lens 15, C _fg (x) is the sum of the contrast of the foreground area, C _bg (x) is the sum of the contrast of the background area, and C(x) is the area of interest The sum of the contrasts (ie, the first focus value). Incidentally, the image coordinates of the central pixel here may be a single center point located in the region of interest, or may be any position in the center region of the region of interest. In addition, the chromaticity coordinates of the central pixel may be the chromaticity coordinates of a single central point located in the region of interest, or may be the average chromaticity coordinates of the central region of the region of interest.

After calculating the first focus value, the processor 40 will determine whether the first focus value meets the focus condition (step S210 ) to determine whether the lens position corresponding to the first focus value is a position that can generate a focus image. The focus condition here refers to whether the first focus value is the local maximum focus value. If so, the processor 40 will control the lens 15 to move to the focus position corresponding to the first focus value (step S212). If not, the processor 40 will obtain and control the lens 15 to move to a new focus position with the local maximum focus value (step S214 ), and the new focus position here will be the same as the lens position with the first focus value to ensure that the lens 15 can focus on the target that the user wants to focus on.

In order to facilitate the understanding of the aforementioned focusing method, practical applications will be listed below for more detailed description.

FIG. 3 is a schematic flowchart of a focusing method of an image capture device according to an embodiment of the present invention.

Please refer to FIG. 1 and FIG. 3 at the same time, the image 302 is the real-time image captured by the processor 40 . Since the image 302 has a relatively complex background area, in a general focusing procedure, the lens 15 will focus on the background area, and present a clearly focused background area and an out-of-focus blurred foreground object. At this time, it is assumed that the user performs a touch operation on the position Ca, where the position Ca is the position of the foreground object. The processor 40 will define the region of interest ROIa centered on the position Ca, and assign weight values to all the pixels 304 of the region of interest ROIa.

In this embodiment, the processor 40 will use a two-dimensional Gaussian function (Gaussian function) to assign the weight values of all the pixels 304, for example, the following formula (3):

Where c=(a, b) is the chromaticity coordinate of the pixel whose image coordinate is (i, j), and c ₀ =(a ₀ , b ₀ ) is the chromaticity coordinate of the central pixel of the image. In the weight assignment map 306a based on the similarity of the image coordinate distance, the area closer to the position Ca in the image 302 will be given a larger weight value. In the weight assignment map 306b based on the similarity of the chromaticity coordinate distance, the area that is closer in color to the position Ca in the image 302 will be given a greater weight. In the weight assignment map 306a and the weight assignment map 306b, a pixel with a lower pixel value represents a lower weight value. In this embodiment, the processor 40 can combine the weight allocation map 306a and the weight allocation map 306b to generate the weight allocation map 306 in a manner of averaging the two, which can effectively assign a higher weight value to the foreground object, while The background area is given a low or nearly 0 weight value.

Taking the curve relationship between the focus value and the lens position shown in FIG. 4 according to an embodiment of the present invention as an example, taking the curve C1 generated by the known focus procedure as an example, the lens will move to the focus position Pb in the background area . However, for the curve C2 generated by the processor 40 after assigning the weights based on the similarity, the lens 15 will move to the focus position Pf of the foreground object to capture a clearer picture of the foreground object.

FIG. 5 is a schematic flowchart of a focusing method of an image capture device according to another embodiment of the present invention.

Please refer to FIG. 1 and FIG. 5 at the same time, the image 502 is the real-time image captured by the processor 40 . At this time, it is assumed that the user performs a touch operation on the position Cb, where the position Cb is the position of the background area. The processor 40 will define the region of interest ROIb centered on the position Cb, and will assign weight values to all the pixels 404 of the region of interest ROIb. In this embodiment, the processor 40 can also use the formula (3) to assign the weight values of all the pixels 504 . In the weight assignment diagram 506a based on the similarity of the image coordinate distance, the area closer to the position Cb in the image 502 will be given a larger weight value. In the weight assignment diagram 506b based on the similarity of the chromaticity coordinate distance, the area that is closer in color to the position Cb in the image 502 will be given a greater weight. In the weight assignment map 506a and the weight assignment map 506b, a pixel with a lower pixel value represents a lower weight value. In this embodiment, the processor 40 can combine the weight assignment map 506a and the weight assignment map 506b to generate the weight assignment map 506 in a manner of averaging the two, which can effectively give a higher weight value to the background area, and Foreground objects are given a low or near-zero weight. The processor 40 will control the lens 15 to move to the focus position in a manner similar to FIG. 4 to capture a picture with a clearer background area.

To sum up, the image capture device and the focusing method thereof proposed by the present invention define the region of interest by the user's touch operation on the real-time image, and perform pixel similarity calculation for the region of interest to calculate More precise focus value. In addition to effectively accelerating focusing, the present invention can also reduce the failure of focusing, so as to greatly improve user experience.

Although the present invention has been disclosed as above with examples, it is not intended to limit the present invention. Any person skilled in the art can make some changes and modifications without departing from the spirit and scope of the present invention. The protection scope of the present invention shall be determined by the claims defined.

Claims (9)

1. A focusing method of an image capturing device is suitable for the image capturing device with an image sensor and a touch screen, wherein the image sensor comprises a lens, and the focusing method comprises the following steps:

capturing a real-time image by using the image sensor, and displaying the real-time image on the touch screen;

detecting a touch operation on the touch screen to select an interested area related to the touch operation from the instant image;

calculating a first focus value of the region of interest according to the similarity of each pixel of the region of interest relative to at least one central pixel;

judging whether the first focusing value meets the focusing condition; and

when the first focus value accords with the focusing condition, controlling the lens to move to a focusing position corresponding to the first focus value;

wherein the step of calculating the first focus value of the region of interest comprises:

calculating the similarity of each of the pixels with respect to the central pixel;

calculating a weight value corresponding to the similarity of each pixel, wherein the greater the similarity is, the greater the weight value corresponding to the similarity is; and

and obtaining the first focus value of the interested region according to the contrast of each pixel and the weight value.

2. The focusing method of claim 1, wherein the step of selecting the region of interest associated with the touch operation from the live image comprises:

and setting a region with a preset size as the region of interest by taking the position corresponding to the touch operation in the real-time image as the center.

3. The focusing method of claim 1, wherein the similarity of each pixel with respect to the central pixel is associated with an image coordinate distance between the pixel and the central pixel.

4. The focusing method of claim 1, wherein the similarity of each pixel with respect to the central pixel is related to a chromaticity coordinate distance between the pixel and the central pixel.

5. The focusing method of claim 1, wherein the step of calculating the weight value corresponding to the similarity of each pixel comprises:

and assigning the weight value corresponding to the similarity of each pixel by a Gaussian function.

6. The focusing method of claim 1, wherein the step of determining whether the first focus value meets the focusing condition comprises:

it is determined whether the first focus value has a local maximum focus value.

7. The focusing method according to claim 1, further comprising:

when the first focus value does not meet the focus condition, the lens is obtained and moved to a new focus position.

8. An image capturing device, comprising:

an image sensor including a lens;

a touch screen;

a memory; and

a processor coupled to the image sensor, the touch screen and the memory, and configured to:

capturing a real-time image by using the image sensor, and displaying the real-time image on the touch screen;

detecting a touch operation on the touch screen to select an interested area related to the touch operation from the instant image;

calculating a first focus value of the region of interest according to the similarity of each pixel of the region of interest relative to at least one central pixel;

judging whether the first focusing value meets the focusing condition; and

when the first focus value accords with the focusing condition, controlling the lens to move to a focusing position corresponding to the first focus value;

wherein the step of calculating the first focus value of the region of interest comprises:

calculating the similarity of each of the pixels with respect to the central pixel;

calculating a weight value corresponding to the similarity of each pixel, wherein the greater the similarity is, the greater the weight value corresponding to the similarity is; and

and obtaining the first focus value of the interested region according to the contrast of each pixel and the weight value.

9. The image capturing device of claim 8, wherein the processor is further configured to obtain and move the lens to a new focus position when the first focus value does not satisfy the focus condition.

Images