JP4953770B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus that performs focusing control using face detection technology.

  Conventionally, an autofocus mechanism has been widely used in an imaging apparatus such as a digital camera. Various methods for detecting the focus position in the autofocus mechanism have been proposed. As a typical method, for example, as disclosed in Patent Document 1, the frequency component (excluding the DC component) included in the image signal is maximized at the in-focus position and is included in the image signal. A method of detecting a focused state using the amount of frequency components as an evaluation value is known.

  Such detection of the in-focus position is usually performed on a partial region of the captured image. And in patent document 2, as a method of determining the area | region which detects a focus position, it detects a person's eyes contained in a captured image, and detects a focus position in the area | region containing the detected eyes. Is disclosed.

  On the other hand, in Patent Document 3 and Patent Document 4, when the captured image is divided into a plurality of areas and weighting is performed according to the position of the areas to perform focus control, image distortion due to the characteristics of the optical system is considered. It has been proposed to use weight values.

  Furthermore, Patent Document 5 proposes a method of displaying an image with corrected image distortion in order to accurately grasp the state of the subject, focusing, and framing when the image is distorted by an optical system. Yes.

Japanese Patent Laid-Open No. 3-1668 JP 2005-128156 A JP-A-5-257893 JP-A-6-237413 JP 11-088732 A JP-A-9-251534 M.A.Turk and A.P.Pentland, "Face recognition using eigenfaces", Proc. Of IEEE Conf. On Computer Vision and Pattern Recognition, pp.586-591, 1991.

  In the imaging apparatus, for example, consider a case in which a face area of a subject is detected from images that are sequentially captured for display on a viewfinder, and focus control is performed on the face area. In this case, when there is almost no distortion in the optical system of the imaging apparatus, the captured image can be used as a display image without distortion correction.

  Therefore, the correspondence between the position of the frame indicating the face area detected from the captured image and the face area in the display image is appropriate. Note that a frame displayed for notifying the user of an area for performing focus control is referred to as a focus frame, and an area in a captured image corresponding to the focus frame used for actual focus control is referred to as a focus control area.

  FIG. 6 is a diagram illustrating an example of a display image and a focus frame when the distortion of the optical system is negligible. In the drawing, the grid shown in the image as shown in FIG. 6 is added to make it easy to visually grasp the degree of distortion of the image, and is not actually displayed. FIG. 6 shows that there is no distortion in the square represented by the grid, and the distortion of the optical system is negligible (that is, the distortion correction is not necessary).

  In such a case, the focus frames 111 and 112 are not only displayed at positions corresponding to the face area in the display image, but also correspond well to the face area in the captured image that is the source of the display image. . This is a natural result because distortion correction is not performed when a display image is generated from a captured image.

  Since the correspondence between the focus frames 111 and 112 and the face area of the captured image is appropriate, the focus evaluation value at the face position of the subject is the highest in the focus control in the focus control area. FIG. 8A represents this state.

  On the other hand, as shown in FIG. 7, it is assumed that the captured image is distorted due to the characteristics of the optical system. In FIG. 7, it can be seen that a barrel distortion has occurred due to the deformation of the grid. Since it is not preferable to display an image having distortion that can be discriminated at a glance in this manner, distortion correction is performed when a display image is generated. Then, face detection processing is performed based on the display image whose distortion has been corrected. As a result, even when the captured image is distorted, the same display as that in FIG. 6 is displayed, and it is apparently indistinguishable. However, apparently the relationship between the areas indicated by the focus frames 111 and 112 and the focus control area is different from the case where the captured image is not distorted.

  That is, the positions of the focus frames 111 and 112 corresponding to the face area in the display image after distortion correction are shifted from the positions of the focus frames 113 and 114 corresponding to the face area in the captured image having distortion by an amount corresponding to the distortion. It will be.

  As a result, when the evaluation value of the focus control is obtained using the area in the captured image corresponding to the focus frames 111 and 112, the control may be performed so that the subject different from the face is focused. For example, in the example illustrated in FIG. 7, a high-contrast subject such as a background house is included in the area corresponding to the focus frames 111 and 112 in the captured image. Therefore, control is performed so that the focus is on the house, not the face area. FIG. 8B shows an example of a focus evaluation value in which the evaluation value at a different distance from the face is increased due to the influence of an image other than the face included in the focus control region.

  As described above, in the related art, the focus control area is determined by applying the face area information detected from the display image to the captured image as it is. For this reason, there is no problem if the distortion of the captured image is negligible and the display image is not corrected for distortion.However, if the distortion cannot be ignored and the display image is corrected for distortion, it fits into a non-face area. There was a risk of getting burned.

  The present invention has been made to solve such problems of the prior art, and an object of the present invention is to provide an imaging apparatus capable of correctly focusing on a face area even when a captured image is distorted. And

The above-described objects are: an imaging unit that converts a subject image formed by an optical system into image data; a distortion correction unit that corrects optical distortion caused by the optical system reflected in the image data and outputs corrected image data; Face detection means for performing face detection processing on the image data and the corrected image data, and outputting face area information indicating the detected face area, and among the face area information output by the face detection means, the face area for the image data Based on the information, the image data included in the face area of the image data is specified, and the focus detection means for detecting the in-focus state of the optical system based on the specified image data, and the corrected image data It is achieved by an imaging apparatus of the present invention comprising display control means for displaying on a display device by superimposing a focus frame based on face area information for.

  With such a configuration, according to the present invention, it is possible to realize an imaging apparatus that can correctly focus on a face area even when a captured image is distorted.

<First Embodiment>
Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, an optical system 2 is composed of one or a plurality of lenses and forms a subject image. The optical system 2 also includes a focus lens used for automatic focus control. The optical distortion of the subject image is caused by the aberration of the lens constituting the optical system 2 in the process in which the light beam from the subject passes through the optical system 2.

  The motor 4 moves the focus lens of the optical system 2. The imaging element 10 is a photoelectric conversion element having a number of pixels of about several million pixels, for example, and a CCD sensor or a CMOS sensor is generally used. A subject image formed by the optical system 2 is converted into an electrical signal (image signal) in units of pixels by the image sensor 10. Although the image sensor 10 is usually provided with peripheral components such as a color filter and an optical low-pass filter, the description thereof is omitted because it is not directly related to the present invention.

  The analog image signal read from the image sensor 10 is converted into digital image data by the A / D converter 12.

  The frame buffer 14 includes a DRAM or a memory controller that controls reading and writing of the DRAM. The frame buffer 14 is used to temporarily store image data from the A / D converter 12 and developed image data from the development processing unit 16 described later.

  The development processing unit 16 performs development processing on the digital image data from the A / D converter 12 and the digital image data held in the frame buffer 14. Specifically, for example, YUV data in the previous stage for generating an image data file in JPEG format generally used in a digital camera is generated.

  The distortion correction unit 18 corrects optical distortion generated in the subject image and the captured image data due to aberration of the lenses constituting the optical system 2, for example, pincushion type or barrel type distortion, and image data (correction) representing the subject image without distortion. Image data). In the present embodiment, the distortion correction unit 18 corrects the optical distortion reflected in the image data by adjusting the reading method for the image data stored in the frame buffer 14.

  The face detection unit 20 performs face detection processing on the YUV format image data output from the development processing unit 16 or the corrected image data output from the distortion correction unit 18, and a human face region ( Face area). The face detection result in the face detection unit 20 is supplied to the display control unit 22 and the distance measurement region designation unit 28 as face region information (for example, position information of the face region). The display control unit 22 determines a focus frame indicating the face region in the corrected image data based on the face region information from the face detection unit 20, and generates display data of the focus frame. The focus frame is, for example, a rectangular frame that circumscribes the face area. The display control unit 22 causes the display unit 23 to display the image data from the frame buffer 14 or the corrected image data from the distortion correction unit 18 and the composite image data of the focus frame. In the present embodiment, corrected image data is always displayed for ease of explanation and understanding.

  Note that the face area detected by the face detection unit 20 does not necessarily include the entire face, and may be a partial area of the face. In addition, when it is a partial area | region, it is preferable that an eye area | region is included at least.

  The display unit 23 is, for example, an LCD, and displays a user interface for performing various settings of the digital camera 100, reproduces and displays captured images, and functions as an electronic viewfinder. The display unit 23 may be an external device such as a television receiver or a display monitor.

  The distance measurement area designation unit 28 determines a focus control area used for focus control in the captured image data based on the face area information from the face detection unit 20. Specifically, the distance measurement area designating unit 28 corresponds to the face area indicated by the face area information (for example, a rectangular area circumscribing the face area) for the YUV format image data output from the development processing unit 16. Is designated as the focus control area. Then, the distance measurement area designation unit 28 outputs information on the designated focus control area (for example, information indicating the position and size of the focus control area) to the focus evaluation value acquisition unit 30.

  The focus evaluation value acquisition unit 30 acquires luminance (Y) component data corresponding to the focus control area from the development processing unit 16 based on the information regarding the focus control area supplied from the distance measurement area specifying unit 28. Then, a focus evaluation value indicating the in-focus state of the optical system is calculated from the luminance component data. The focus evaluation value is supplied to the focus control unit 32. The focus evaluation value acquisition unit 30 and the focus control unit 32 are collectively referred to as a focus detection unit.

  The focus control unit 32 receives the evaluation value from the focus evaluation value acquisition unit 30, controls the focus motor drive unit 34, and searches for a position where the evaluation value is maximized. The focus motor drive unit 34 controls the motor 4 according to an instruction from the focus control unit 32 to move the focus lens included in the optical system 2.

Next, the operation of the distortion correction unit 18 in this embodiment will be further described.
Since the optical lens has aberration, optical distortion occurs in the subject image formed through the optical lens. Since the imaging device converts the subject image formed by the optical system 2 into an image signal as it is, the image signal is also affected by the optical distortion of the subject image. Typical optical distortion includes “thread-wound distortion” as shown in FIG. 2A and “barrel distortion” as shown in FIG. FIG. 2 shows that the image of the subject in the range indicated by the dotted line is formed in a solid line shape due to optical distortion.

The optical distortion of the subject image can be corrected by, for example, converting the subject image into digital data and rearranging the pixel data according to the distortion.
For example, consider the case of correcting an image in which a barrel distortion occurs as shown in FIG. FIG. 3 shows a state where points A to D are imaged at points a to d due to distortion. In this case, the distorted subject image indicated by the solid line is converted into digital image data by the A / D converter 12 and temporarily stored in the frame buffer 14.

  When generating the corrected image, the distortion correction unit 18 uses the pixel a as the pixel A, the pixel b as the pixel B, the pixel c as the pixel C, and the pixel D. By reading out each of the d-point pixels as the pixels, a distortion-corrected image can be generated. When the number of pixels between A and B is different from the actual number of pixels between a and b due to distortion, the number of pixels can be matched by interpolation or thinning. Various methods for correcting optical distortion have been proposed in the past, and any correction method can be used theoretically in the present invention, so no further explanation will be given.

Next, the operation of the face detection unit 20 will be described.
In the present embodiment, a known face detection technique can be used to detect a human face area.
Specifically, as described in Non-Patent Document 1, a method using an eigenface by principal component analysis, or feature points such as eyes, nose, and mouth as described in Patent Document 6 The method using can be used. These methods determine whether a human face is included in the input image based on the pattern matching result between the input image and a plurality of standard patterns.

  In the present embodiment, the face detection unit 20 holds a standard pattern of a person's face in advance. The face detection unit 20 performs pattern matching between the captured image data stored in the DRAM of the frame buffer 14 or the corrected image data output from the distortion correction unit 18 and the standard pattern, so that the human face is detected. It is determined whether or not it is included. When an area (face area) determined to be a human face is detected, the face detection unit 20 outputs information about each face area to the display control unit 22 and the distance measurement area designating unit 28. The information about the face area may be any information that allows the display control unit 22 and the distance measurement area designating unit 28 to specify the face area. For example, the coordinate value that defines the minimum rectangle circumscribing the face area And so on.

Next, the autofocus operation in the digital camera 100 of this embodiment will be described.
In general, when a subject includes a person, it is often desired to take a picture focused on the person, but the subject may be focused on the background, particularly when the person is not in the center of the shooting range. For this reason, it is considered that by using the face detection technique and performing focus control so as to focus on the face of a person in the subject, it is possible to increase the possibility of providing a photographing result according to the user's intention.

  However, as described as the problem of the prior art, conventionally, the focus control area is designated using the result of face detection for the display image as it is. Therefore, when the display image is corrected for distortion, a shift occurs between the face area detected in the display image and the face area in the captured image used for focus control, and an expected focus control result. May not be obtained.

  In order to eliminate this shift, it is conceivable to use image data after distortion correction for focusing control, but this is not desirable from the following points. One is that when distortion correction is performed, high-frequency components that are important in obtaining an evaluation value for focusing control are lost, and focusing accuracy is reduced. The other is that the distortion correction process is a high-load process and takes a long processing time, which increases the time until the in-focus position is determined.

  Therefore, in the present embodiment, face detection is performed on captured image data. Thereby, even when the captured image data is distorted, the focusing control area can be correctly associated with the face area in the captured image. On the other hand, face detection for displaying the focus frame is also performed on the display image data.

  Since both face detection processes are performed using one face detection unit 20, the face detection process for the captured image data and the face detection process for the display image data are time-division processed.

FIG. 4 is a timing chart for explaining the operation timing of each part of the digital camera 100.
FIG. 4A shows digital image data output from the A / D converter 12 and has a frame rate of 30 fps (approximately 33 mSec interval) in order to cause the display unit 23 to function as an electronic viewfinder. FIG. 4B shows a timing at which a result (captured image data) obtained by developing the digital image data output from the A / D converter 12 by the development processing unit 16 is output. The captured image data is output later than the digital image data by the time required for the development process. As described above, the captured image data is temporarily stored in the frame buffer 14.

  FIG. 4C shows the timing at which distortion correction processing by the distortion correction unit 18 is performed after the captured image data is stored in the frame buffer 14. As shown in FIG. 4C, the distortion correction unit 18 starts reading the captured image data from the time when it is accumulated to some extent (about half of the image in this case) in the frame buffer 14 to generate distortion correction data. .

  FIG. 4D shows the type and timing of data supplied to the face detection unit 20. As described above, the captured image data An (n is an integer of 1 or more) from the development processing unit 16 and the corrected image data Bn from the distortion correction unit 18 are switched in a time division manner and input to the face detection unit 20. ing.

  In this embodiment, as shown in FIG. 4D, the captured image data An and the corrected image data Bn after distortion correction are alternately input as A1, B2, A4, B5, A7,. . Here, the An and Bn input timings are not equally spaced in time because the distortion correction processing reads the captured image data with a delay in the frame buffer 14, and the distortion correction processing requires more time than the development processing. by. Thus, when An and Bn are alternately supplied and face detection processing is performed, face detection is performed every three frames for both captured image data and corrected image data. The speed is 10 times / second. This speed provides a sufficient amount of information for displaying the focus frame and determining the focus control area.

  Rather, in the display of the focus frame, if the display update interval is short, for example, the display position is switched sensitively due to camera shake or the like, making it difficult to see the focus frame. Therefore, it is sufficient to update several times / second with temporal hysteresis. Similarly, when performing focus control using the face region as the focus control region, the focus control region moves frequently because the focus distance is determined by the hill-climbing method of evaluation values as shown in FIG. It is more preferable that the movement is slow to some extent. For this reason, it is sufficient that information on the face area is given about 10 times / second in each application.

  In the present embodiment, in order to facilitate understanding and explanation of the invention, face area detection for focus control and face area detection for focus frame display are performed regardless of the degree of optical distortion. It was. As described above, even if such processing is performed, there is no problem in practical use. However, when distortion correction is not necessary, the focus frame display face area detection result is used to determine the focus control area. You may do it.

  The necessity of distortion correction can be determined based on whether or not conditions for correcting distortion (correction conditions) are determined in advance in accordance with the characteristics of the optical system 2 and whether or not the correction conditions are met. The characteristic of the optical system 2 may be, for example, the relationship between the focal length (field angle) of the lens and the stop.

  As described above, according to the present embodiment, in an imaging apparatus capable of performing focus control on a face area, the face area used for focus control is detected from image data before distortion correction. Thereby, even when the subject image has optical distortion, the focus control can be correctly performed on the face region, and the focus control with high accuracy is realized.

  Furthermore, the face area for displaying the focus frame can be detected from the image data after distortion correction used for display. In this case, even when the subject image has optical distortion, it is possible to display a focus frame corresponding to the face area in the image data without distortion.

  In addition, since face detection for image data before distortion correction and face detection processing for image data after distortion correction are performed in a time-sharing manner by one face detection unit, the circuit scale is not increased. In addition, since face detection for image data before distortion correction and face detection for image data after distortion correction may be performed on discrete frames, it can be sufficiently performed by a face detection unit similar to the conventional one. Only a slight design change is required with respect to the conventional configuration.

<Second Embodiment>
FIG. 5 is a block diagram illustrating a functional configuration example of a digital camera 110 as an example of an imaging apparatus according to the second embodiment of the present invention.

  5, the same components as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. The digital camera 110 of this embodiment is that it has two face detection units (a first face detection unit 24 and a second face detection unit 26). Each of the first and second face detection units 24 and 26 may have the same configuration as the face detection unit 20 in the first embodiment.

  The first face detection unit 24 performs face detection processing on the corrected image data, and the second face detection unit 26 performs face detection processing on the image data independently. Therefore, it is possible to execute the face detection processing performed in a time division manner in parallel in the first embodiment, and the face area information is output frames of the image sensor 10 for both focus control and focus frame display. It can be acquired at the same speed as the rate (30 fps) 30 times / second.

  As described above, as in the first embodiment, even when the subject image has optical distortion, the focus control can be correctly performed on the face area. Further, even when the subject image has optical distortion, it is possible to display a focus frame corresponding to a face area in image data without distortion. Furthermore, in this embodiment, although the circuit scale increases, finer processing is possible, and it is possible to perform focus control with excellent followability when shooting a person with large movement.

1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows the pincushion type | mold distortion which is a typical example of an optical distortion, and the type | mold distortion. It is a figure explaining an example of the correction method of barrel distortion. It is a timing chart for demonstrating the operation timing of each part of the digital camera 100 of the 1st Embodiment of this invention. It is a block diagram which shows the function structural example of the digital camera 110 as an example of the imaging device which concerns on the 2nd Embodiment of this invention. It is a figure which shows the example of a display image and a focus frame in case the distortion of an optical system is a grade which can be disregarded. It is a figure explaining the shift | offset | difference of the face area | region in the captured image when the distortion of an optical system is not negligible, and the face area | region of the image for a display. It is a figure explaining the difference in the focus evaluation value in the state of FIG.6 and FIG.7.

Claims (3)

Imaging means for converting a subject image formed by the optical system into image data;
Distortion correcting means for correcting optical distortion caused by the optical system reflected in the image data and outputting corrected image data;
Face detection means for performing face detection processing on the image data and the corrected image data, and outputting face area information indicating the detected face area;
Among the face area information output by the face detection means, the image data included in the face area of the image data is specified based on the face area information for the image data, and based on the specified image data Focus detection means for detecting the in-focus state of the optical system ;
An image pickup apparatus comprising: a display control unit configured to superimpose a focus frame based on face area information on the corrected image data and display the correction image data on a display device.   The face detection means includes a first face detection unit that performs a face detection process on the image data, and a second face detection unit that performs a face detection process on the corrected image data. The imaging device according to claim 1.   The imaging apparatus according to claim 1, wherein the face detection unit performs the face detection process on the image data and the face detection process on the corrected image data in a time-sharing manner.

Images