JP4732303B2 - Imaging device - Google Patents

Description

  The present invention relates to an imaging apparatus, and more particularly to an imaging apparatus that suppresses the influence of distortion of a subject image generated in an optical system on focusing control.

  Conventionally, an autofocus mechanism has been widely used in an imaging apparatus such as a digital camera. Various methods for detecting the focus position in the autofocus mechanism have been proposed. As a typical method, for example, as disclosed in Patent Document 1, the frequency component (excluding the DC component) included in the image signal is maximized at the in-focus position and is included in the image signal. A method of detecting a focused state using the amount of frequency components as an evaluation value is known.

  Such detection of the in-focus position is usually performed on a partial region of the captured image. And in patent document 2, as a method of determining the area | region which detects a focus position, it detects a person's eyes contained in a captured image, and detects a focus position in the area | region containing the detected eyes. Is disclosed.

  On the other hand, in Patent Document 3 and Patent Document 4, when the captured image is divided into a plurality of areas and weighting is performed according to the position of the areas to perform focus control, image distortion due to the characteristics of the optical system is considered. It has been proposed to use weight values.

  Furthermore, Patent Document 5 proposes a method of displaying an image with corrected image distortion in order to accurately grasp the state of the subject, focusing, and framing when the image is distorted by an optical system. Yes.

Japanese Patent Laid-Open No. 3-1668 JP 2005-128156 A JP-A-5-257893 JP-A-6-237413 JP 11-088732 A JP-A-9-251534 M.A.Turk and A.P.Pentland, "Face recognition using eigenfaces", Proc. Of IEEE Conf. On Computer Vision and Pattern Recognition, pp.586-591, 1991.

  In the imaging apparatus, for example, consider a case in which a face area of a subject is detected from images that are sequentially captured for display on a viewfinder, and focus control is performed on the face area. In this case, when there is almost no distortion in the optical system of the imaging apparatus, the captured image can be used as a display image without distortion correction.

  Therefore, the correspondence between the position of the frame indicating the face area detected from the captured image and the face area in the display image is appropriate. Note that the frame displayed to notify the user of the area where focus control is performed is the focus frame, and the area in the captured image corresponding to the focus frame used for actual focus control is the focus control area (image signal). Area).

  FIG. 6 is a diagram illustrating an example of a display image and a focus frame when the distortion of the optical system is negligible. In the drawing, the grid shown in the image as shown in FIG. 6 is added to make it easy to visually grasp the degree of distortion of the image, and is not actually displayed. FIG. 6 shows that there is no distortion in the square represented by the grid, and the distortion of the optical system is negligible (that is, the distortion correction is not necessary).

  In such a case, the focus frames 111 and 112 are not only displayed at positions corresponding to the face area in the display image, but also correspond well to the face area in the captured image that is the source of the display image. . This is a natural result because distortion correction is not performed when a display image is generated from a captured image.

  Since the correspondence between the focus frames 111 and 112 and the face area of the captured image is appropriate, the focus evaluation value at the face position of the subject is the highest in the focus control in the focus control area. FIG. 8A represents this state.

  On the other hand, as shown in FIG. 7, it is assumed that the captured image is distorted due to the characteristics of the optical system. In FIG. 7, it can be seen that a barrel distortion has occurred due to the deformation of the grid. Since it is not preferable to display an image having distortion that can be discriminated at a glance in this manner, distortion correction is performed when a display image is generated. Then, face detection processing is performed based on the display image whose distortion has been corrected. As a result, even when the captured image is distorted, the same display as that in FIG. 6 is displayed, and it is apparently indistinguishable. However, apparently the relationship between the areas indicated by the focus frames 111 and 112 and the focus control area is different from the case where the captured image is not distorted.

  That is, the positions of the focus frames 111 and 112 corresponding to the face area in the display image after distortion correction are shifted from the positions of the focus frames 113 and 114 corresponding to the face area in the captured image having distortion by an amount corresponding to the distortion. It will be.

  As a result, when the evaluation value of the focus control is obtained using the area in the captured image corresponding to the focus frames 111 and 112, the control may be performed so that the subject different from the face is focused. For example, in the example illustrated in FIG. 7, a high-contrast subject such as a background house is included in the area corresponding to the focus frames 111 and 112 in the captured image. Therefore, control is performed so that the focus is on the house, not the face area. FIG. 8B shows an example of a focus evaluation value in which the evaluation value at a different distance from the face is increased due to the influence of an image other than the face included in the focus control region.

  As described above, in the related art, the focus control area is determined by applying the face area information detected from the display image to the captured image as it is. For this reason, there is no problem if the distortion of the captured image is negligible and the display image is not corrected for distortion.However, if the distortion cannot be ignored and the display image is corrected for distortion, it fits into a non-face area. There was a risk of getting burned.

  Further, the point that the distortion of the subject image affects the accuracy of the focus control is not limited to the case where the face area is set as the focus control area. For example, even when the user designates the focus control area on the display screen, a shift occurs between the area designated in the display image and the area of the image data used for the focus control. Similar problems arise.

  The present invention has been made to solve such problems of the prior art, and an object of the present invention is to provide an imaging apparatus capable of focusing accurately even when a captured image is distorted. To do.

The above-described objects are: an imaging unit that converts a subject image formed by an optical system into an image signal; a distortion correction unit that corrects optical distortion caused by the optical system reflected in the image signal and outputs corrected image data; based on the corrected image data, and setting means for setting a focus frame to detect the focus state of the optical system, depending on the focus frame set by the setting means, the area of the image signal output by the imaging means A designating unit for designating, and a focus detecting unit for detecting a focus state of the optical system based on an image signal of the region designated by the designating unit, wherein the designating unit is an area of the corrected image data corresponding to the focus frame. to determine the area corresponding to the focus frame in the distortion correcting unit image signal prior to application to distortion correction correction reverse processing by the imaging instrumentation of the present invention characterized by specifying the person area It is achieved by.

  With such a configuration, according to the present invention, it is possible to realize an imaging device capable of focusing with high accuracy even when the captured image is distorted.

<First Embodiment>
Preferred embodiments of the present invention will be described below with reference to the accompanying drawings.
FIG. 1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to the first embodiment of the present invention.

  In FIG. 1, an optical system 2 is composed of one or a plurality of lenses and forms a subject image. The optical system 2 also includes a focus lens used for automatic focus control. The optical distortion of the subject image is caused by the aberration of the lens constituting the optical system 2 in the process in which the light beam from the subject passes through the optical system 2. The optical system 2 can supply information such as an angle of view and a diaphragm as lens information in response to a request from another component.

  The motor 4 moves the focus lens of the optical system 2. The imaging element 10 is a photoelectric conversion element having a number of pixels of about several million pixels, for example, and a CCD sensor or a CMOS sensor is generally used. A subject image formed by the optical system 2 is converted into an electrical signal (image signal) in units of pixels by the image sensor 10. Although the image sensor 10 is usually provided with peripheral components such as a color filter and an optical low-pass filter, the description thereof is omitted because it is not directly related to the present invention.

  The analog image signal read from the image sensor 10 is converted into digital image data by the A / D converter 12.

  The frame buffer 14 includes a DRAM or a memory controller that controls reading and writing of the DRAM. The frame buffer 14 is used to temporarily store image data from the A / D converter 12 and developed image data from the development processing unit 16 described later.

  The development processing unit 16 performs development processing on the digital image data from the A / D converter 12 and the digital image data held in the frame buffer 14. Specifically, for example, YUV data in the previous stage for generating an image data file in JPEG format generally used in a digital camera is generated. The processing result is stored in the frame buffer 14 as captured image data.

  The distortion correction unit 18 corrects optical distortion generated in the subject image and the captured image data due to aberration of the lenses constituting the optical system 2, for example, pincushion type or barrel type distortion, and image data (correction) representing the subject image without distortion. Image data). In the present embodiment, the distortion correction unit 18 corrects the optical distortion reflected in the image data by adjusting the reading method for the image data stored in the frame buffer 14.

  The face detection unit 20 performs face detection processing on the corrected image data output from the distortion correction unit 18 to detect a human face region (face region) included in the subject. The face detection result in the face detection unit 20 is supplied to the display control unit 22 and the inverse distortion correction unit 26 as face area information (for example, position information of the face area). The display control unit 22 determines a focus frame indicating the face region in the corrected image data based on the face region information from the face detection unit 20, and generates display data of the focus frame. The focus frame is, for example, a rectangular frame that circumscribes the face area. The display control unit 22 causes the display unit 23 to display the image data from the frame buffer 14 or the corrected image data from the distortion correction unit 18 and the composite image data of the focus frame. In the present embodiment, corrected image data is always displayed for ease of explanation and understanding.

  Note that the face area detected by the face detection unit 20 does not necessarily include the entire face, and may be a partial area of the face. In addition, when it is a partial area | region, it is preferable that an eye area | region is included at least.

  The display unit 23 is, for example, an LCD, and displays a user interface for performing various settings of the digital camera 100, reproduces and displays captured images, and functions as an electronic viewfinder. The display unit 23 may be an external device such as a television receiver or a display monitor.

  Based on the face detection area from the face detection unit 20, the inverse distortion correction unit 26 obtains a face area in the captured image data before distortion correction corresponding to the face area in the corrected image data. The obtained face area information is output to the distance measurement area designating unit 28. Specifically, the inverse distortion correction unit 26 applies a process opposite to that of the distortion correction unit 18 to the face area detected by the face detection unit 20 to obtain a corresponding area in the captured image data before distortion correction. .

  The distance measurement area designating unit 28 determines a focus control area used for focus control in the captured image data based on the face area information from the inverse distortion correction unit 26. Specifically, the distance measurement area designating unit 28 corresponds to the face area indicated by the face area information from the inverse distortion correction unit 26 (for example, the face area) with respect to the YUV format image data output from the development processing unit 16. Is specified as the focus control area. Then, the distance measurement area designation unit 28 outputs information on the designated focus control area (for example, information indicating the position and size of the focus control area) to the focus evaluation value acquisition unit 30.

  The focus evaluation value acquisition unit 30 acquires luminance (Y) component data corresponding to the focus control area from the development processing unit 16 based on the information regarding the focus control area supplied from the distance measurement area specifying unit 28. Then, a focus evaluation value indicating the in-focus state of the optical system is calculated from the luminance component data. The focus evaluation value is supplied to the focus control unit 32. The focus evaluation value acquisition unit 30 and the focus control unit 32 are collectively referred to as a focus detection unit.

  The focus control unit 32 receives the evaluation value from the focus evaluation value acquisition unit 30, controls the focus motor drive unit 34, and searches for a position where the evaluation value is maximized. The focus motor drive unit 34 controls the motor 4 according to an instruction from the focus control unit 32 to move the focus lens included in the optical system 2.

Next, the operation of the distortion correction unit 18 in this embodiment will be further described.
Since the optical lens has aberration, optical distortion occurs in the subject image formed through the optical lens. Since the imaging device converts the subject image formed by the optical system 2 into an image signal as it is, the image signal is also affected by the optical distortion of the subject image. Typical optical distortion includes “thread-wound distortion” as shown in FIG. 2A and “barrel distortion” as shown in FIG. FIG. 2 shows that the image of the subject in the range indicated by the dotted line is formed in a solid line shape due to optical distortion.

The optical distortion of the subject image can be corrected by, for example, converting the subject image into digital data and rearranging the pixel data according to the distortion.
For example, consider the case of correcting an image in which a barrel distortion occurs as shown in FIG. FIG. 3 shows a state where points A to D are imaged at points a to d due to distortion. In this case, the distorted subject image indicated by the solid line is converted into digital image data by the A / D converter 12 and temporarily stored in the frame buffer 14.

  When generating the corrected image, the distortion correction unit 18 uses the pixel a as the pixel A, the pixel b as the pixel B, the pixel c as the pixel C, and the pixel D. By reading out each of the d-point pixels as the pixels, a distortion-corrected image can be generated. When the number of pixels between A and B is different from the actual number of pixels between a and b due to distortion, the number of pixels can be matched by interpolation or thinning. Various methods for correcting optical distortion have been proposed in the past, and any correction method can be used theoretically in the present invention, so no further explanation will be given.

Next, the operation of the face detection unit 20 will be described.
In the present embodiment, a known face detection technique can be used to detect a human face area.
Specifically, as described in Non-Patent Document 1, a method using an eigenface by principal component analysis, or feature points such as eyes, nose, and mouth as described in Patent Document 6 The method using can be used. These methods determine whether a human face is included in the input image based on the pattern matching result between the input image and a plurality of standard patterns.

  In the present embodiment, the face detection unit 20 holds a standard pattern of a person's face in advance. The face detection unit 20 performs pattern matching between the captured image data stored in the DRAM of the frame buffer 14 or the corrected image data output from the distortion correction unit 18 and the standard pattern, so that the human face is detected. It is determined whether or not it is included. When an area (face area) determined to be a human face is detected, the face detection unit 20 outputs information about each face area to the display control unit 22 and the distance measurement area designating unit 28. The information about the face area may be any information that allows the display control unit 22 and the distance measurement area designating unit 28 to specify the face area. For example, the coordinate value that defines the minimum rectangle circumscribing the face area And so on.

Next, the operation of the inverse distortion correction unit 26 will be described.
The inverse distortion correction unit 26 performs a process opposite to the distortion correction of the optical system 2 performed by the distortion correction unit 18. For example, if the dotted line in FIG. 3 is an image after distortion correction, it corresponds to a process of performing conversion (inverse conversion) to an image before distortion correction indicated by a solid line. Therefore, the points A to D are converted into points a to d, respectively.

  However, while the distortion correction unit 18 performs correction processing on the entire captured image data in principle, the inverse distortion correction unit 26 only includes an area corresponding to the face area information from the face detection unit 20 in the corrected image data. However, it is different in that correction processing is performed. Further, since the area can be specified if the coordinates of the pixels surrounding the area after reverse correction are obtained, it is not necessary to perform processing for all the pixels in the area. Therefore, the processing load is small, and correction processing can be performed in a short time even if there are a plurality of face regions.

  The inverse distortion correction unit 26 stores information on what correction the distortion correction unit 18 has performed, and may perform inverse conversion using the information, or may store the distortion characteristics of the optical system 2 in advance. In addition, distortion characteristics corresponding to shooting conditions may be used. Similar to the distortion correction unit 18, the inverse distortion correction unit 26 can be realized by applying a known method, and thus detailed description thereof is omitted.

  In this way, the inverse distortion correction unit 26 obtains an area corresponding to the area indicated by the face area information from the face detection unit 20 in the captured image data before correction as a focus control face area, and the position and size thereof. The face area information capable of specifying the height is output to the distance measurement area designating unit 28.

Next, the autofocus operation in the digital camera 100 of this embodiment will be described.
In general, when a subject includes a person, it is often desired to take a picture focused on the person, but the subject may be focused on the background, particularly when the person is not in the center of the shooting range. For this reason, it is considered that by using the face detection technique and performing focus control so as to focus on the face of a person in the subject, it is possible to increase the possibility of providing a photographing result according to the user's intention.

  However, as described as the problem of the prior art, conventionally, the focus control area is designated using the result of face detection for the display image as it is. Therefore, when the display image is corrected for distortion, a shift occurs between the face area detected in the display image and the face area in the captured image used for focus control, and an expected focus control result. May not be obtained.

  In order to eliminate this shift, it is conceivable to use image data after distortion correction for focusing control, but this is not desirable from the following points. One is that when distortion correction is performed, high-frequency components that are important in obtaining an evaluation value for focusing control are lost, and focusing accuracy is reduced.

  Therefore, in this embodiment, the face area in the image data before distortion correction is obtained by performing a process opposite to the distortion correction process on the result of the face detection process performed on the image data after distortion correction. Then, by performing focus control in the focus control area determined based on the face area, accurate focus control is realized without increasing the load of face detection processing.

FIG. 4 is a timing chart for explaining the operation timing of each part of the digital camera 100.
FIG. 4A shows digital image data output from the A / D converter 12 and has a frame rate of 30 fps (approximately 33 mSec interval) in order to cause the display unit 23 to function as an electronic viewfinder. FIG. 4B shows a timing at which a result (captured image data) obtained by developing the digital image data output from the A / D converter 12 by the development processing unit 16 is output. The captured image data is output later than the digital image data by the time required for the development process. As described above, the captured image data is temporarily stored in the frame buffer 14.

  FIG. 4C shows the timing at which distortion correction processing by the distortion correction unit 18 is performed after the captured image data is stored in the frame buffer 14. As shown in FIG. 4C, the distortion correction unit 18 starts reading the captured image data from the time when it is accumulated to some extent (about half of the image in this case) in the frame buffer 14 to generate distortion correction data. .

FIG. 4D shows the output of the distortion correction unit 18, that is, the input timing of the face detection unit 20. The distortion correction process itself has a large processing load and takes time to complete correction, but there is almost no time difference from input to start of output.
4E shows the output timing of the face detection unit 20, and FIG. 4F shows the output timing of the inverse distortion correction unit 26.

  As shown in FIG. 4, the output of the inverse distortion correction unit 26 for the image data of the first frame is completed at a timing at which the image data of the third frame starts to be output. The focus evaluation value acquisition unit 30 acquires an evaluation value using image data that matches the output timing of the inverse distortion correction unit 26 that is the source of the focus control region supplied from the distance measurement region specification unit 28. Therefore, in the example of FIG. 4, the frame buffer 14 needs to have a capacity for temporarily storing image data for at least three frames.

  As described above, there is a delay of several frames before the acquisition of the evaluation value for focus control occurs. However, it is possible to specify a focus control area for the image of each frame, and for a subject with large movement. Even in such a case, it is possible to perform focusing control with sufficient follow-up.

  As described above, according to the present embodiment, in an imaging apparatus capable of performing focus control on a face area, the face area used for focus control is detected from the image data after distortion correction. Obtained by correcting the reverse distortion in the area. Thereby, even when the subject image has optical distortion, the focus control can be correctly performed on the face region, and the focus control with high accuracy is realized.

  Since the face detection process for image data after distortion correction is a process performed to display the focus frame, the load of the face detection process does not increase. In addition, since inverse distortion correction only needs to be performed on the face area, the processing load is small, high-speed processing is possible, and focus control can be performed with sufficient follow-up even when the movement of the face area is large. it can.

<Second Embodiment>
FIG. 5 is a block diagram illustrating a functional configuration example of a digital camera 110 as an example of an imaging apparatus according to the second embodiment of the present invention.

  5, the same components as those in FIG. 1 are denoted by the same reference numerals, and the description thereof is omitted. The digital camera 110 of the present embodiment is provided with a focusing area designating unit 21 and a new operation unit 35 instead of the face detecting unit 20 of FIG.

  In the digital camera 100 according to the first embodiment, the camera side automatically sets the focus frame based on the face area. On the other hand, the digital camera 110 of the present embodiment is different in that the user can specify a focus frame.

  As described above, the problem that the optical distortion causes a shift between the area indicated by the displayed focus frame and the area in the image data used for focusing control is limited to the case where the focus frame is determined based on the face area. Exist. The present invention can also be applied to a digital camera in which a user designates a focus frame.

  In the digital camera 110 of the present embodiment, the operation unit 35 is provided for a user to input an instruction to the digital camera 110 and includes an input device such as a switch, a button, a dial, a touch panel, and a joystick. FIG. 9 is a diagram for explaining a focus frame setting operation in the digital camera 110 of the present embodiment.

  Direction keys 72 and setting buttons 73 are included in the operation unit 35. The user can designate the position of the focus frame 71 by moving the focus frame 71 in the display screen to a desired position by operating the direction key 72 and pressing the setting button 73. Here, the case where the size of the focus frame 71 is fixed has been described as an example, but the size of the focus frame 71 may be set within a certain range.

  The operation of the operation unit 35 is input to the in-focus area specifying unit 21, and at least information regarding the position of the focus frame position and size at the time when the setting button 73 is pressed is input to the inverse distortion correction unit 26.

  In the same manner as in the first embodiment, the inverse distortion correction unit 26 obtains a corresponding area of the focus frame 71 in the captured image data before distortion correction, and provides the corresponding area information to the distance measurement area designation unit 28. Thereafter, processing is performed as in the first embodiment, and focus control is performed on the focus control area.

  As described above, according to the present embodiment, it is possible to perform precise focusing control according to the focus frame designated by the user for the corrected image data through the operation unit regardless of the presence or absence of optical distortion.

<Other embodiments>
When a face area is detected in a place where optical distortion is large (first embodiment) or when a focus frame is specified (second embodiment), the reverse distortion correction unit 26 finally performs reverse correction. Thus, the corresponding area in the captured image data obtained is greatly distorted. Then, the accuracy of focus control in a region with a large distortion is lower than the accuracy of focus control in a region with a small distortion. Therefore, for example, when focus control is performed on a plurality of focus control areas when a plurality of face areas are detected or a plurality of focus frames are specified, the priority of the focus control areas with large distortion decreases. It is possible to make it not be used in some cases. In other words, the focus control area having a smaller distortion is used with a higher priority.

  It is known that the optical distortion increases as the angle of view increases (the wider the angle), and increases as it approaches the periphery of the image. Therefore, for example, if the angle of view of the optical system 2 is equal to or larger than a predetermined angle of view, and the detection position of the face area (or the position of the designated focus frame) is more peripheral than a predetermined range, the face area is It can be determined that the distortion is large.

  Specifically, for example, the inverse distortion correction unit 26 acquires the angle of view of the optical system 2 as lens information. Then, the inverse distortion correction unit 26 serving as a determination unit includes the size of the angle of view, the face area information from the face detection unit 20 (first embodiment), and the focus frame position information from the in-focus area designation unit 21 ( According to the second embodiment, it is determined whether or not these regions are in a region having a large optical distortion. And when it determines with it being in an area | region where distortion is large, it is also possible not to perform reverse correction processing itself. Alternatively, a flag indicating that the distortion is large is added to the obtained area information and output to the distance measurement area designating unit 28. The distance measurement area designating unit 28 does not designate an area with large distortion as a focus control area, or assigns a lower priority or weight than an area with small distortion. The focus evaluation value acquisition unit 30 obtains evaluation values for each of the focus control areas, weights them, and outputs them to the focus control unit 32.

  By performing the focusing control by weighting the focusing control area with less distortion, the focusing accuracy can be improved. In addition, when a focus control region having a large distortion is not used, the processing load of focus control can be reduced.

1 is a block diagram illustrating a functional configuration example of a digital camera 100 as an example of an imaging apparatus according to a first embodiment of the present invention. It is a figure which shows the pincushion type | mold distortion which is a typical example of an optical distortion, and the type | mold distortion. It is a figure explaining an example of the correction method of barrel distortion. It is a timing chart for demonstrating the operation timing of each part of the digital camera 100 of the 1st Embodiment of this invention. It is a block diagram which shows the function structural example of the digital camera 110 as an example of the imaging device which concerns on the 2nd Embodiment of this invention. It is a figure which shows the example of a display image and a focus frame in case the distortion of an optical system is a grade which can be disregarded. It is a figure explaining the shift | offset | difference of the face area | region in the captured image when the distortion of an optical system is not negligible, and the face area | region of the image for a display. It is a figure explaining the difference in the focus evaluation value in the state of FIG.6 and FIG.7. It is a figure explaining the setting operation | movement of the focus frame in the digital camera 110 of the 2nd Embodiment of this invention.

Claims (4)

Imaging means for converting a subject image formed by the optical system into an image signal;
Distortion correcting means for correcting optical distortion caused by the optical system reflected in the image signal and outputting corrected image data;
Based on the corrected image data, and setting means for setting a focus frame to detect the focus state of said optical system,
Depending on the focus frame set by the setting means, designating means for designating an area of the image signal output by the imaging means,
A focus detection unit that detects a focused state of the optical system based on an image signal of an area designated by the designation unit;
The designation unit obtains a region corresponding to the focus frame in the image signal before distortion correction by applying a process reverse to the correction by the distortion correction unit to the region of the corrected image data corresponding to the focus frame. imaging apparatus characterized by specifying the area. Face detection means for performing face detection processing on the corrected image data and outputting face area information representing the detected face area;
The imaging apparatus according to claim 1, wherein the setting unit sets the focus frame based on the face area information. An operation unit for a user to input an instruction to the imaging apparatus;
The setting means, the imaging apparatus according to claim 1 or 2, wherein the setting the focus frame on the basis of an instruction input to the operation unit. When a plurality of the focus frames are set , the focus detection unit preferentially uses an image signal in an area corresponding to the focus frame set at a position where the optical distortion due to the optical system is small among the plurality of focus frames. the imaging apparatus according to any one of claims 1 to 3, characterized in that to detect the focus state of the optical system Te.