JP6016622B2 - Imaging apparatus, control method therefor, and program - Google Patents

Description

  The present invention relates to an imaging apparatus, a control method thereof, and a program.

  2. Description of the Related Art In recent imaging apparatuses such as digital video cameras, a specific subject such as a human face is detected from an image included in a photographing signal output from an image sensor, and auto focus (AF) control is performed on the detected subject. Some are equipped with functions.

  In AF control, a TV-AF method is widely used in which a photographing optical system is focused based on a signal (focus signal) representing the sharpness (contrast) of a video signal generated using an image sensor.

  Here, the focus signal is generated by using a video signal of a part of the focus detection area on the captured screen. This is because when there are a plurality of subjects having different distances in the photographed image, only the subject in the focus detection area is focused. Then, by detecting the position on the screen of a specific subject such as a person's face using the subject detection technique and setting a focus detection area at the position, the detected subject can be focused.

  By the way, with recent improvements in image processing technology, a function for electronically correcting various image distortions in an imaging apparatus has been installed. FIG. 4 is a diagram illustrating an example of image distortion in the imaging apparatus.

  4 (A) to 4 (I), a rectangular frame indicates the entire image area, and a grid indicated by a dotted line is schematically added to visually indicate the distortion state of the image. Is.

  FIG. 4A shows an image after distortion correction, and the grid is square when the distortion is corrected.

  On the other hand, FIGS. 4B to 4I show images before distortion correction output from the image sensor.

  Among these, FIGS. 4B and 4C show distortion due to distortion of the optical system, FIG. 4B shows barrel distortion, and FIG. 4C shows pincushion distortion. .

  4D to 4F show rolling shutter distortion that occurs when a CMOS sensor is used as an image sensor.

  Specifically, FIG. 4D shows a case where the image pickup apparatus is shaken in the horizontal direction (pan), FIG. 4E shows a case where the image pickup apparatus is shaken in the vertical direction (pitch), and FIG. The case (roll) distortion is shown.

  4 (G) to 4 (I) show distortion of the subject image due to camera shake of the photographer. Specifically, FIG. 4 (G) shows horizontal camera shake, FIG. 4 (H) shows vertical camera shake, and FIG. 4 (I) shows subject image rotation due to camera shake in the rotation direction.

  In an actual captured image, an image distortion in which these various distortions are combined in accordance with the characteristics of the imaging optical system and the movement of the imaging apparatus occurs.

  Therefore, the imaging apparatus estimates these distortion states based on information such as the imaging optical system characteristic parameters and the movement of the imaging apparatus detected by a gyroscope, an acceleration sensor, etc., and deforms the captured image by a known method. By correcting the distortion, the image without distortion shown in FIG. 4A is generated and displayed on the image display unit or recorded on the image recording medium.

  Here, the subject detection process described above is executed on the image of FIG. 4A after distortion correction.

  For example, if a human face is detected, the face is detected by pattern matching with the image based on the pattern information of the arrangement of eyes, nose, mouth, etc., so an image without distortion is more suitable for matching. Because.

  On the other hand, the focus signal described above is generated from the images as shown in FIGS. 4B to 4I before distortion correction. This is because when the image is enlarged or reduced along with the deformation of the image by distortion correction, the sharpness of the video signal changes, and an accurate focus signal cannot be obtained.

  As described above, the subject detection is performed on the image after distortion correction, and the focus detection area is set on the image before distortion correction. Therefore, the following problems occur.

  For example, when a barrel-shaped distortion as shown in FIG. 4B occurs in the photographed image, the subject image at point 402 in FIG. 4B moves to point 401 in FIG. 4A after distortion correction. . Therefore, if the focus detection area is set in the image of FIG. 4B using the position where the face is detected in the image of FIG. 4A, the focus detection area of the image is moved by the distortion correction. The position may deviate from the subject image, and the detected subject may not be focused.

  In order to solve such a problem, a process reverse to the distortion correction (inverse distortion correction process) is applied to the area where the subject is detected to obtain an area on the image before distortion correction, and A method for setting a focus detection area has been proposed (see, for example, Patent Document 1).

  According to this method, since the amount of image movement due to distortion correction is returned by the inverse distortion correction processing, the focus detection area is set in the face area on the image before distortion correction, so the subject cannot be focused. Can be prevented.

  As the focus detection area, a rectangular area parallel to the four sides of the screen is generally used. This is because the focus signal is generated from the video signal by known signal processing as described below.

  First, a signal of a partial area corresponding to the inside of the focus detection area is extracted from the luminance signal for one line of the horizontal scanning line of the video signal.

  Then, a band pass filter having a predetermined pass band is applied to the extracted luminance signal, a high frequency component signal of the luminance signal is extracted, and the peak value of the signal is stored. This process is executed for the number of lines in the focus detection area in the vertical direction of the screen, and the sum of the peak values of the stored high-frequency component signals for each line is used as the focus signal.

  Thus, since the focus signal is generated based on the signals of the partial areas of the plurality of horizontal scanning lines arranged vertically, the focus detection area has a rectangular shape in which the partial areas are arranged vertically.

JP 2008-118387 A

  On the other hand, when the area on the image before distortion correction is obtained by applying the inverse distortion correction processing to the area where the subject is detected as in the conventional technique, the shape of the obtained area is the four sides of the screen. It will not be a rectangle parallel to.

  This is apparent from the fact that the square grid in the image of FIG. 4A after distortion correction has a distorted shape in FIGS. 4B to 4F before distortion correction.

  For this reason, there has been a problem that the position and size of the proper focus detection region cannot be determined simply by applying the inverse distortion correction process.

  In addition, since the inverse distortion correction processing requires complicated calculation, the calculation load of the calculation unit of the imaging apparatus increases, and the cost and power consumption of the calculation unit increase or the processing delay increases and the moving subject increases. There is also a problem that the cost and performance of the image pickup apparatus are adversely affected, such as a decrease in the following performance.

  An object of the present invention is to provide an imaging apparatus capable of appropriately setting a focus detection region for a specific subject with a small calculation load, a control method thereof, and a program in an imaging apparatus that performs image distortion correction. .

  In order to achieve the above object, an imaging apparatus according to claim 1 includes a correction unit that corrects distortion of a subject image obtained by imaging a subject, and a specific subject within the correction image corrected by the correction unit. Detecting means for detecting a specific subject area indicating a representative point, representative point calculating means for calculating coordinates of a representative point that is a point in the specific subject area detected by the detecting means, and correction by the correcting means A focal point is detected in an inverse distortion correction calculation unit that calculates a pre-correction coordinate that is a coordinate of a point in the subject image as a representative point, and an area including the pre-correction coordinate calculated by the reverse distortion correction calculation unit. And setting means for setting as a focus detection area.

  According to the present invention, in an imaging apparatus that performs image distortion correction, a focus detection area for a specific subject can be set appropriately and with a small calculation load.

1 is a diagram illustrating a schematic configuration of an imaging apparatus according to an embodiment of the present invention. It is a figure for demonstrating the method to set the focus detection area performed by the camera microcomputer in FIG. It is a flowchart which shows the procedure of the focus detection area | region setting process performed by the camera microcomputer in FIG. It is a figure which shows the image distortion example in an imaging device.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a schematic configuration of an imaging apparatus 100 according to an embodiment of the present invention.

  In FIG. 1, a camera microcomputer 110 is a microcomputer that controls the operation of the entire imaging apparatus 100, and calculation and control processing such as inverse distortion correction calculation according to the present embodiment are also executed by this microcomputer.

  The focus lens 101 adjusts the focus of the imaging optical system. In general, the imaging optical system includes a plurality of lenses including a focus lens, but other lenses are omitted here.

  The imaging element 102 is a photoelectric conversion element configured by a CCD sensor or a CMOS sensor, and the imaging signal processing unit 103 performs various image processes on the output signal of the imaging element 102 to generate a video signal.

  The distortion correction processing unit 104 performs a known distortion correction process on the output of the imaging signal processing unit 103 to correct distortion of the subject image. Various correction parameters for the distortion correction processing are set by the camera microcomputer 110. As described above, the distortion correction processing unit 104 corresponds to a correction unit that corrects distortion of a subject image obtained by imaging a subject.

  The image display unit 105 displays an image or video indicated by the video signal whose distortion has been corrected by the distortion correction processing unit 104. The image recording unit 106 records the video signal from the distortion correction processing unit 104 on a recording medium such as a semiconductor memory, a magnetic tape, and an optical disk.

  The focus detection area setting unit 107 extracts only the signal of the area used for generating the focus signal from the video signal before distortion correction from the imaging signal processing unit 103, and the focus detection area is set by the camera microcomputer 110.

  The focus signal generation unit 108 extracts a high frequency component from the signal output from the focus detection area setting unit 107 and generates a focus signal. The focus signal is a value representing the sharpness (contrast state) of the video generated based on the output signal from the image sensor 102, but the sharpness of the focused video is high, and the sharpness of the blurred video is high. Is low, and is used for focus adjustment control executed by the camera microcomputer 110 as a value representing the focus state of the imaging optical system.

  The face detection processing unit 109 performs a known face detection process on the image signal after distortion correction output from the distortion correction processing unit 104, and detects a human face area in the shooting screen. As a known face detection process, for example, a method of extracting a skin color region from the gradation color of each pixel represented by image data and detecting a face with a matching degree with a face outline pattern prepared in advance is cited. It is done. In addition, a method of performing pattern recognition from facial feature points such as the extracted eyes, nose and mouth can also be used. The face detection processing unit 109 outputs the detection result to the camera microcomputer 110. The face detection processing unit 109 corresponds to detection means for detecting a specific subject region indicating a face that is a specific subject in the corrected image.

  The focus lens drive unit 111 includes an actuator for moving the focus lens 101 and its drive circuit. As the actuator, a stepping motor, a DC motor, a vibration motor, a voice coil motor, or the like is generally used. The camera microcomputer 110 executes a focus adjustment control process by a known TV-AF method based on the focus signal output from the focus signal generation unit 108, and controls the focus lens drive unit 111 to drive the focus lens 101. By doing so, the focus adjustment of the imaging optical system is executed.

  FIG. 2 is a diagram for explaining a method of setting a focus detection area executed by the camera microcomputer 110 in FIG.

  FIG. 2A is an example of an image after distortion correction output from the distortion correction processing unit 104. A person 201 is shown in the image, and a subject area 202 that is a face area of the person detected by the face detection processing unit 109 is detected.

  FIG. 2B is an image before distortion correction output from the imaging signal processing unit 103, and the person 203 is distorted.

  When the focus detection area is set at the same position as the subject area 202 detected in FIG. 2A, the focus detection area 204 is a rectangular area indicated by a dotted line in the drawing. However, since the position of the person image is moved due to image distortion, the focus detection area 204 includes not only the face but also the background area.

  In this case, if a subject with high contrast is captured in the background region, focus adjustment control is executed so that the background is in focus, and the person is out of focus.

  2C is an image before distortion correction as in FIG. 2B, and the subject area 205 indicated by the dotted line in the figure is subjected to inverse distortion correction processing to the subject area 202 detected in FIG. The area | region obtained by applying is shown.

  Since the subject area 205 in this case has a distorted shape, the object area 205 is not a rectangular area parallel to the four sides of the screen, which is a preferable shape as the focus detection area, and cannot be applied as the focus detection area. is there.

  In this case, since the inverse distortion correction process is applied to the entire subject area 202, the calculation load increases.

  On the other hand, according to the setting of the focus detection area performed by the camera microcomputer 110 according to the present embodiment, the process of setting the focus detection area is performed as follows.

  FIG. 2D is an example of the same image after distortion correction as in FIG. 2A, and the subject region 206 is detected.

  Here, the point 207 is a point located at the center of the subject area 205, and hereinafter, the point 207 is a representative point representing the position of the subject area 205. A point on the image before distortion correction obtained by applying inverse distortion correction processing to the representative point 207 is a corresponding point 208 in FIG.

  As can be seen from the figure, the corresponding point 208 is substantially the center of the face area on the image before distortion correction. Furthermore, a rectangular area 209 indicated by a dotted line in FIG. 2E is an area having the same size as the subject area 205 detected from the image after distortion correction in FIG. Yes.

  The rectangular area 209 obtained in this way substantially includes the face area on the image before distortion correction, and is a rectangular area parallel to the four sides of the screen. For this reason, by using the rectangular area 209 as the focus detection area, it is possible to focus the person correctly. The rectangular area 209 translates the specific subject area (subject area 206) from the coordinates of the representative point 207 toward the pre-correction coordinates (coordinates of the corresponding points 208) by the distance between the representative point coordinates and the pre-correction coordinates. This is a parallel movement region. The specific subject area (subject area 206), the focus detection area (rectangular area 209), and the subject image (entire part of FIG. 2E) have a rectangular shape, and two of the two parallel sides of the subject image are parallel. One set of two sides is parallel to one of the two parallel sides of the focus detection area, and the other two parallel sides of the subject image are It is parallel to the other set of two parallel sides.

  Note that a rectangular area 210 indicated by a dotted line in FIG. 2F is a shape in which the focus detection area is more easily focused on the person 203.

  This rectangular area 210 is an area that is slightly narrower than the rectangular area 209 obtained in FIG. 2 (E) by multiplying a predetermined coefficient in the width direction. This is an area extended by a ratio.

  The subject area 206 detected by the face detection processing unit 109 is not necessarily a size suitable for focus adjustment control, and is too large for the face area to include a lot of background areas or too small. May not be suitable for focus adjustment.

  Therefore, by multiplying the width and height of the subject area 206 by a predetermined coefficient, a focus detection area more suitable for focus adjustment control can be obtained. That is, with respect to the parallel movement region, one set of two sides of the two parallel sides of the parallel movement region is multiplied by a predetermined coefficient, and the other set of two sides is set to the other side. A rectangle obtained by multiplying a predetermined coefficient may be set as the focus detection area.

  In addition, when focusing on a person, by extending the focus detection area to the bottom of the screen, the torso part can be included in the focus detection area in addition to the face area, and the subject can be focused more reliably. . That is, a rectangle in which two sides of the vertical translation region in the correction image are further extended downward with respect to the translation region may be set as the focus detection region.

  Since these processes only apply enlargement / reduction and parallel movement at a predetermined ratio to the rectangular area 209 obtained in FIG. 2 (E), it is possible to execute these processes with a slight calculation load.

  Thus, in the setting of the focus detection area performed by the camera microcomputer 110, a point before distortion correction corresponding to the representative point of the subject area on the image after distortion correction is obtained by reverse distortion correction processing. A rectangular area based on the position of the point before distortion correction is used as the focus detection area.

  This makes it possible to set a focus detection area that can be correctly focused on the subject. Furthermore, since the inverse distortion correction calculation is executed only for the representative points, the calculation load can be reduced.

  FIG. 3 is a flowchart showing a procedure of focus detection area setting processing executed by the camera microcomputer 110 in FIG.

  In FIG. 3, information on the subject area 206 output from the face detection processing unit 109 is acquired (step S301). This information includes the coordinates of the reference point of the area (for example, the upper left vertex of the rectangular area), the width and height of the area, and the like.

  Next, the coordinates of the representative point 207 of the subject area 206 are calculated (step S302). As described above, the center point of the subject area 206 is preferable as the representative point 207, but it may not necessarily be the center point as long as it is suitable for setting the focus detection area. This step S302 corresponds to representative point calculation means for calculating the coordinates of a representative point that is a point within the detected specific subject area.

  Then, inverse distortion correction processing is applied to the coordinates of the representative point 207 to calculate the coordinates (pre-correction coordinates) of the corresponding point 208 on the image before distortion correction (step S303). This step S303 corresponds to inverse distortion correction calculation means for calculating the pre-correction coordinates that are the coordinates of the points in the subject image that become the representative points by being corrected by the distortion correction processing unit 104.

  Next, the rectangular area 209 with the coordinates of the corresponding point 208 as a reference is calculated (step S304). As described above, the rectangular area 209 can be obtained as an area having the same width and height as the subject area 206 with the corresponding point 208 as the center, but is not limited thereto.

  Since the image distortion increases as the distance from the center point of the image increases, the center of the rectangular area 209 may be shifted from the corresponding point 208 as the corresponding point 208 moves away from the image center, for example.

  In addition, as described above, the width and height of the area may be enlarged or reduced by multiplying the width and height of the subject area 206 by a predetermined coefficient, respectively, or may be expanded downward in the screen where the body is located. May be.

  Then, the focus detection area is set in the focus detection area setting unit 107 so that the rectangular area 209 calculated in step S304 becomes the focus detection area (step S305), and this process ends. Steps S304 and S305 correspond to setting means for setting a region including the calculated pre-correction coordinates as a focus detection region for detecting a focus.

  The focus detection area setting described above is repeatedly executed by the camera microcomputer 110 at predetermined intervals such as a generation cycle of a vertical synchronizing signal of a video signal or an integer multiple thereof.

  In this way, even when the subject or the imaging apparatus 100 moves and the position of the subject area on the screen moves, it is possible to keep the focus on the subject by following the position of the focus detection area.

  In the embodiment described above, a person's face is detected as the subject detection means. However, any object can be used as long as it can detect the subject area included in the video. For example, a part of a human body other than the face may be detected, or a specific color or pattern area may be detected.

  As described above, according to the present embodiment, the position on the image before distortion correction corresponding to the representative point of the subject area detected from the image after distortion correction is obtained by inverse distortion correction calculation, and A focus detection area is set based on the position. As a result, the shape of the focus detection region suitable for acquiring the focus signal can be set, and the calculation load for executing the inverse distortion correction calculation can be reduced.

  The embodiment described above can be applied to a video camera, a digital still camera, a mobile terminal with a camera, and the like.

(Other embodiments)
The present invention is also realized by executing the following processing. That is, software (program) that realizes the functions of the above-described embodiments is supplied to a system or apparatus via a network or various storage media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program code. It is a process to be executed. In this case, the program and the storage medium storing the program constitute the present invention.

DESCRIPTION OF SYMBOLS 100 Imaging device 101 Focus lens 102 Image sensor 103 Imaging signal processing part 104 Distortion correction processing part 105 Image display part 106 Image recording part 107 Focus detection area setting part 108 Focus signal generation part 109 Face detection processing part 110 Camera microcomputer 111 Focus lens drive Part

Claims (18)

Correction means for correcting distortion of the subject image obtained by imaging the subject;
Detecting means for detecting a specific subject area indicating a specific subject in the corrected image corrected by the correcting means;
Representative point calculation means for calculating coordinates of a representative point which is a point in the specific subject area detected by the detection means;
Inverse distortion correction calculating means for calculating a pre-correction coordinate which is a coordinate of a point in the subject image that becomes the representative point by being corrected by the correction means;
An imaging apparatus comprising: setting means for setting, as a focus detection area for detecting a focus, an area including coordinates before correction calculated by the inverse distortion correction calculation means.   The setting means includes a translation area obtained by translating the specific subject area by a distance between the coordinates of the representative point and the coordinates before correction from the coordinates of the representative point toward the coordinates before correction. The imaging apparatus according to claim 1, wherein the imaging apparatus is set as a region.   The specific subject region, the focus detection region, and the subject image have a rectangular shape, and two sets of two parallel sides of the subject image have two sets of the focus detection region. The two parallel sides are parallel to one set of sides, and the other two sets of parallel sides of the subject image are parallel to the other set of two parallel sides of the focus detection region. The imaging apparatus according to claim 2.   The setting means further multiplies the parallel movement area by a predetermined coefficient to one of two sets of two parallel sides of the parallel movement area, and sets the other set. 4. The imaging apparatus according to claim 3, wherein a rectangle obtained by multiplying two sides by another predetermined coefficient is set as the focus detection region.   The setting means sets, as the focus detection area, a rectangle obtained by extending two sides of the parallel movement area in the vertical direction in the correction image downward with respect to the parallel movement area. Item 4. The imaging device according to Item 3.   The imaging apparatus according to claim 1, wherein the representative point is a center point of the specific subject area. A method for controlling an imaging apparatus,
A correction step for correcting distortion of the subject image obtained by imaging the subject;
A detection step of detecting a specific subject region indicating a specific subject in the corrected image corrected by the correction step;
A representative point calculating step of calculating coordinates of a representative point that is a point in the specific subject area detected by the detecting step;
A reverse distortion correction calculating step for calculating a pre-correction coordinate that is a coordinate of a point in the subject image that becomes the representative point by being corrected by the correction step;
A control method comprising: a setting step of setting a region including the pre-correction coordinates calculated by the inverse distortion correction calculation step as a focus detection region for detecting a focus.   In the setting step, the specific subject region is translated from the coordinate of the representative point toward the pre-correction coordinate in the direction of the pre-correction coordinate by a distance between the representative point coordinate and the pre-correction coordinate. The control method according to claim 7, wherein the control method is set as a region.   The specific subject region, the focus detection region, and the subject image have a rectangular shape, and two sets of two parallel sides of the subject image have two sets of the focus detection region. The two parallel sides are parallel to one set of sides, and the other two sets of parallel sides of the subject image are parallel to the other set of two parallel sides of the focus detection region. The control method according to claim 8.   In the setting step, one set of two sides of two parallel sides of the translation region is further multiplied by a predetermined coefficient with respect to the translation region, and the other set The control method according to claim 9, wherein a rectangle obtained by multiplying the two sides by another predetermined coefficient is set as the focus detection region.   The setting step is characterized in that, with respect to the parallel movement area, a rectangle obtained by extending two sides of the parallel movement area in the vertical direction in the correction image downward is set as the focus detection area. Item 10. The control method according to Item 9.   The imaging apparatus according to claim 7, wherein the representative point is a center point of the specific subject area. A program for causing a computer to execute a control method of an imaging apparatus,
The control method is:
A correction step for correcting distortion of the subject image obtained by imaging the subject;
A detection step of detecting a specific subject region indicating a specific subject in the corrected image corrected by the correction step;
A representative point calculating step of calculating coordinates of a representative point that is a point in the specific subject area detected by the detecting step;
A reverse distortion correction calculating step for calculating a pre-correction coordinate that is a coordinate of a point in the subject image that becomes the representative point by being corrected by the correction step;
A program comprising: a setting step of setting an area including the pre-correction coordinates calculated by the inverse distortion correction calculating step as a focus detection area for detecting a focus.   In the setting step, the specific subject region is translated from the coordinate of the representative point toward the pre-correction coordinate in the direction of the pre-correction coordinate by a distance between the representative point coordinate and the pre-correction coordinate. 14. The program according to claim 13, wherein the program is set as an area.   The specific subject region, the focus detection region, and the subject image have a rectangular shape, and two sets of two parallel sides of the subject image have two sets of the focus detection region. The two parallel sides are parallel to one set of sides, and the other two sets of parallel sides of the subject image are parallel to the other set of two parallel sides of the focus detection region. 15. The program according to claim 14, wherein:   In the setting step, one set of two sides of two parallel sides of the translation region is further multiplied by a predetermined coefficient with respect to the translation region, and the other set 16. The program according to claim 15, wherein a rectangle obtained by multiplying two sides by another predetermined coefficient is set as the focus detection area.   The setting step is characterized in that, with respect to the parallel movement area, a rectangle obtained by extending two sides of the parallel movement area in the vertical direction in the correction image downward is set as the focus detection area. Item 15. The program according to Item 15.   The image pickup apparatus according to claim 13, wherein the representative point is a center point of the specific subject area.

Images