JP4981758B2 - Focus adjustment apparatus and method - Google Patents

Description

  The present invention relates to a focus adjustment device and a focus adjustment method used in an imaging apparatus, and more particularly to a focus adjustment device and a focus adjustment method for performing automatic focus adjustment based on an image obtained at the time of moving image shooting.

  In autofocus (AF) control of a video camera or the like, an AF evaluation value signal indicating the sharpness (contrast state) of a video signal generated using an image sensor is generated, and a focus lens that maximizes the AF evaluation value signal The TV-AF method for searching for the position of the TV is the mainstream.

  However, when a person is photographed, there is a problem that the person is not in focus and is in the background because of the contrast between the person who is the main subject and the background.

  In order to solve such a problem, an imaging apparatus having a face recognition function is known. For example, an imaging device that sets a focus detection area including a recognized face area and performs AF (see, for example, Patent Document 1), or an imaging device that detects the eyes of a person and performs AF based on the eyes ( For example, see Patent Document 2).

JP 2006-227080 A JP 2001-215403 A

  However, in a video camera that shoots a moving image, unlike in the case of shooting a still image, AF may become unstable when the recognition of a human face is not stable. For example, when the face moves, the main face moves from the center to the end due to camera shake, etc., or when it moves backward and the size of the face decreases, it immediately focuses on the background. As a result, the stability of AF may decrease.

  The present invention has been made in view of the above problems, and an object of the present invention is to enable stable focus control even when the position or size of an object such as a face changes.

To achieve the above object, focusing device of the present invention, the image represented by the image signal obtained by photographing, a detecting means for detect a predetermined object, from the image signal, AF evaluation value generating means for taking out a high frequency component and generating an AF evaluation value; and automatic focus adjusting means for performing focus adjustment using the AF evaluation value. The AF evaluation value generating means is detected by the detecting means. A first setting means for setting a detected object area; a second setting means for setting a focus detection area used for detecting a focus state including the object area; and the object area; For each of the focus detection areas, a calculation unit that calculates an AF evaluation value for each area, and at least one of the position and size of the area of the object, the area of the object relative to the focus detection area A determination means for determining a calculation ratio; and based on the determined addition ratio, the AF evaluation value for each area of the object area and the AF evaluation value for each area of the focus detection area are weighted and added, and the automatic focus A generating means for generating an AF evaluation value used by the adjusting means, and the determining means is determined to be decreased when the addition ratio obtained this time is less than the previously obtained addition ratio. The previously set addition ratio determined last time is used.

The focal adjustment method of the present invention, the detection means, the image represented by the image signal obtained by photographing, a detection step of detect a predetermined object, AF evaluation value generation means, An AF evaluation value generating step for extracting a high frequency component from the image signal and generating an AF evaluation value, and an automatic focus adjustment means, wherein the automatic focus adjustment means has an automatic focus adjustment step for performing focus adjustment using the AF evaluation value, In the AF evaluation value generation step, a first setting step for setting the region of the object detected in the detection step, and a second setting for setting a focus detection region used for detection of the focus state including the region of the object. For each of the object region and the focus detection region, a calculation step for calculating an AF evaluation value for each region, and at least one of the position and the size of the region of the object, Focus test A determination step of determining an addition ratio of the area of the object to the exit area, and an AF evaluation value for each area of the object area and an AF evaluation value for each area of the focus detection area based on the determined addition ratio A generation step of generating an AF evaluation value used in the automatic focus adjustment step by weighted addition, and the determination step decreases when the addition ratio obtained this time is less than the addition ratio obtained last time. It is characterized in that the previously set addition ratio is used for a preset time after it is determined that the

  As described above, according to the present invention, focus control can be stably performed even when the position or size of an object such as a face changes.

  The best mode for carrying out the present invention will be described below in detail with reference to the accompanying drawings. However, the dimensions, shapes, relative arrangements, and the like of the components exemplified in the present embodiment should be changed as appropriate according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to.

  FIG. 1 is a block diagram illustrating a configuration of a video camera as an example of an imaging apparatus according to an embodiment of the present invention. Note that although a video camera is described in this embodiment, the present invention can also be applied to other imaging apparatuses that can continuously acquire images, such as a digital still camera.

  In FIG. 1, 101 is a first fixed lens, 102 is a variable magnification lens that moves in the optical axis direction and performs variable magnification, and 103 is a diaphragm. Reference numeral 104 denotes a second fixed lens, and reference numeral 105 denotes a focus compensator lens (hereinafter referred to as “focus lens”) having both a function of correcting the movement of the focal plane due to zooming and a focusing function. The first fixed lens 101, the variable magnification lens 102, the diaphragm 103, the second fixed lens 104, and the focus lens 105 constitute an imaging optical system.

  Reference numeral 106 denotes an image sensor as a photoelectric conversion element constituted by a CCD sensor or a CMOS sensor. Reference numeral 107 denotes a CDS / AGC circuit that samples the output of the image sensor 106 and adjusts the gain. A camera signal processing circuit 108 performs various image processing on the output signal from the CDS / AGC circuit 107 to generate a video signal.

  Reference numeral 109 denotes a monitor constituted by an LCD or the like, which displays a video signal from the camera signal processing circuit 108. A recording unit 115 records the video signal from the camera signal processing circuit 108 on a recording medium such as a magnetic tape, an optical disk, or a semiconductor memory.

  Reference numeral 110 denotes a zoom drive source for moving the variable magnification lens 102, and reference numeral 111 denotes a focusing drive source for moving the focus lens 105. The zoom drive source 110 and the focusing drive source 111 are configured by actuators such as a stepping motor, a DC motor, a vibration motor, and a voice coil motor.

  Reference numeral 112 denotes an AF gate that passes only signals in an area (focus detection area) used for focus detection among output signals of all pixels from the CDS / AGC circuit 107. The AF signal processing circuit 113 extracts a high frequency component, a luminance difference component (difference between the maximum value and the minimum value of the luminance level of the signal that has passed through the AF gate 112), and the like from the signal that has passed through the AF gate 112, and an AF evaluation value Generate a signal. The AF evaluation value signal is output to the camera / AF microcomputer 114. The AF evaluation value signal represents the sharpness (contrast state) of the image generated based on the output signal from the image sensor 106, but the sharpness changes depending on the focus state of the image pickup optical system. The signal represents the focus state of the imaging optical system.

  The camera / AF microcomputer 114 controls the entire operation of the video camera, and performs AF control for moving the focus lens 105 by controlling the focusing drive source 111 based on the AF evaluation value signal.

  The face detection unit 116 performs a known face recognition process on the image signal, detects a face area in the shooting screen, and transmits the detection result to the camera / AF microcomputer 114. The camera / AF microcomputer 114 sets the focus detection area so as to include the detected face area based on the transmitted face area detection result, and transmits information to the AF gate 112. As a face recognition process, for example, a method is known in which a skin color region is extracted from the gradation color of each pixel represented by image data, and the face is detected with a matching degree with a face contour plate prepared in advance. ing. In addition, there is a method of detecting a face by extracting feature points of a face such as eyes, nose and mouth using a known pattern recognition technique, but the present invention is not limited to face recognition processing. Any method may be used.

  Next, focus adjustment control (AF control) performed by the camera / AF microcomputer 114 will be described with reference to FIG. The AF control is executed according to a computer program stored in the camera / AF microcomputer 114.

  When AF control is started, a face detection result is taken from the face detection unit 116 in step S201. In step S202, the face detection unit 116 determines whether face detection is successful. If successful, the process proceeds to step S203, and if not, the process proceeds to step S218.

  In step S203, a face frame indicating the face area detected based on the face detection result is set (first setting step), and an AF frame indicating the focus detection area including the detected face area is set. (Second setting step). Here, it is assumed that a normal frame having a preset size at the center of the screen is set as an initial state of the AF frame, and the face frame whose size is set without changing the center of the normal frame. The AF frame is set by enlarging it to include. In step S204, an AF evaluation value (area-specific AF evaluation value) is acquired from the image signals in the face frame and the AF frame (calculation step). In step S205, an addition ratio of the AF evaluation value for the face frame to the AF frame is obtained according to the position of the face frame on the screen. Here, the closer to the center of the screen, the higher the addition ratio, and the lower the area near the screen, the lower the addition ratio.

  In step S206, it is determined whether or not the addition ratio obtained this time has decreased compared to the addition ratio based on the face frame position obtained last time. If not decreased, the process proceeds to step S207, and if decreased, the process proceeds to step S208.

  In step S207, it is determined whether a predetermined time has elapsed since it was first determined that the addition ratio obtained in step S205 has decreased. If the predetermined time has elapsed, the process proceeds to step S208, and if not, the process proceeds to step S209. In step S208, the AF evaluation value of the face frame is recalculated using the addition ratio corresponding to the current face frame position, and the process proceeds to step S210. On the other hand, in step S209, the AF evaluation value of the face frame is recalculated using the previously set addition ratio, and the process proceeds to step S210. As a result, when the face temporarily moves in the peripheral direction of the screen, the in-focus state can be maintained, and when the face is recognized closer to the center of the screen than the previous time, the face is immediately adjusted to the recognized face. Can be burnt.

  Further, in step S210, the addition ratio for the face frame with respect to the AF frame is obtained according to the size of the face frame. Here, the larger the face frame, the higher the addition ratio, and the smaller the face frame, the lower the addition ratio. In step S211, it is determined whether or not the addition ratio obtained this time is smaller than the addition ratio based on the size of the face frame obtained last time. If it has not decreased, the process proceeds to step S213, and if it has decreased, the process proceeds to step S212.

  In step S212, it is determined whether a predetermined time has elapsed since it was first determined that the addition ratio obtained in step S210 has decreased. If the predetermined time has elapsed, the process proceeds to step S213, and if not, the process proceeds to step S214. In step S213, the AF evaluation value of the face frame is recalculated using the addition ratio corresponding to the current face frame size, and the process proceeds to step S215. On the other hand, in step S214, the AF evaluation value of the face frame is recalculated using the previously set addition ratio, and the process proceeds to step S215. As a result, the focused state can be maintained even if the face temporarily becomes smaller on the screen, and when a larger face is recognized on the screen than the previous time, the recognized face is immediately focused on. Can do.

In step S215, the AF evaluation value in the face frame obtained as described above and the AF evaluation value in the AF frame are weighted and added to obtain a final AF evaluation value (AF evaluation value generation step). . For example, the addition ratio based on the position of the face frame relative to the AF frame is a, the addition ratio based on the size is b, the AF evaluation value of the face frame obtained in step S213 or S214 is x, and the AF frame obtained in step S204. If the AF evaluation value of y is y, the final AF evaluation value is
Final AF evaluation value = a × b × x + y

It can ask for.
In step S216, AF control is performed using the AF evaluation value obtained in step S215 (automatic focus adjustment process).

  On the other hand, if it is determined in step S202 that face detection has failed, it is determined in step S218 whether face detection failure has continued for a certain period of time. If it is not continuous for a certain time, the process proceeds to step S219, and if it is continuous, the process proceeds to step S221. In step S219, the AF frame is maintained at the size of the AF frame set in the previous process, and in step S220, an AF evaluation value in the AF frame is acquired. In step S216, AF control is performed using the AF evaluation value acquired in step S220.

  In step S221, the AF frame is set to the normal frame in the initial state, and the AF evaluation value in the normal frame is acquired in step S222. In step S216, AF control is performed using the AF evaluation value acquired in step S222.

  Next, the AF frame setting described in FIG. 2 will be described with reference to FIG.

  First, in the initial state, as shown in FIG. 3A, the AF frame is set to a normal frame near the center of the screen. This is because the photographer generally places the subject at the center of the screen, and in this embodiment, the area at the center of the screen is determined as the normal frame. However, the present invention is not limited to this, and the focus Any region that is effective as a detection region may be used.

  Next, a face frame is set based on the detection result of the face recognition process. For example, when the face frame is at the position shown in FIG. 3B, the AF frame is set by enlarging the normal frame to the size including the face frame without changing the center of the normal frame (step S203).

  If a face is not detected as a result of the face recognition process, and if a predetermined time has not passed since the face is not detected, the setting is made in FIG. 3B as shown in FIG. The performed AF frame is maintained as it is (step S219). As shown in FIG. 3B, since the area having the face frame is also included in the AF frame, the fluctuation of the AF evaluation value when the face recognition process cannot be temporarily performed can be reduced.

  On the other hand, when a certain time has elapsed since the face is no longer detected, the AF frame is returned to the normal frame as shown in FIG. 3D (step S221), and an AF evaluation value is acquired. Thereby, center priority AF is realizable.

  In this way, the AF frame is set so that priority is given to the center subject in moving image shooting. When a face is recognized, the AF evaluation value obtained by weighted addition of the AF evaluation value of the AF frame and the face frame is used for AF. Control is performed to focus on the recognized face. Further, when the face is not recognized, the in-focus state of the screen can be optimally controlled by using the AF frame including the latest face frame as it is for a certain time after the face is not detected.

  Next, the addition ratio will be described. Since a photographer often shoots a subject to be photographed mainly near the center of the screen, the face near the center is highly likely to be a face to be photographed. Therefore, a data table is prepared in advance so that the addition ratio is high near the center of the screen. In addition, since the face to be photographed mainly occupies a large size on the screen, there is a high possibility that the large face is the face to be photographed. Therefore, a data table is prepared in advance so that the addition ratio of a large face is high. If the face frame is large, the addition ratio is high, and if the face frame is small, the addition ratio is low.

  Here, the feature of the change in the addition ratio in the present embodiment will be described with reference to FIG. 4 as an example of the case where the face frame moves in the peripheral direction of the screen.

  For example, when the subject's face moves in the peripheral direction of the screen as shown in FIG. 4B from the state where the subject's face is at the center as shown in FIG. 4A, the preset time is added. Do not reduce the ratio. Accordingly, the focused state can be maintained even when the subject moves temporarily or the face temporarily moves from the center to the end due to camera shake of the photographer.

  Conversely, when the main face moves from the state of FIG. 4B to the center of the screen as shown in FIG. 4A, it is possible to immediately focus on the face by immediately increasing the addition ratio. .

  Similarly, even if the face frame becomes smaller, the focus state can be maintained when the face temporarily becomes smaller by not reducing the addition ratio for a predetermined time. Further, when the size of the face increases, it is possible to immediately focus on the face by increasing the addition ratio immediately without waiting for a predetermined time.

  In this embodiment, as an example of the detection unit, the human face is detected by the face detection unit 116, but a specific target other than the human face, such as an animal or a car, is detected. It doesn't matter if you do. Furthermore, the position in the imaging screen may be input from an external input means, or the position of the imaging screen may be determined by detecting the line of sight of the photographer who is looking at the viewfinder, thereby detecting the object.

  In the above-described embodiment, a case where all of the process of changing the size of the AF frame according to whether the face detection is successful and the process of changing the addition ratio according to the position and size of the face frame are performed. explained. However, it is of course possible to configure to perform any one or two of these processes.

<Other embodiments>
Note that the present invention can be applied to a system including a plurality of devices (for example, a host computer, an interface device, a camera head, etc.), or a device (for example, a digital video camera, a digital still camera, etc.) including a single device. You may apply to.

  The object of the present invention can also be achieved as follows. First, a storage medium (or recording medium) that records a program code of software that implements the functions of the above-described embodiments is supplied to a system or apparatus. Then, the computer (or CPU or MPU) of the system or apparatus reads and executes the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention.

  Further, by executing the program code read by the computer, not only the functions of the above-described embodiments are realized, but also the following can be achieved. That is, when the operating system (OS) running on the computer performs part or all of the actual processing based on the instruction of the read program code, the functions of the above-described embodiments are realized by the processing. It is. Examples of the storage medium for storing the program code include a flexible disk, hard disk, ROM, RAM, magnetic tape, nonvolatile memory card, CD-ROM, CD-R, DVD, optical disk, magneto-optical disk, MO, and the like. Can be considered. Also, a computer network such as a LAN (Local Area Network) or a WAN (Wide Area Network) can be used to supply the program code.

It is a block diagram which shows the structure of the video camera in embodiment of this invention. It is a flowchart which shows AF control by the camera AF microcomputer in embodiment of this invention. It is a figure explaining the setting control of AF frame in an embodiment of the invention. It is a figure explaining the setting control of the addition ratio in embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 1st fixed lens 102 Variable magnification lens 103 Aperture 104 2nd fixed lens 105 Focus lens 106 Image pick-up element 107 CDS / AGC circuit 108 Camera signal processing circuit 109 Monitor 110 Zoom drive source 111 Focusing drive source 112 AF gate 113 AF signal processing circuit 114 Camera / AF microcomputer 115 Recording unit 116 Face detection unit

Claims (9)

The image represented by the image signal obtained by photographing, a detecting means for detect a predetermined object,
AF evaluation value generating means for extracting a high frequency component from the image signal and generating an AF evaluation value;
Automatic focus adjustment means for performing focus adjustment using the AF evaluation value,
The AF evaluation value generating means includes
First setting means for setting a region of the object detected by the detection means;
A second setting means for setting a focus detection area used for detection of a focus state, including the area of the object;
For each of the object area and the focus detection area, a calculation means for calculating an AF evaluation value for each area;
Determining means for determining an addition ratio of the area of the object to the focus detection area based on at least one of a position and a size of the area of the object;
Based on the determined addition ratio, the AF evaluation value for each region of the object region and the AF evaluation value for each region of the focus detection region are weighted and added, and an AF evaluation value used by the automatic focus adjustment unit is obtained. Generating means for generating,
The determining means uses the previously determined addition ratio for a preset time after it is determined that the addition ratio determined this time has decreased when the addition ratio determined this time is less than the previously determined addition ratio. Focusing device.   The said 2nd setting means does not change the magnitude | size of the said focus detection area for the preset time after it stops being detected from the state in which the said target object was detected. Focusing device.   The focus adjustment apparatus according to claim 1, wherein the determination unit determines a higher addition ratio as the region of the object is closer to the center.   The focus adjustment apparatus according to claim 1, wherein the determination unit determines a higher addition ratio as the size of the region of the object is larger.   The focus adjustment apparatus according to claim 1, wherein the predetermined object is a human face. Detection means, the image represented by the image signal obtained by photographing, a detection step of detect a predetermined object,
An AF evaluation value generating means for extracting a high frequency component from the image signal and generating an AF evaluation value;
An automatic focus adjustment means, and an automatic focus adjustment step of performing focus adjustment using the AF evaluation value,
In the AF evaluation value generation step,
A first setting step for setting a region of the object detected in the detection step;
A second setting step for setting a focus detection region used for detection of a focus state, including the region of the object;
For each of the object area and the focus detection area, a calculation step for calculating an AF evaluation value for each area;
A determination step of determining an addition ratio of the area of the object to the focus detection area based on at least one of the position and size of the area of the object;
Based on the determined addition ratio, the AF evaluation value for each region of the object region and the AF evaluation value for each region of the focus detection region are weighted to generate an AF evaluation value used in the automatic focus adjustment step. A production process,
In the determining step, when the addition ratio obtained this time is less than the addition ratio obtained last time, the addition ratio determined last time is used for a preset time after it is determined that the addition ratio has been reduced. Focus adjustment method.   The size of the focus detection area is not changed in the second setting step for a preset time after the object is no longer detected from the detected state. Focus adjustment method.   The program for making a computer perform each process of the focus adjustment method of Claim 6 or 7.   A computer-readable storage medium storing the program according to claim 8.

Images