JP5913902B2 - Automatic focusing device and automatic focusing method - Google Patents

Description

According to the present invention, when an imaging target includes a predetermined subject such as a person, a feature part (such as a face part) such as a person is extracted, and automatic focusing is performed in an imaging apparatus that performs focus adjustment in consideration of the result. The present invention relates to an apparatus and a method.

In autofocus (AF) control, which is automatic focus adjustment for a video camera or the like, the following TV-AF system is the mainstream. That is, an AF evaluation value signal indicating the sharpness (contrast state) of a video signal generated by photoelectrically converting light from a subject using an image sensor, and a focus lens that maximizes the AF evaluation value signal Search for the location of. However, when a person is photographed in an imaging apparatus having a face detection function, it is difficult to obtain a sufficient AF evaluation value because a human face generally has low contrast. Therefore, when a change in the distance to the subject occurs, it is difficult to specify the in-focus direction with a change in the AF evaluation value, and the subject image is blurred because the lens position cannot follow the subject. was there.

In order to solve such problems, the detected face sizes are compared, and if the face size increases or decreases, it is determined that the subject has moved closer or infinitely and follows the subject. A method of driving a focus lens has been proposed (see Patent Document 1 and Patent Document 2).

JP 2009-31760 JP JP 2008-276214 A

However, in the autofocus using the face detection described above, since the shooting parameters such as the depth of field and the state of the subject are not considered, the following can occur. In other words, depending on the detected size variation of the face and the setting of the parameters at the time of shooting, the direction of the subject's distance change may be misjudged, or even if there is no blur on the subject, movement determination is unnecessary. The accuracy of autofocus may be reduced.

The present invention has been made in view of the above points. The purpose is to perform AF on a subject person using the TV-AF method in moving image shooting and the like, and when a change in the distance of the subject occurs, determine the distance change in consideration of the state of the subject to stabilize AF. It is an object of the present invention to provide an automatic focusing apparatus and method capable of improving the performance and accuracy.

The automatic focus adjustment apparatus of the present invention is set among an image sensor that photoelectrically converts light from a subject that has passed through an imaging optical system including a focus lens into an electric signal, and an electric signal output from the image sensor. Generating means for generating an AF evaluation value from the electrical signal corresponding to the focus detection area, detecting means for detecting a predetermined subject image from the electrical signal output from the image sensor, and detection of the subject detected by the detecting means Discriminating means for discriminating whether or not there is a distance change in the optical axis direction, control means for controlling the focus by driving the focus lens based on the AF evaluation value, and a detection result by the detecting means is a stable state of the subject Subject stability determination means for determining whether or not the result is The subject stability determination means includes subject information storage means for storing the size or position of the subject image detected by the detection means, and the latest size or position and time of the subject image detected by the detection means. First difference calculation means for calculating a difference value with the size or position of the subject image previously stored in the subject information storage means at an interval, and calculation of the difference value by the first difference calculation means A difference storage means for storing a result, and a change amount between a latest difference value stored in the difference storage means and a difference value previously stored at a time interval. It is determined that the subject is stable when the detected steepness of the size or position of the subject image is less than a threshold value for a predetermined time continuously. The control unit detects the detection unit when driving the focus lens in accordance with a change in the distance in the optical axis direction of the subject detected by the detection unit according to a determination result by the determination unit. If the amount of change in the size of the subject image when the subject stability determination means determines that the subject is stable is greater than a predetermined amount, a follow-up driving operation corresponding to the change in the distance of the subject in the optical axis direction Is controlled by the focus lens, and the amount of change in the size of the subject image detected by the detection unit when the subject stability determination unit determines that the subject is stable is smaller than the predetermined amount. In this case, the following driving operation is limited in accordance with the change in the distance of the subject in the optical axis direction detected by the detecting means.

The automatic focus adjustment method of the present invention includes a step of photoelectrically converting light from a subject that has passed through an imaging optical system including a focus lens into an electrical signal, and setting among electrical signals output in the conversion step A generation step of generating an AF evaluation value from an electrical signal corresponding to the focus detection area, a detection step of detecting a predetermined subject image from the electrical signal output in the conversion step, and the subject detected in the detection step A determination step for determining whether or not there is a change in the distance in the optical axis direction, a control step for driving and controlling the focus lens based on the AF evaluation value, and a detection result in the detection step A subject stability determination step for determining whether or not the result is in a stable state. The subject stability determination step includes a subject information storage step for storing the size or position of the subject image detected in the detection step, and a latest size or position and time of the subject image detected in the detection step. A first difference calculation step for calculating a difference value between the size or position of the subject image stored in the subject information storage step at an interval before and a difference value obtained by calculation in the first difference calculation step A difference storage step for storing the calculation result of the difference between the latest difference value stored in the difference storage step and the difference value stored previously with a time interval. When the steepness of change in the size or position of the subject image detected in the detection step, which is the amount of change, is less than a threshold value for a predetermined time continuously The serial subject is determined to be stable. In the control step, when the focus lens is driven and controlled in accordance with the distance change in the optical axis direction of the subject detected in the detection step according to the determination result in the determination step, the detection is detected in the detection step, If the amount of change in the size of the subject image when the subject is determined to be stable in the subject stability determination step is greater than a predetermined amount, a follow-up driving operation corresponding to the change in the distance of the subject in the optical axis direction The focus lens drive control is performed, and the amount of change in the size of the subject image that is detected in the detection step and determined that the subject is stable in the subject stability determination step is smaller than the predetermined amount. In this case, the following driving operation is limited in accordance with the change in the distance of the subject in the optical axis direction detected in the detection step.

As described above, according to the present invention, the state of the subject is analyzed, and the presence / absence of a change in the distance of the subject is determined using information on the subject in a stable state, so that erroneous determination is reduced and unnecessary. Judgment can be avoided. As a result, the stability and accuracy of autofocus can be improved.

1 is a block diagram illustrating a configuration of a video camera that is an embodiment of the present invention. The flowchart which shows the process of the camera / AF microcomputer of one Example of this invention. The flowchart which shows the process of the micro drive operation | movement of one Example of this invention. The figure which shows the micro drive operation | movement of one Example of this invention. The flowchart which shows the process of the mountain climbing operation | movement of one Example of this invention. The figure which shows the mountain climbing operation | movement of one Example of this invention. The flowchart which shows the process of the movement determination of one Example of this invention. The flowchart which shows the process of the stability determination of one Example of this invention. The flowchart which shows the process of the movement determination threshold value setting of one Example of this invention. The figure which shows the threshold value according to the zoom lens position of one Example of this invention. The figure which shows the correction coefficient K according to the reference | standard face size of one Example of this invention. The figure which shows the correction coefficient K 'according to the imaging | photography mode of one Example of this invention. The figure which shows the calculation method of the reliability showing the side face of one Example of this invention.

A feature of the present invention is to determine whether the amount of change in the size of a subject image when it is determined that a predetermined subject (such as a person's face) is stable during autofocusing is larger or smaller than a predetermined amount. However, if it is small, focus adjustment is performed by limiting the tracking drive operation in accordance with the distance change in the optical axis direction of the subject, and if it is large, the tracking drive operation in accordance with the distance change in the optical axis direction of the subject is performed. The focus adjustment is performed as drive control of the focus lens.

Examples of the present invention will be described below. FIG. 1 shows a configuration of a video camera (image pickup apparatus) including an automatic focus adjustment apparatus according to an embodiment of the present invention. In the following embodiments, a video camera will be described, but the present invention can also be applied to other imaging devices such as a digital still camera.

In FIG. 1, 101 is a first fixed lens, 102 is a variable magnification lens that moves in the optical axis direction and performs variable magnification, and 103 is a diaphragm. Reference numeral 104 denotes a second fixed lens, and reference numeral 105 denotes a focus compensator lens (also referred to as a focus lens in this specification) that has both a function of correcting the movement of the focal plane due to zooming and a focusing function. The first fixed lens 101, the variable power lens 102, the stop 103, the second fixed lens 104, and the focus lens 105 constitute an imaging optical system for imaging light from the subject. Reference numeral 106 denotes an image sensor as a photoelectric conversion element constituted by a CCD sensor or a CMOS sensor. The image sensor 106 photoelectrically converts the imaged light into an electrical signal. Reference numeral 107 denotes a CDS / AGC circuit that samples the output of the image sensor 106 and adjusts the gain. A camera signal processing circuit 108 performs various image processing on the output signal from the CDS / AGC circuit 107 to generate a video signal. Reference numeral 109 denotes a monitor constituted by an LCD or the like, which displays a video signal from the camera signal processing circuit 108. A recording unit 115 records the video signal from the camera signal processing circuit 108 on a recording medium such as a magnetic tape, an optical disk, or a semiconductor memory.

Reference numeral 110 denotes a zoom drive source for moving the variable magnification lens 102. Reference numeral 111 denotes a focusing drive source for moving the focus lens 105. The zoom drive source 110 and the focusing drive source 111 are configured by actuators such as a stepping motor, a DC motor, a vibration type motor, and a voice coil motor, respectively. Reference numeral 112 denotes an AF gate that passes only signals in a region (focus detection region) used for focus detection among output signals of all pixels from the CDS / AGC circuit 107. The AF signal processing circuit 113 extracts a high frequency component from the signal that has passed through the AF gate 112 and generates an AF evaluation value. That is, the AF signal processing circuit 113 constitutes a generation unit that generates an AF evaluation value from a focus detection area set for an electric video signal. The AF evaluation value is output to the camera / AF microcomputer 114 which is a control means. The AF evaluation value represents the sharpness (contrast state) of an image generated based on the output signal from the image sensor 106, and the sharpness changes depending on the focus state (degree of focusing) of the imaging optical system. As a result, a signal representing the focus state of the imaging optical system is obtained. The camera / AF microcomputer 114 as a control unit controls the operation of the entire video camera, and controls the focusing drive source 111 to move the focus lens 105 and adjust the focus based on the AF evaluation value. Also do.

The face detection unit 116 according to the present embodiment performs a known face detection process on the image signal to detect a human face area in the shooting screen. That is, the face detection unit 116 constitutes detection means for detecting a predetermined subject (here, a face) from the electrical signal. The detection result is transmitted to the camera / AF microcomputer 114. Based on the detection result, the camera / AF microcomputer 114 transmits information to the AF gate 112 so as to set the focus detection area at a position including the face area in the shooting screen. As face detection processing, a skin color region is extracted from the gradation color of each pixel represented by image data, and a face is detected based on a matching degree with a face contour plate prepared in advance, or a known pattern recognition technique is used. There is a method of performing face detection by extracting facial feature points such as eyes, nose and mouth.

Further, the face detection unit 116 calculates the reliability of the face and the reliability of the profile. The face reliability is calculated based on, for example, the matching degree with the face contour plate, and is expressed in five stages in descending order of matching degree. In addition, as shown in FIG. 13, the calculation of the reliability indicating the likelihood of a side face is performed by dividing the area so that the face detection frame is divided into left and right with reference to the center position of both eyes of the detected face (dotted line in the figure). Extract. Then, the skin color region included in the left and right regions is extracted, and the area is calculated by counting the number of pixels in the skin color region. The reliability representing the likelihood of profile is calculated using the ratio of the skin color area included in the left and right screens. If the left and right skin color areas have the same ratio, the probability of profile is assumed to be low, and the reliability indicating the likelihood of profile is set low, and if the ratio is significantly different, the probability of profile is assumed to be high Set high. In this embodiment, the reliability is set to 5 levels according to this ratio.

Reference numeral 117 denotes an aperture driving source, which includes an actuator for driving the aperture 103 and its driver. From the signal read by the CDS / AGC circuit 107, in order to obtain the luminance value of the photometric frame in the screen, the photometric value is obtained by the luminance information detection / calculation circuit 118, and the photometric photometric value is normalized by calculation. Is done. Then, the camera / AF microcomputer 114 calculates the difference between the photometric value and the target value set so that proper exposure can be obtained. Thereafter, the correction driving amount of the diaphragm is calculated from the calculated difference, and the driving of the diaphragm driving source 117 is controlled by the camera / AF microcomputer 114.

Next, AF control performed by the camera / AF microcomputer 114 will be described with reference to FIG. This AF control is executed according to a computer program stored in the camera / AF microcomputer 114. Step 201 indicates the start of processing. In step 202, a minute driving operation is performed, and it is determined whether the in-focus position is in focus or not. Detailed operation will be described later. In Step 203, if it is determined that the focus is in focus in Step 202, the process goes to Step 209 to perform the focusing process, and if it is not determined to be in focus in Step 202, the process goes to Step 204. In Step 204, if the direction can be determined in Step 202, go to Step 205 to perform the hill-climbing driving operation, and if the direction cannot be determined in Step 202, return to Step 202 and continue the minute driving operation. In Step 205, hill-climbing driving is performed to drive the lens at high speed in the direction in which the AF evaluation value increases. Detailed operation will be described later.

In Step 206, if it is determined in Step 205 that the peak of the AF evaluation value has been exceeded, the process goes to Step 207. If it is not determined in Step 206 that the peak of the AF evaluation value has been exceeded, the process returns to Step 205 and hill-climbing driving is continued. In Step 207, the lens is returned to the lens position where the AF evaluation value during hill-climbing driving reaches a peak. In Step 208, if the peak lens position is returned in Step 207, the process returns to Step 202 and the minute driving operation is performed again. If not returned to the peak lens position in Step 207, the process returns to Step 207 and the operation of returning to the peak lens position is continued. .

Next, the focusing operation from Step 209 will be described. In Step 209, the AF evaluation value is held. In Step 210, the latest AF evaluation value is acquired. In Step 211, the AF evaluation value held in Step 209 is compared with the AF evaluation value newly acquired in Step 210. If there is a difference of a predetermined level or more, it is determined that the operation is restarted, and the process proceeds to Step 202 to restart the minute driving operation. If it is not determined to restart in Step 211, go to Step 212. In Step 212, the lens is stopped, and the process returns to Step 210 to continue the restart determination.

Next, the minute driving operation will be described with reference to FIG. Step 301 indicates the start of processing. In Step 302, the latest face detection position / size is acquired, an AF frame that is a focus detection area is set based on the information, and an AF evaluation value is acquired. In Step 303, a process for determining movement of the subject from the acquired face size is performed. This process will be described with reference to FIG. In Step 304, it is determined whether or not the subject has moved nearer or infinitely. If the subject has moved, the process proceeds to Step 305. Otherwise, the process proceeds to Step 306. In this embodiment, the size of the subject image whose amount of change is monitored is the size of a person's face (which may be another part of the body) obtained by the face detection unit 116 serving as detection means. Therefore, as will be described later, when the amount of change in face size is within a predetermined amount, it is determined that there is no subject distance change, and when the amount of change is greater than the predetermined amount, it is determined that there is a subject distance change. Is done. When the camera / AF microcomputer 114 determines that there is no change in the distance of the subject, the camera / AF microcomputer 114 performs the focus adjustment by prohibiting the tracking control by the tracking control means. The focus control is performed by allowing the tracking control by the tracking control means. Such switching is performed in Step 304 above.

In Step 305, it is determined whether the result of the movement determination is approaching or moving away. When approaching, it progresses to Step 314, the mountain climbing to a near direction is determined, it progresses to Step 313, and a process is complete | finished. Otherwise, the process proceeds to Step 315 to decide mountain climbing in the infinite direction, and the process proceeds to Step 313 to end the process. The driving for switching the hill-climbing direction in accordance with the moving direction of the subject is the subject tracking drive or tracking control. By performing the tracking drive, AF responsiveness and accuracy are improved. In order to execute this function, the camera / AF microcomputer 114 of the control unit includes a determination unit that determines whether or not there is a change in the distance of the subject based on a detection result by the face detection unit 116 that is a detection unit, and a focus according to the determination result. Tracking control means for causing the lens to follow a change in the distance of the subject is included. In this way, the determining means determines that the subject is approaching when the amount of change in the size of the subject image is larger than a predetermined amount, and the control means drives the focus lens to the closest side to move the subject. Follow. On the other hand, the discriminating means discriminates that the subject is far away when the amount of change in the size of the subject image is larger than a predetermined amount, and the control means drives the focus lens to the infinite side to follow the subject. . In this embodiment, when the change amount of the size of the subject image detected by the detection unit is larger than a predetermined amount, the control unit performs a hill-climbing drive operation of the focus lens in accordance with the change in the distance of the subject in the optical axis direction. . When the amount of change in the size of the subject image detected by the detecting means is smaller than a predetermined amount, the focus lens is finely driven.

Subsequently, in Step 306, if the AF evaluation value captured in Step 302 is larger than the previous AF evaluation value, the process proceeds to Step 307. If the AF evaluation value captured in Step 302 is smaller than the previous AF evaluation value, the process proceeds to Step 308. In Step 307, the focus lens is driven by a predetermined amount in the previous forward direction. On the other hand, in Step 308, the focus lens is driven by a predetermined amount in the reverse direction of the previous time. FIG. 4 shows the time course of the lens operation. Here, the AF evaluation value A for the electric charge accumulated in the CCD (imaging device) during A is taken in by TA, and the AF evaluation value B for the electric charge accumulated in the CCD during B is taken in by TB. In TB, the AF evaluation values A and B are compared, and if A <B, they move in the forward direction, while if A> B, they are in the reverse direction. In Step 309, if the direction determined to be the in-focus direction is continuously the same number of times, the process proceeds to Step 310. If the direction is not continuously advanced the same number of times, the process proceeds to Step 311. In Step 311, if the focus lens repeats reciprocation within a predetermined range a predetermined number of times, the process proceeds to Step 312, and if the focus lens is not within the predetermined range for a predetermined time, the process proceeds to Step 313 and the process is terminated. As a result, the operation returns to the minute driving operation in Step 202 of FIG. In Step 310, assuming that the direction can be determined, the process proceeds to Step 313, where the processing is terminated and the process proceeds to hill climbing driving in Step 205 of FIG. In Step 312, the process is terminated because the in-focus determination has been made, and the process proceeds to a restart determination routine after Step 210 in FIG. 2.

Next, the hill-climbing driving operation will be described with reference to FIG. Step 501 indicates the start of processing. In Step 502, the latest face detection position / size is acquired, an AF frame (focus detection area) is set based on the information, and an AF evaluation value is acquired. In Step 503, if the AF evaluation value captured in Step 502 is larger than the previous AF evaluation value, the process proceeds to Step 504. If the AF evaluation value captured in Step 502 is smaller than the previous AF evaluation value, the process proceeds to Step 505. In Step 504, the focus lens is driven at a predetermined speed in the previous forward direction, and the process proceeds to Step 508 to end the process. On the other hand, in Step 505, if the AF evaluation value has decreased beyond the peak, the process proceeds to Step 506 and it is determined that the peak has been exceeded. The process proceeds to Step 508, where the processing is terminated, and the process proceeds to Step 202 of Step 202 through Step 206 in FIG. If the AF evaluation value does not decrease beyond the peak in Step 505, the process proceeds to Step 507, the focus lens is driven at a predetermined speed in the direction opposite to the previous time, and the process proceeds to Step 508.

FIG. 6 illustrates the lens operation. Here, in the case of A, since the AF evaluation value has decreased beyond the peak, the hill-climbing driving operation is terminated assuming that the in-focus point exists, and the operation shifts to the minute driving operation. On the other hand, in the case of B, since the AF evaluation value decreases without a peak, it is reversed as having the wrong direction, and the hill-climbing driving operation is continued. The movement amount per fixed time, that is, the driving speed is larger than the driving speed of the minute driving. When the size of face detection increases continuously, the AF responsiveness can be improved by setting the driving speed at the time of hill-climbing driving large. As described above, the camera / AF microcomputer 114 performs control so as to increase the AF evaluation value by moving the focus lens while repeating the restart determination → micro drive → mountain climbing drive → micro drive → restart determination. Yes.

Next, a process for determining whether the subject is near or infinite will be described with reference to FIGS. First, the movement determination process in Step 303 of FIG. 3 will be described with reference to FIG. Step 701 indicates the start of the movement determination process. In Step 702, processing for calculating the average of detected face sizes is performed. The detected face size is stored in a memory in the camera / AF microcomputer, and an average of 10 stored images is calculated using a moving average method. The averaged face size is stored in FaceAveSize. The FaceAveSize has 20 arrays, and a history is stored in FaceAveSize [0] to [19] each time an average value is calculated. Here, the latest average value is stored in FaceAveSize [0].

In Step 703, it is determined whether or not there is a movement determination flag. When the movement determination flag determined in the process described later is set, the process proceeds to Step 704. Otherwise, the process proceeds to Step 707. In Step 707, it is determined whether or not the average face size FaceAveSize calculated in Step 702 is a stable value. Details of this processing will be described later with reference to FIG. In Step 708, the stability flag determined in Step 707 is determined. If the stability flag is SET, the process proceeds to Step 709. If not, the process proceeds to Step 720, and the process ends.

Here, when the stability flag is CLEAR, it means that the movement cannot be determined because the state of the subject is not stable. In this case, the process proceeds to Step 306 and subsequent steps through Step 304 in FIG. In Step 709, it is determined whether the face reference size is set. The face reference size FaceBaseSize is the size of the face that serves as a reference when performing movement determination. When the face reference size FaceBaseSize is 0, no value is set, so the process proceeds to Step 719, and in Step 719, FaceAveSize [0] is substituted into FaceBaseSize. Also in this case, the process proceeds to Step 306 and subsequent steps through Step 304 in FIG.

If the face reference size FaceBaseSize is not 0, the process proceeds to Step 710. In Step 710, a movement determination threshold value setting process is performed. This process is a process for setting a face size change threshold value used for movement determination according to the shooting parameters and the state of the subject. Details of this processing will be described later with reference to FIG. Subsequently, in Step 711, the face reference size FaseBaceSize is compared with the current face size FaceAveSize [0]. If the current face size is small, there is a possibility of moving away, so the process proceeds to Step 712. Otherwise, there is a possibility of approaching, so the process proceeds to Step 713. In Step 712, the distance determination is performed. If the difference between the face reference size FaseBaceSize and the current face size FaceAveSize [0] is greater than or equal to THfar, it is determined that the subject has moved away, and the process proceeds to Step 717. If THfar is not exceeded in Step 712, the process proceeds to Step 714. In Step 717, after setting the distance flag of the movement determination flag which means that the subject has moved away, the process proceeds to Step 720 and ends. In this case, the process proceeds to Step 305 and subsequent steps through Step 304 in FIG. In Step 713, approach determination is performed. If the difference between the face reference size FaseBaceSize and the current face size FaceAveSize [0] is greater than or equal to THnear, it is determined that the subject has approached, and the process proceeds to Step 718. If THnear is not exceeded in Step 713, the process proceeds to Step 714. In Step 718, after setting the approach flag of the movement determination flag that means that the subject has approached, the process proceeds to Step 720 and the process is terminated. Also in this case, the process proceeds to Step 305 and subsequent steps through Step 304 in FIG. As described above, the determination unit included in the camera / AF microcomputer 114 determines whether or not there is a change in the distance of the subject by monitoring the amount of change in the size of the subject image when it is determined to be stable.

In Step 714, when the face reference size is set and the state is stable, the timer FaceJudgeTimer indicating how many times the movement determination is repeated is incremented, and the process proceeds to Step 715. In Step 715, it is determined whether FaceJudgeTimer is equal to or less than TimerTH. This process is intended to initialize the determination process when there is no movement of the subject even though the movement determination has continued for about 2 seconds. Therefore, TimerTH is set to 120, which is a value corresponding to 2 seconds, and it is determined whether this value has been exceeded. In this embodiment, since a system that performs processing 60 times per second is assumed, the value corresponding to 2 seconds is 120. If FaceJudgeTimer exceeds TimerTH, the process proceeds to Step 716 to perform initialization processing. If not, the process proceeds to Step 720 and the process ends. In this case, the process returns to Step 702 to continue the movement determination. In Step 716, FaceJudgeTimer is initialized to 0, and the process proceeds to Step 705. In Step 705, FaseBaceSize is initialized to 0, and the process proceeds to Step 706. In Step 706, the movement determination flag is cleared and initialization is performed. After the initialization process is completed, the process proceeds to Step 720 and the process ends. In this case, the process proceeds to Step 306 and subsequent steps through Step 304 in FIG.

If the movement determination flag is set in Step 703, the process proceeds to Step 704 in order to determine whether or not the movement of the subject has ended. In Step 704, FaseAveSize is compared to determine the end of movement. In order to obtain the difference between the current FaseAveSize [0] and the previous history FaseAveSize [1], FaseAveSizeDiff = FaseAveSize [0] −FaseAveSize [1] is calculated, and the process proceeds to Step 704 ′. In Step 704 ′, when the subject is approaching, it is determined that the movement is finished when the face size is small (FaseAveSizeDiff is a negative value). When the subject is far away, the face size increases (
If FaseAveSizeDiff is a positive value), it is determined that the movement has ended. When the movement of the subject is completed, the process moves to Step 705 to perform initialization processing. When that is not right, it changes to Step 720 and complete | finishes a process. In this case, the process returns to Step 702 to continue the movement determination.

Next, the subject stability determination process in Step 707 of FIG. 7 which is a feature of the present invention will be described with reference to FIG. Step 801 in FIG. 8 indicates the start of processing. Next, in Step 802, the camera / AF microcomputer 114 calculates a secondary differential value of the average face size. The specific calculation method of the secondary differential value is shown below. First, the camera / AF microcomputer 114 calculates a first differential value by calculating a difference between the average face size calculated in Step 702 of FIG. 7 and the average face size calculated a predetermined time ago, and the camera / AF microcomputer It is recorded in the memory in 114. Next, the camera / AF microcomputer 114 calculates a change amount between the primary differential value recorded in the memory and the primary differential value calculated in a predetermined time and recorded in the memory, thereby obtaining a secondary. Calculate the differential value. The calculated secondary differential value is a value indicating how much the primary differential value of the average face size calculated this time has changed with respect to the primary differential value of the average face size before a predetermined time. In order to execute such a function, the subject stability determination means included in the camera / AF microcomputer 114 of the control means includes subject information storage means, first difference calculation means, and difference storage means. The subject information storage unit stores the size of the subject image detected by the detection unit (face detection unit 116). The first difference calculation means calculates a difference value between the latest size of the subject detected by the detection means and the size of the subject image previously stored in the subject information storage means with a time interval. The difference storage means stores the difference value calculation result by the first difference calculation means. Here, the steepness of the change in magnitude, which is the secondary differential value, is the amount of change between the latest difference value stored in the difference storage means and the difference value previously stored with a time interval. . As will be described later, the size of the subject image may be changed to the position of the subject image. In step 803, the camera / AF microcomputer 114 determines whether the secondary differential value calculated in step 802 is equal to or less than a threshold value. When the calculated secondary differential value is less than or equal to the threshold value, the subject is stable because the amount of change in the primary differential value of the average face size calculated this time is smaller than the primary differential value of the average face size before a predetermined time. It judges that it is carrying out, and progresses to Step 804. On the other hand, if the calculated secondary differential value is larger than the threshold value, the change amount of the primary differential value of the average face size calculated this time is larger than the primary differential value of the average face size before a predetermined time, so that the subject is It judges that it is not stable and proceeds to Step 807.

In this embodiment, the threshold is empirically set to a value of 10% of the average face size. The predetermined time before is defined as 15 frames before as a value that can be determined by the threshold whether or not the secondary differential value is stable. However, in the present embodiment, the set threshold value and the predetermined time are values that can be freely changed by the system. Next, in Step 804, the camera / AF microcomputer 114 counts up StableTimer and proceeds to Step 805. However, StableTimer is a variable that counts a period in which the calculated average face size is continuously stable. Therefore, it is possible to determine that the average face size after the stable timer exceeds a predetermined period (hereinafter referred to as stable TH) is a stable average face size. Here, StableTH is 15 frames in this embodiment. Further, StableTH is a variable that counts a period during which the average face size is continuously stable, and therefore, a period that is equal to or shorter than a predetermined time (15 frames in the present embodiment) used when calculating the secondary differential value. Need to be. In Step 807, since the secondary differential value of the average face size is larger than the threshold value, the camera / AF microcomputer 114 clears StableTimer (0 in this embodiment).

Next, in Step 805, the camera / AF microcomputer 114 determines whether or not StableTimer is less than StableTH. If it is less than StableTH, the process proceeds to Step 806. If it is greater than StableTH, the process proceeds to Step 808. In Step 806, the camera / AF microcomputer 114 determines that the average face size calculated in Step 702 is not continuously stable until the StableTimer exceeds StableTH, turns off the stability flag, and finishes the process of FIG. To do. On the other hand, in Step 808, the camera / AF microcomputer 114 determines that the average face size calculated in Step 702 is a value that is stable continuously for a period of time when the StableTimer is equal to or greater than StableTH, and sets the stability flag to ON. Terminate the process. As described above, the subject stabilization processing means included in the camera / AF microcomputer 114 has a predetermined degree of steepness (secondary differential value) of the change in the size of the subject image (first differential value) detected by the detection means. It is determined that the subject is stable when the time is continuously below the threshold.

In the processing of FIG. 8 described above, it is determined whether or not the subject is stable based on the average face size. However, whether or not the subject is stable by calculating a secondary differential value with respect to the position of the face or the human body. It may be a method of judging. Further, in the case of a system capable of detecting eyes and mouth, a method of calculating whether or not the second differential value is calculated with respect to the distance between both eyes (or eyes and mouth) and determining whether or not the system is stable may be used. Furthermore, a method may be used in which the ratio of the distance between the eyes and the mouth with respect to the distance between both eyes is calculated, and a secondary differential value is calculated with respect to the calculated ratio to determine whether or not it is stable. In short, if it is information that can determine the stability of the movement of a predetermined subject, the size of a predetermined part (organ etc.) placed on the subject, the distance between multiple parts or organs, the multiple sizes and distances Any shape information of the subject, such as relative information about the ratio between the two, may be used.

Next, the movement determination threshold value setting process in Step 710 of FIG. 7 will be described with reference to FIG. Step 901 in FIG. 9 indicates the start of processing. Step 902 is processing for acquiring the lens position of the variable magnification lens (zoom lens) 102. If the driving source of the variable power lens is a step motor, it is the number of steps, and is used to determine which position the lens is on the tele side or the wide side. Subsequently, in Step 903, the reference movement determination threshold Th corresponding to the current zoom position is acquired from the relationship graph between the zoom lens position and the reference movement determination threshold Th shown in FIG. In FIG. 10, the vertical axis represents the change rate (%) of the face size, and the horizontal axis represents the zoom lens position. The relationship between the zoom lens position and the reference movement determination threshold Th is a graph calculated from the relationship between the front end of the depth of field and the amount of change in face size at each zoom lens position with the aperture value Fbase being constant. is there.

If the subject moves in a state where the predetermined subject is in focus, blur will start to be recognized on the screen from the point where the depth of field is exceeded, so the reference movement determination threshold value is smaller than the depth of field. Set. Accordingly, it is possible to determine whether or not the subject has moved before the subject starts to blur and to drive the focus lens, thereby improving the follow-up performance of the subject. When the actual reference movement determination threshold Th is calculated, a table of the reference movement determination threshold Th corresponding to each zoom lens position is held in the camera / AF microcomputer 114, and the threshold Th corresponding to the zoom lens position is determined.

The depth of field is calculated using Equation 1 below. If the subject distance is s, and the front and rear ends of the depth of field are Dn and Df, respectively, the result is as follows.
Dn = s (Hf) / (H + s-2f) ... Equation 1a
Df = s (Hf) / (Hs) ... Equation 1b
The hyperfocal length H is expressed by Equation 2 where f is the focal length of the lens, N is the aperture value of the lens, and c is the diameter of the allowable circle of confusion.
H = f * f / N * c Equation 2

In step 904, the current aperture value Fno is acquired. The aperture value is obtained from the drive amount of the aperture 103. In Step 905, the reference movement determination threshold is corrected according to the aperture. Since the depth of field changes according to the aperture value, the acquired current aperture value Fno is compared with the aperture value Fbase when the reference movement determination threshold value in FIG. 10 is calculated. Based on Equation 3, a corrected movement determination threshold Th2 is calculated.
Th2 = (current aperture value Fno / reference aperture value Fbase) * reference movement determination threshold Th.
Thus, the threshold value or the predetermined amount is determined by the depth of field, and is set to an amount smaller than the amount of change in the size of the subject image when the subject moves the depth of field. It is a feature.

Subsequently, in Step 906, the reference face size FaceBaseSize is acquired. In Step 907, the reference movement determination threshold value is corrected according to the reference face size FaceBaseSize. FIG. 11 is a graph showing the relationship between the reference face size and the correction coefficient K. The vertical axis represents the change rate (%) of the face size, and the horizontal axis represents the reference face size. As the reference face size FaceBaseSize increases, the detected face size varies, and correction is required. Based on the acquired reference face size and correction coefficient K graph, a corrected movement determination threshold Th3 is calculated based on Equation 4.
Th3 = correction coefficient K * movement determination threshold Th2 Expression 4
Thus, the subject situation is the size of the subject image detected by the detection means, and the threshold value or the predetermined amount is changed according to the size of the subject image, and a reference value for the size of the subject image. It is changed according to the difference.

Subsequently, in Step 908, the face reliability is obtained from the signal of the face detection unit 116. As described above, the reliability of the face is evaluated in five stages of 1 to 5, evaluation 5 has the highest reliability, and evaluation 1 has the lowest reliability. In Step 909, the movement determination threshold value is corrected using the face reliability. When the reliability is low, the detected face size varies greatly, or even if it is small, there is a possibility of erroneous detection. Therefore, it is necessary to perform threshold correction and set the threshold high. In this embodiment, when the face reliability is 3 or less, the correction coefficient K ′ is set to 1.5, and the corrected movement determination threshold Th4 is calculated based on Expression 5 using the acquired face reliability. To do.
Th4 = correction coefficient K ′ * movement judgment threshold Th3 Expression 5
Thus, the subject situation is a reliability indicating the likelihood of the face detected by the detection means, and the threshold value or the predetermined amount is changed according to the reliability indicating the likelihood of the face, and the reliability If is low, it is changed to a larger amount.

Next, in Step 910, the reliability of profile likelihood is acquired from the signal of the face detection unit 116. As described above with reference to FIG. 13, the reliability of profile likelihood is evaluated in five stages of 1 to 5. Evaluation 5 has the highest possibility of profile and evaluation 1 has the lowest possibility of profile. In Step 911, the movement determination threshold value is corrected using the reliability of the profile likelihood. When the reliability of the side face is high, the detected face size varies greatly, and even if it is small, there is a possibility of erroneous detection. Therefore, it is necessary to correct the threshold value and set the threshold value high. In the present embodiment, when the reliability of the side face likelihood is 3 or more, the correction coefficient K ″ is set to 1.5, and the movement after correction is performed based on Expression 6 using the acquired reliability of the side face likelihood. A determination threshold Th5 is calculated.
Th5 = Correction coefficient K ″ * Movement determination threshold Th4 Equation 6
In this way, the subject situation is a reliability indicating whether the face detected by the detection means is facing sideways, and the threshold value or the predetermined amount corresponds to a reliability indicating whether the face is facing sideways. If the reliability is high, the amount is changed to a larger amount.

Subsequently, in Step 912, the photographing mode is acquired from the information in the camera / AF microcomputer 114. A general imaging apparatus has a plurality of shooting modes as shown in FIG. 12 in order to set shooting parameters optimal for a shooting scene. In Step 913, the movement determination threshold value is corrected according to the shooting mode. FIG. 12 is a table showing the relationship between the shooting mode and the correction coefficient K ′ ″. In a shooting mode in which the movement of the subject is assumed to be large, the coefficient K ′ ″ is set to a value of 1 or less so that the threshold value is set low, thereby increasing the autofocus response. Further, in the shooting mode in which the movement of the subject is assumed to be small, the value of the coefficient K ′ ″ is set high so as to increase the threshold for movement determination, and importance is attached to the stability of autofocus. Based on the acquired photographing mode and the correction coefficient K ′ ″ determined from FIG. 12, a corrected movement determination threshold Th6 is calculated based on Expression 7.
Th6 = correction coefficient K ″ ′ * movement determination threshold Th5 Expression 7
Thus, the shooting parameter is a shooting mode, and the threshold value or the predetermined amount is changed according to the shooting mode, and a large amount is set when the shooting mode is set to a mode for shooting a subject with much movement. When the shooting mode is set to a mode for shooting a subject with little movement, the amount is changed to a small amount.

Subsequently, in Step 914, the degree of focus is acquired. The method for calculating the degree of focus is a value obtained by dividing the peak hold value of the AF evaluation value in the evaluation frame by the difference between the maximum value and the minimum value of the luminance level of each line. This degree of focus is represented by 0 to 1, and in the case of a focused subject, the difference between the peak hold of the AF evaluation value and the luminance level tends to be the same value, so the degree of focus approaches 1. An object with a low degree of focus is likely to be blurred and the reliability of the face size may be low, so the threshold value for movement determination is set to be high. For example, using the obtained degree of focus, when the degree of focus is 0.5 or less, the correction coefficient K ″ ″ is set to 1.5, and the movement determination threshold value after correction based on Expression 8 Th7 is calculated.
Th7 = Correction coefficient K '''' * Movement determination threshold Th6 ... Equation 8
In this way, the imaging parameter is the degree of focus estimated from the state of the AF evaluation value level and the driving state of the focus lens, and the threshold value or the predetermined amount is changed according to the degree of focus. If the degree is low, it is changed to a larger amount. These correction factors K ', K'',K''', K '''' are determined after sufficient measurement according to the camera, and are not limited to these values. Absent.

Subsequently, in Step 916, a limit is set to the movement determination threshold Th7 calculated in Step 915. In the case of the correction method described above, the movement determination threshold value is several times the reference movement determination threshold value due to accumulation of corrections, and may not be appropriate as the movement determination threshold value. Therefore, the upper limit of the threshold is set. If the movement determination threshold Th7 is greater than or equal to twice the reference movement determination threshold Th, the process proceeds to Step 918; otherwise, the process proceeds to Step 917. In this embodiment, the limit is set to double in order to maintain the accuracy of the movement determination, but this value can be arbitrarily set to a value that can perform sufficient measurement and reduce erroneous determination. In Step 917, Th7 is substituted for the movement determination threshold THface, and the process proceeds to Step 919. In Step 918, since the movement determination threshold is too large, a limit is set for the threshold. A value obtained by doubling the reference movement determination threshold Th is substituted for the final movement determination threshold THface, and the process proceeds to Step 919.

In Step 919, the threshold for distance determination and approach determination is changed based on the movement determination threshold THface. For face detection, there is a method of determining the face size based on the interval between eyes. When a person turns sideways, the distance between the eyes is reduced, so that the detected face size is reduced even though the distance to the subject has not changed. For this reason, the distance of the subject may be detected. Therefore, by setting the distance detection threshold THfar larger than the approach detection threshold THnear, erroneous determination due to a size change during profile shooting is reduced. In the present embodiment, the distance detection threshold is increased by a factor of 1.5, but this value is set to a value that performs sufficient measurement and reduces erroneous determination. In this way, the subject situation is the result of the determination, and the threshold value or the predetermined amount is set to a different value when the determination result is far away and closer, and when the distance is far away, it approaches. A larger amount is set than when.

The processing flow for setting the movement determination threshold has been described above, and is the processing for calculating the approach detection threshold THnear and the away detection threshold THfar. The coefficients and formulas used in this processing are examples, and the present invention is not limited to these.

As described above, in the above-described embodiment, focus detection is performed on the subject person using the AF evaluation value using face detection. At the same time, the determination means monitors the change in the size of the face with reference to the face size in a state where the subject is stable, thereby making it possible to determine the movement of the subject with high accuracy and stability. If it is determined using the information that there is a predetermined change in the distance of the subject, the focus accuracy can be improved by driving the focus to follow the infinite or close-up movement of the subject. .

For the processing in the above-described embodiments, a storage medium storing software program codes embodying each function may be provided to the system or apparatus. The functions of the above-described embodiments can be realized by the computer (or CPU or MPU) of the system or apparatus reading and executing the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-described embodiments, and the storage medium storing the program code constitutes the present invention. As a storage medium for supplying such a program code, for example, a floppy (registered trademark) disk, a hard disk, an optical disk, a magneto-optical disk, or the like can be used. Alternatively, a CD-ROM, CD-R, magnetic tape, nonvolatile memory card, ROM, or the like can be used. The functions of the above-described embodiments are not only realized by executing the program code read by the computer. This includes the case where the OS (operating system) running on the computer performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. ing. Furthermore, the program code read from the storage medium may be written in a memory provided in a function expansion board inserted into the computer or a function expansion unit connected to the computer. Thereafter, the CPU of the function expansion board or function expansion unit performs part or all of the actual processing based on the instruction of the program code, and the functions of the above-described embodiments are realized by the processing. It is a waste.

As mentioned above, although preferable embodiment thru | or Example of this invention was demonstrated, this invention is not limited to the said embodiment thru | or Example, A various deformation | transformation and change are possible within the range of the summary.

105... Focus lens, 106... CCD (image sensor), 113... AF signal processing circuit (generation means), 114 .. Camera / AF microcomputer (control means, discrimination means, subject stability judgment means), 116. Face detector (detection means)

Claims (6)

An image sensor that photoelectrically converts light from a subject that has passed through an imaging optical system including a focus lens into an electrical signal;
Generating means for generating an AF evaluation value from an electrical signal corresponding to a set focus detection region among electrical signals output from the image sensor;
Detecting means for detecting a predetermined subject image from an electrical signal output from the image sensor;
Discriminating means for discriminating whether or not there is a change in distance in the optical axis direction of the subject detected by the detecting means;
Subject stability determination means for determining whether the detection result by the detection means is a result of the subject in a stable state;
Control means for performing focus adjustment by driving and controlling the focus lens based on the AF evaluation value, in accordance with a change in distance in the optical axis direction of the subject detected by the detection means in accordance with a determination result by the determination means In addition, when the focus lens is driven and controlled, the amount of change in the size of the subject image detected by the detection unit when the subject stability determination unit determines that the subject is stable is greater than a predetermined amount. If larger, the tracking drive operation corresponding to the distance change in the optical axis direction of the subject is performed as the drive control of the focus lens, and the subject is stabilized by the subject stability determination unit detected by the detection unit. If the amount of change in the size of the subject image when determined to be smaller than the predetermined amount, the detection means detects the change in the distance in the optical axis direction of the subject. And control means for limiting said tracking drive operation was,
An autofocus device having
The subject stability determination means includes
Subject information storage means for storing the size or position of the subject image detected by the detection means;
A difference value between the latest size or position of the subject image detected by the detection unit and the size or position of the subject image previously stored in the subject information storage unit at a time interval is calculated. 1 difference calculating means;
Difference storage means for storing the calculation result of the difference value by the first difference calculation means,
The subject stability determination means includes
A change in the size or position of the subject image detected by the detection means, which is the amount of change between the latest difference value stored in the difference storage means and the difference value previously stored with a time interval. An automatic focus adjustment apparatus characterized by determining that the subject is stable when the degree of steepness is less than a threshold value for a predetermined time continuously . The discrimination means includes
And wherein the object stability judgment the object image is determined to be stable by means calculates a difference magnitude which is detected at time intervals, to determine the presence or absence of change in distance of the object The automatic focusing apparatus according to claim 1 . 3. The automatic focus adjustment apparatus according to claim 1, wherein the size of the subject image is a size of a human face or body part obtained by the detection means. The size of the object image in the object stability determining means, according to claim 1, characterized in that at least two of the distance between the organ of the organs which are located on the face of the person obtained by the detecting means or 2. The automatic focusing apparatus according to 2. Wherein the position of the subject image, the automatic focusing device according to any one of claims 1 to 4, characterized in that the position of portions of a face or body of a person obtained by the detecting means. Photoelectrically converting light from a subject that has passed through an imaging optical system including a focus lens into an electrical signal;
Generating an AF evaluation value from an electrical signal corresponding to a set focus detection area among electrical signals output in the conversion step;
A detection step of detecting a predetermined subject image from the electrical signal output in the conversion step;
A discriminating step for discriminating whether or not there is a change in distance in the optical axis direction of the subject detected in the detecting step;
A control step for controlling the focus by driving the focus lens based on the AF evaluation value; and
A subject stability determination step for determining whether or not a detection result in the detection step is a result in a stable state of the subject;
An automatic focusing method including:
The subject stability determining step includes:
A subject information storage step for storing the size or position of the subject image detected in the detection step;
A difference value between the latest size or position of the subject image detected in the detection step and the size or position of the subject image previously stored in the subject information storage step at a time interval is calculated. 1 difference calculating step;
A difference storage step for storing a difference value calculation result obtained by the calculation of the first difference calculation step;
In the subject stability determination step,
A change in the size or position of the subject image detected in the detection step, which is the amount of change between the latest difference value stored in the difference storage step and the difference value previously stored with a time interval. When the steepness level of the subject is less than the threshold value for a predetermined time continuously, the subject is determined to be stable,
In the control step, when the focus lens is driven and controlled in accordance with the distance change in the optical axis direction of the subject detected in the detection step according to the determination result in the determination step, the detection is detected in the detection step, If the amount of change in the size of the subject image when the subject is determined to be stable in the subject stability determination step is greater than a predetermined amount, a follow-up driving operation corresponding to the change in the distance of the subject in the optical axis direction The focus lens drive control is performed, and the amount of change in the size of the subject image that is detected in the detection step and determined that the subject is stable in the subject stability determination step is smaller than the predetermined amount. In this case, the following driving operation is limited in accordance with the change in the distance of the subject in the optical axis direction detected in the detection step. Automatic focusing how to.

Images