JPH0391377A - Automatic focus adjustment device - Google Patents

Abstract

PURPOSE:To reduce power consumption and to improve the durability of the device by detecting an external disturbance of a high frequency component of a video and assigning a means to each of state of the external disturbance to designate the operating mode of the automatic focus mode thereby reducing the number of times of system operation. CONSTITUTION:An object image made incident from an image pickup lens 1 and formed onto a CCD 2 is supplied to a camera system signal processing circuit 3 as a luminance signal and inputted to an automatic focus control system 4 as a video signal. The high frequency component of the luminance signal of the video signal is extracted by a band pass filter section 41. The high frequency component passing through a low pass filter whose cut-off frequency is 200kHz-1.5MHz for the purpose of synchronization and passing through a high pass filter whose cut-off frequency is 400kHz-1.5MHz for the purpose of high focus accuracy is selected properly by a switch, and the component is rectified by a half wave rectifier and digitized by an A/D converter 42 in 3.58MHz.6-bit. The high frequency component in a focus detection area equivalent to 1/4 of the entire screen area is integrated from the high frequency component signal by an integration device 43 comprising a gate array and the result is fed to a microcomputer 44.

Description

DETAILED DESCRIPTION OF THE INVENTION [Object of the Invention] (Industrial Application Field) The present invention relates to an automatic focusing device for automatically focusing a video camera or the like.

(Conventional technology) This year marks exactly 15 years since the appearance of consumer video cameras, and the recent COD (Charge Coupled D)
The market for video cameras/camera integrated VTRs is rapidly growing, supported by the stable supply of solid-state image sensing devices such as VCRs.

The current popularity of video cameras is due to the ease of operation and compactness of the video camera/camera integrated VTR format, as well as the fact that they are equipped with automatic functions such as autofocus, auto white balance, and auto iris, and the ability to perform these functions. Advances in technology have made it easier for anyone to take pictures without mistakes.

There are various types of autofocus methods, but they can be broadly classified into active methods and passive methods, depending on whether or not the system itself generates a reference signal for distance measurement. For example, active methods include methods such as Koramori, which emit ultrasonic waves and capture the reflected sound to measure the distance to an object, and methods that emit infrared light in the form of a spot and detect the position of the reflected light. On the other hand, a passive method determines the distance only from the video signal without emitting any light or sound.

Incidentally, in recent years, a method (passive method) that takes advantage of the fact that solid-state image sensors have no image distortion and uses a mountain-climbing servo to extract focus information from a video signal has begun to attract attention. The basic principle of this "mountain-climbing servo autofocus" was proposed by NHK's Ishida et al. in 1966.

(Jun Ishida-1 Yasushi Fujimura “Automatic focus adjustment of TV cameras using mountain climbing servo method J” NHK Giken Research, Vol.
.

17, No. 1, 1965 PP21-37> Hill-climbing servo autofocus is a so-called passive method in which sharpness (contrast) measurement is submerged in M for distance measurement, and is a type that measures distance through an imaging lens. The principle is to automatically adjust the focus by controlling the optical focus adjustment mechanism so that the sharpness of the video signal, that is, the high frequency component, is maximized.In this method, the control is procedural. This method was finally realized with the advent of components such as gate arrays and microcomputers and recent advances in software technology.The feature of this method is that the distance measurement sensor is an image sensor. Since it can be used in common with other devices, it has good focus accuracy, and since it can be processed purely electronically and by software, it is very advantageous in terms of miniaturization.

There are two ways of thinking about mountain climbing methods.

One is a trial method and the other is a perturbation method. The trial method is
The feature is that the lens is moved once to extract distance information that has been degenerated into a plane. Just like a person flips a coin when trying to focus, he first turns the focus ring in one direction, and if it's in focus, he turns it in the same direction, and if it's out of focus, he turns it in the opposite direction and stops at the point where the contrast is the strongest. It is the same way of performing an action.

On the other hand, in the perturbation method, the optical path length from the lens to the image sensor is always slightly fluctuated (perturbed) by some method, and the high frequency components contained in the video signal are modulated at the frequency of the perturbation. This method determines whether the front bin is the rear bin based on the amplitude and phase of the component.However, the perturbation method requires a separate mechanical means to modulate the optical path length, which has the disadvantage of increasing the number of parts. In both the trial method and the perturbation method, the operation object of autofocus is basically to focus on a stationary subject image.

The ascending servo type autofocus determines whether the subject is in focus (in focus) or out of focus based on a trial operation. Therefore, if the series of distance measurement operations is performed in an extremely short time compared to the movement of the subject, there will be little problem with the movement of the subject, but the time interval for sampling focus information for distance measurement is When the distance is about one field of a television, aliasing occurs due to sampling with the movement of the subject, and the causality of the mountain-climbing servo operation is disrupted, resulting in malfunction (misfocusing).

Further, if the expected change in signal amount due to a series of ranging operations is sufficiently larger than the change due to the movement of the subject, problems with the movement of the subject are unlikely to occur.

However, if the expected change in the signal amount is smaller than the change due to the movement of the subject, the causality of the ascending servo operation is broken and a malfunction (misfocus) occurs.

For this reason, it is necessary to move the focus ring with a large amplitude so that the expected change in signal amount due to the series of distance measurement operations is sufficiently large compared to the change due to the movement of the subject. A phenomenon occurred in which the images became blurry, making the images extremely difficult to view.

As an example, we have explained the mountain-climbing autofocus, which is most noticeably affected by external disturbances, but the infrared autofocus,
This phenomenon occurs regardless of the method, such as the CL method, the ultrasonic method, or the phase difference method, and the operation of making the image difficult to see is the same.

(Question B to be solved by the invention) When using mountain-climbing servo autofocus for video shooting, if disturbances such as camera shake, subject movement, zooming, and panning occur that disrupt the causality of the mountain-climbing motion, , malfunction (misfocus).

Therefore, when such a disturbance occurs, in order to retrieve the distance information that has been degenerated into a plane by moving the lens once, it is necessary to
It was necessary to ensure causality in the separation of lens movement and distance information. For example, if the expected change in signal amount due to a series of distance measurement operations is small compared to the change due to the movement of the subject, the causality of the mountain climbing servo operation will be broken and malfunction (misfocusing) will occur. The focus ring must be moved with a large amplitude so that the change is sufficiently large compared to the change due to movement, which causes a phenomenon in which the focus becomes significantly blurred during use, resulting in an image that is very difficult to see.

Therefore, the present invention aims to eliminate the above-mentioned drawbacks, such as camera shake,
It is an object of the present invention to provide an automatic focus adjustment device that can perform smooth and reliable focusing by maintaining the causality of a mountain climbing servo operation against changes in photographing conditions due to the movement of a subject or the like.

[Structure of the invention]

(Means for Solving the Problems) The present invention obtains a video signal of an optical image formed on an image sensor by an imaging lens, and automatically adjusts the focus of the imaging lens based on the high frequency component of this video signal. In the automatic focus adjustment device for adjustment, a slow fluctuation extraction means extracts a change in the scene or subject imaged by the imaging lens from the high frequency component, and a slow fluctuation extraction means for extracting changes in the scene or subject imaged by the imaging lens from the high frequency component, and a fast fluctuation extracting means for extracting a fluctuation component; a state estimating means for estimating a state related to focus adjustment of the imaging lens based on the extraction results of both of these extracting means; and the imaging lens estimated by the state estimating means. The camera has a configuration including a mode determining means that determines the operating speed and whether or not the mechanism for adjusting the focus of the imaging lens can operate in accordance with the state of the imaging lens.

(Function) In the automatic focus adjustment device of the present invention, the slow fluctuation extracting means extracts a change in the scene or subject imaged by the imaging lens from the high frequency component.The fast fluctuation extraction means extracts a change in the scene or subject imaged by the imaging lens from the high frequency component. The state estimating means estimates the state related to the focus adjustment of the imaging lens based on the extraction results of both the extracting means. The operating speed and whether the mechanism for adjusting the focus of the imaging lens can be operated are determined in accordance with the state of the imaging lens estimated by.

(Example) Hereinafter, an example of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing an embodiment of an automatic focus adjustment device of the present invention. 1 is an imaging lens capable of zooming that forms an optical image of the subject on CCD 2, 2 is a COD that photoelectrically converts the optical image and converts it into an image signal, and 3 is a device that processes the image signal input from COD 2 and adjusts the brightness. 4 is an automatic focus control system that controls the focus of the imaging lens; 41 is a bandpass filter section that extracts high frequency components from the luminance signal output from the camera system signal processing circuit 3; Select a filter (BPF) a that extracts a frequency band of 200 KH to 1.5 MHz, a filter (BPF) b that extracts a frequency band of 400 KH to 1.5 MHz, and the output signal of either of these filters a or b. It has a changeover switch and a rectifier that rectifies the selected output signal. 42 is an A/D converter that digitizes the input signal, 43 is an integrator that integrates the output signal from the A/D converter, and 44 is a motor drive circuit 45 that controls the motor drive circuit 45 based on the signal output from the integrator 43. 5 is a focus morph that drives the focus mechanism of the imaging lens; 46 is a microcomputer that controls the focus of the imaging lens 1;
is a pulse detection circuit that detects the output pulse of the motor drive circuit 45 and feeds it back to the microcomputer 44.

Next, the operation of this embodiment will be explained. A subject image incident through the m-image lens 1 and formed on the CCD 2 is input as a luminance signal from a camera system signal processing circuit 3 to an automatic focus control system 4 as a video signal. The luminance signal of the video signal is passed through a bandpass filter section 4.
1, its high frequency components are extracted. The reason why the upper band of the luminance signal is limited in this manner is to remove aliasing noise and distortion when digitizing the luminance signal in the A/D converter 42. The high frequency component is 200KH to perform pull-in.
z~1.5 MHz through a low-pass filter a and 400KHz-1,5M to ensure focusing accuracy.
The signal is passed through a high-pass filter b of Hz, half-wave rectified by an appropriate switch, and digitized by an A/D converter 42 at 3.58 MHz/6 bits.

Since this A/D converted signal is high-speed and cannot be input directly to the microcomputer 44, it is sent to the second
The high frequency components within the focus detection area as shown in the figure are integrated, and the microcomputer 44 outputs them every 1/60 seconds.
is sent as focus data to The microcomputer 44 judges the input situation according to the shooting state, and changes the focus morph 5 to two types of modes in a predetermined direction via a motor drive circuit 45, taking into consideration the influence of inertia, time delay, etc. Move it with. These two modes include
There are two modes: a mode that operates quickly in response to large changes (large amplitude mode) and a mode that operates slowly in response to small changes (small amplitude mode).

Specifically, it is as shown in the table below.

FIG. 3 is a diagram showing the basic configuration of the software of the microcomputer 44. The software consists of mountain climbing core software 31 and a disturbance state estimator 32.

FIG. 4 is a diagram showing the concept of the operation of the mountain climbing core software 31, and FIG. 5 is a flowchart of the mountain climbing software. In other words, for the mountain climbing part, first turn the lens to the right or left and watch the increase/decrease in the focus data with Ml, then once you find the direction in which the focus data increases, continue turning it in that direction until you find the peak, and once the peak of the focus data is slightly changed. M3, which determines whether the focus data has exceeded the peak value and switches the bandpass filter as necessary, and M4, which reversely rotates the motor 2- by the amount that exceeds the peak of the focus data to return it to the peak, and the in-focus state. M
It is admonished from each mode of 5. Note that the disturbance state estimation part, which is the gist of this embodiment, is attached to M5.

Here, an example of the basic configuration of the disturbance state estimating section 32 is shown in FIG. The basic operation of this example is to take a moving average of the focus data,
After detecting a change in the scene from this change, the current focus data is time-shifted until the center of gravity is aligned with the moving average, and disturbances such as hand movement are detected from the difference between the two.

The high-frequency signal 100 taken into the microcomputer 44 for each field is added by three data by the moving average calculation means 321 to obtain a new signal in order to improve the S/N.
It is treated as FV which is data. In addition, the final F when the camera is once in focus due to the mountain climbing operation of the climbing core software 31.
Let V be FVO. The moving average calculating means 321 calculates a moving average ENO (Envelope) of subject information by adding, for example, ten consecutive FV data after focusing.
rope: envelope> is obtained. Here, the number of data to be averaged by the moving average calculating means 321 is selected depending on the disturbance situation and response speed, but here it is 1.
It was set to 0. Furthermore, the moving average calculation means 321 calculates ENO and F.
From the difference in VO, EN=1(ENO-10XFVO)/
(IOXFVO) is created as a measure of intersection, and is output to the slow fluctuation detection means 322. At the same time, the delay means 323 delays the FV by 5 data, aligns the center of gravity with the ENO, and outputs the FV to the subtraction means 324.

The subtraction means 324 calculates FL=I (EN-FV)/EN(
Fluctuation (fluctuation) is calculated and outputted to the fast fluctuation calculation means 325. Fast fluctuation detection means 3
25 determines whether the EN exceeds a certain threshold level ENT, and outputs the result to the state estimation and operation mode designation means 326.

EN determined from the moving average ENO indicates slow fluctuations, such as changes in the scene or object, and swing FL indicates fast fluctuations, such as camera shake. Furthermore, fast fluctuation detection means 325
When FL is evaluated on an average frequency scale of 4 degrees, that is, it is evaluated whether the integrated value of PL within 40 data intervals (approximately 2 seconds) of FV exceeds a certain threshold level FLT, and the result is is output to the state estimation and operation mode designation means 326.

In this example, FLT was 3% due to camera shake experiments.
, ENT was 7%, and the average frequency scale data interval was also 40. These constants are selected as appropriate depending on the usage situation on the user side.

The moving average of the high frequency component indicates the magnitude of the contrast of the object itself (corresponding to the position), and the instantaneous value when the center of gravity has moved indicates the rate of change in magnitude, that is, the speed. Both can form a diadarum corresponding to a phase plane to estimate the state of the system, and output the estimation result to the mountain climbing core software 31.

Here, the moving average calculation means 321 and the slow fluctuation detection means 3
Reference numeral 22 constitutes slow fluctuation extraction means, delay means 323 and subtraction means 324 constitute fast fluctuation extraction means, and state estimation and operation mode designation means 326 constitutes state estimation means and mode determination means.

Figure 7 shows (a) focus, (b) blur and camera shake, and (C) focus when photographing a poster at a distance of 3 m using a handheld video camera with a 1/2-inch optical system and a focal length of 54 mm. This is an example showing fluctuations in high frequency signal level in a still image state. There is a close relationship between the level fluctuation of the high-frequency signal and the magnitude or presence of camera shake. When the subject is in focus, the apparent spatial frequency of the subject decreases or increases each time the subject image moves over the photosensitive pixels of the image sensor, so fluctuations in the high frequency level, that is, the effects of camera shake, are Direct tax increases FL. On the other hand, when blur occurs, the circle of confusion spreads and there is no high-frequency component of the subject's spatial frequency components, so there is no fluctuation in the high-frequency level, so F
L is small. Furthermore, when the scene changes, there is generally no correlation between the scenes before and after the change, so the high frequency level itself changes significantly on a regular basis, and the FL also changes transiently. At this time, if blur occurs, the FL will become smaller, but if the image is in focus, the FL will not become smaller.

6 From the above, the states of EN and FL can be simply classified into large and small states, and the state of the disturbance can be given as a pseudo phase plane as shown in Table 1.

Therefore, the state estimation and operation mode estimation means 326 performs the operation shown in Table 2, which assigns a meaning to each state of fluctuation indicated by the pseudo phase plane and specifies the autofocus operation mode. This control specifies whether the operation is a large amplitude operation or a small amplitude operation.

Margin below 7 Table 2 What you should be careful about here is when both EN and FL are small. This state includes a degenerate case where a still image is being taken and a case where the image is largely blurred and stopped. Therefore, it is necessary to try operating autofocus once to confirm. The relationship between the size of EN and FL and the autofocus operation speed and stop is as shown in the third section below.
As shown in the table.

Margin below 8 Table 3 The disturbance state estimating unit 32 of this embodiment (1) does not activate the autofocus operation for a subject for which the causality of the mountain-climbing motion is broken, (2) ensures that the causality of the mountain-climbing motion is confirmed. (3) Activating autofocus operation for guaranteed subjects, and adaptively selecting an operation mode depending on the situation when activating autofocus operation.
Since the blur state can be detected for a subject that is significantly blurred, once the blur state is determined, a focusing operation is activated for confirmation, and by performing control that can reliably recover from the blur state. This can prevent unnecessary autofocus operations from being triggered frequently, or autofocus operations for subjects whose causality in mountain-climbing motion has been disrupted, resulting in incorrect focusing. Since it is possible to always perform reliable focusing on a focusable subject, it is possible to provide an easy-to-see screen as a result.

In the previous embodiment, the magnitude of the disturbance was evaluated using the integral value of FL within a unit time, but it can also be evaluated using an average frequency scale. For example, it can be determined by evaluating how many times a certain threshold FLT is exceeded within 40 FV data intervals (approximately 2 seconds).Also, the time interval for determining the disturbance state can be determined by the operator depending on the situation. In addition, in the previous embodiment, the disturbance was estimated in four states and the control state was given in the form of discrete operation modes, but this is not the only option, and the disturbance state can be It may be more or less, and fuzzy control may be performed using the disturbance EN, FL analog value, lens focal length, and iris value as a membership function.Also, the autofocus method is limited to mountain climbing servo. Needless to say, it can be similarly applied to triangulation type, infrared method, etc.

〔Effect of the invention〕

As described above, according to the automatic focus adjustment device of the present invention,
By detecting high-frequency component disturbances in the video and assigning a meaning to each disturbance state, we can specify the movement modes from O1 to Focus, and as a result, we often find unnecessary images for subjects whose causality in mountain-climbing movements is broken. It becomes possible to start an autofocus operation, detect a blur state even for a subject that has stopped due to a large blur, and reliably recover from this blur state. Naturally, if the causality of the mountain-climbing motion is maintained, it becomes possible to focus more reliably. As a secondary benefit, the number of system operations is reduced, leading to lower power consumption and improved device durability.

[Brief explanation of drawings]

FIG. 1 is a block diagram showing one embodiment of the automatic focus adjustment device of the present invention, FIG. 2 is a diagram showing an example of the focus detection area of the integrator shown in FIG. 1, and FIG. Figure 4 is a conceptual diagram showing the operational concept of the mountain climbing core software shown in Figure 3. Figure 5 is a diagram showing an example of the software configuration of the microcomputer shown in Figure 3. Figure 5 is a conceptual diagram showing the operation concept of the mountain climbing core software shown in Figure 3. 6 is a block diagram showing a detailed configuration example of the disturbance state estimating section shown in FIG. 3, and FIG. 7 is a block diagram showing a detailed configuration example of the disturbance state estimation section shown in FIG. FIG. 3 is a diagram showing an example of level fluctuation of a high-frequency signal in a still image. l...Imaging lens 3...Camera system signal processing circuit 4...Auto focus control system 32...Disturbance state estimation section 42...A/D converter 44...Microcomputer

Claims (1)

[Claims] In an automatic focus adjustment device that obtains a video signal of an optical image formed on an image sensor by an imaging lens, and automatically adjusts the focus of the imaging lens based on a high-frequency component of the video signal, the high-frequency component a slow fluctuation extracting means for extracting a change in the scene or subject imaged by the imaging lens from the above, and a fast fluctuation extraction means for extracting a vibration component due to camera shake or the like taken by the imaging lens from the high frequency component; a state estimating means for estimating the state related to the focus adjustment of the imaging lens based on the extraction results of both of the extracting means; An automatic focus adjustment device characterized by comprising: mode determining means for determining the operating speed of the adjusting mechanism and whether or not the mechanism can operate.