JPH08331430A - Hand shake correcting device and video camera - Google Patents

Abstract

PURPOSE: To effectively correct hand shake, to improve followup ability to shake caused by panning or tilting and further to effectively utilize the surplus picture elements of a CCD image sensor. CONSTITUTION: This device is provided with a mode detection circuit 84 for discriminating whether intentional shake such as panning or tilting or non- intentional shape at least as the state (mode) of shake from motion vector data detected from a video signal and the integrated output of a low-pass filter 94. Based on this mode discriminate signal, the low-pass filter 94 operates the amount of hand shake correction from the motion vector data and a hand shake correct signal is found from a limiter 71.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shake correction device and a video camera for correcting a camera shake component of an image.

[0002]

2. Description of the Related Art In recent years, so-called CCD (charge coupl)
ed device, solid-state image sensor) A handy type video camera equipped with an image sensor is widely used.

Since the above-mentioned video camera is often used for hand-held photographing, there is a problem that camera shake is likely to occur during photographing. In this way, if a camera shake occurs during shooting, for example, when a zoomed-up image is reproduced, the image quality deteriorates due to the camera shake, and the reproduced image becomes very difficult to see.

For this reason, in recent years, a video camera has been commercialized which is equipped with a camera shake correction device capable of correcting the camera shake, thereby correcting the camera shake during photographing.

In addition, with a video camera, panning (a method of shooting by swinging the camera left and right), tilting (a method of shooting by moving the camera from top to bottom or from bottom to top), etc. are sometimes performed at the time of shooting. Many.

Here, a conventional structure of the image stabilizing apparatus mounted on the video camera will be described with reference to FIGS. 22 and 23. Although there are various types of camera shake correction methods, an example using a so-called memory control method will be described here. When the camera shake is detected, the memory control method takes out a part of the video signal obtained by capturing a video with the CCD image sensor of the camera as an image frame, and according to the amount of camera shake, the image frame of the previous field and the image of the current field. This is a method of correcting camera shake by moving the frames so that they are aligned with each other, and by matching these two image frames with each other. In addition, here, as an example of a method for detecting the amount of camera shake, an angular velocity detection method is adopted. The angular velocity detection method is a method in which an angular velocity sensor such as a piezoelectric vibrating gyro is used to detect an angular velocity caused by camera shake, and the amount of camera shake is determined according to the detected angular velocity.

In FIG. 22, the terminal 120 is supplied with angular velocity data from an angular velocity sensor. This angular velocity data is sent to the high pass filter 121. The high-pass filter 121 is a filter that mainly removes low-frequency components caused by panning and tilting of the video camera from the angular velocity data, and allows hand-shake components to pass through as they are.

The output data from the high-pass filter 121 is supplied to the multiplier 127, and the total gain adjuster 12 is supplied.
8 is multiplied by a predetermined multiplication coefficient from
After being multiplied by the multiplication coefficient according to the zoom magnification in the optical zoom in 9, the data is sent to the low-pass filter 154.
Since the gain of the correction signal obtained by the optical zoom and angular velocity sensor of the video camera is not necessarily the design center value, the total gain adjuster 128 uses the multiplication coefficient for correcting the variation of the gain. It is provided for the purpose of generating. Further, the zoom gain table 130 stores a plurality of multiplication coefficients for gain correction according to the zoom magnification in the optical zoom of the video camera.
A multiplication coefficient corresponding to the current zoom magnification of the optical zoom is read from 30 and sent to the multiplier 129. The output data from the multiplier 129 is sent to the low pass filter 154.

The low-pass filter 154 integrates the data supplied from the multiplier 129 at the previous stage using the integration coefficient from the integration coefficient table 136.

Here, the integral coefficient stored in the integral coefficient table 136 has a relationship with the integral output of the low-pass filter 154 as shown in FIG. 23, for example. An integration coefficient corresponding to the integrated value (LPF integrated value) of the low-pass filter 154 is extracted from the integration coefficient table 136, and the low-pass filter 154 uses this integration coefficient to integrate the data supplied from the multiplier 129. To do. The curve showing the relationship between the integration coefficient and the low-pass filter integration value shown in FIG. 23 shows both the horizontal (H) direction and the vertical (V) direction. Also, this figure 2
Among the low-pass filter integrated values (LPF integrated values) in 3, the integrated value (for example, SH) corresponds to half the number of pixels of the horizontal surplus area of the CCD image sensor, and the integrated value (for example, SV) is the CCD. This corresponds to half the number of pixels of the surplus area in the vertical direction of the image sensor. That is,
As shown in FIG. 23, in the existing shake correction apparatus, the correction processing for camera shake and the convergence processing at the time of panning and tilting are performed using a common integral coefficient.

The output data of the low-pass filter 154 comes to be output from the terminal 145 as a camera shake correction signal. The video camera performs camera shake correction processing for correcting the camera shake component based on the camera shake correction signal.

[0012]

However, as described above, if a common integral coefficient is used for the correction of camera shake and the convergence processing at the time of panning and tilting, the correction area for camera shake cannot be made large, and panning, The followability of tilting is also not good. Here, in order to expand the correction range (amplitude) of camera shake, it is sufficient to expand the linear correction area, but as a side effect, the amount of residual camera shake increases. On the contrary, in order to improve the followability of panning and tilting, it is necessary to reduce the linear correction area,
As a side effect in this case, the correction performance is deteriorated.

Further, as shown in FIG. 23, in the image stabilization apparatus adopting the conventional memory control system, the surplus pixels of the CCD image sensor in the vertical direction are, for example, ± 40 pixels in the vertical direction and, for example, ± 60 pixels in the horizontal direction. Even if there are pixels, the current situation is that only about ± 10 pixels are used as the linear correction area.

Therefore, the present invention has been made in view of such a situation, and it is possible to effectively correct camera shake, and the followability to camera shake due to panning or tilting is good, and further, the CCD image sensor. It is an object of the present invention to provide a camera shake correction device and a video camera that can effectively use the surplus pixels of.

[0015]

SUMMARY OF THE INVENTION A camera shake correction apparatus according to the present invention comprises a motion detecting means for detecting a motion of an image from a video signal, and at least an artificial shake state based on the motion detecting signal from the motion detecting means. A shake state determining means capable of distinguishing a shake and a hand shake, and a shake correction amount that calculates a shake correction amount from the motion detection signal based on a shake state determining signal from the shake state determining means, and outputs a shake correction signal. By having a signal output means, the above-mentioned subject is solved.

Further, the video camera of the present invention comprises an image pickup means for generating an electric signal according to the light incident on the image pickup surface,
The image pickup device includes an optical system that forms an incident light image on the image pickup surface of the image pickup device, and a video signal generation device that generates a video signal from the electric signal of the image pickup device. A motion detecting unit for detecting, a shake state determining unit capable of determining at least an intentional shake and a hand shake as a shake state based on a motion detection signal from the motion detecting unit, and a shake state from the shake state determining unit. A shake correction signal output means for calculating a shake correction amount from the motion detection signal based on the determination signal and outputting the shake correction signal;
The above problem is solved by having a shake correction unit that performs shake correction according to the shake correction signal.

[0017]

According to the present invention, the shake state determination means determines the shake state, and the shake correction means calculates the shake correction amount according to the shake state. Therefore, if a shake state, for example, an intentional shake such as panning or tilting and a non-hand shake are discriminated, a correction amount according to the shake state, that is, zero during panning or tilting, The calculated value can be used when camera shake occurs.

[0018]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings.

The image stabilization apparatus of this embodiment is installed in, for example, a handy type video camera, and there are various types of image stabilization methods as will be described later. Here, for example, a so-called memory control method is used. Will be described. The size of the CCD image sensor used in this embodiment is ± 4 in the vertical direction relative to the image frame.
8 pixels larger and ± 63 horizontal than the image frame
The one that is larger by the pixel is used. Therefore, the maximum amount of camera shake correction is ± 48 pixels in the vertical direction and ± 63 pixels in the horizontal direction.

Further, in the embodiment of the present invention, for example, a so-called motion vector detecting method is adopted as a method for detecting the amount of the hand shake. The motion vector detection method is
This is a method of detecting the amount of movement and the direction of the subject by obtaining the difference between the image signals of the subject of the current field and the previous field stored in the semiconductor memory by image processing.

The configuration of a video camera equipped with the image stabilization apparatus of the embodiment of the present invention will be described with reference to FIG.

In FIG. 1, light from a subject or the like that has entered through the optical system 1 enters the CCD image sensor 2 and is converted into an electric signal by the CCD image sensor 2. The optical system 1 may, for example,
A lens system for forming an image on the D image sensor 2,
An optical zoom mechanism when moving the lens system for zooming and a drive system for the optical zoom mechanism, a focusing mechanism when moving the lens system for focusing, and a drive system for the focusing mechanism, an iris mechanism and the iris mechanism It is composed of a drive system. Further, the CCD image sensor 2 in this case is composed of an optical filter and three CCD image sensors for receiving the respective colored lights.

The image pickup signal from the CCD image sensor 2 is subjected to auto-gain control for automatically adjusting the signal gain in the signal adjusting circuit 3 and sample and hold, and then digitalized by the analog / digital (A / D) converter 4. It is converted into an image pickup signal. This digital image pickup signal is sent to the camera signal processing circuit 5.

The camera signal processing circuit 5 digitally processes signal processing of the CCD color camera such as generation of a luminance (Y) signal and a chroma (C) signal from a digital image pickup signal. The camera signal obtained by the camera signal processing circuit 5 is output from the output terminal 6.

The camera control circuit 9 controls the drive of the optical zoom mechanism, the drive control for performing the autofocus control in the focusing mechanism, the drive control for performing the automatic iris control in the iris mechanism, and the entire system of the video camera. Controls other than the image stabilization processing, which will be described later, such as the timing control of the

The camera shake correction instructing means 16 is composed of, for example, a button provided on the housing of the video camera, and is used by the user of the video camera to instruct whether to perform the camera shake correction. This image stabilization instruction means 16
When the user instructs to perform camera shake correction by operating, the camera control circuit 9 sends a signal to the image control circuit 8 indicating that camera shake correction processing should be performed.

Further, the motion detection circuit 18 is a circuit for obtaining and outputting the deviation having the highest frame correlation (or field correlation) with respect to the camera signal output from the camera signal processing circuit 5. The motion detection circuit 18
As a method of detecting a motion vector in, a so-called representative point matching method may be used, for example. This representative point matching method finds a correlation value between a representative point in a block of a current frame (field) and a pixel distant from the representative point in the block of the next frame (field) and the cumulative value of these correlation values. Then, the motion vector is obtained. The motion vector data detected by the motion detection circuit 18 is sent to the image control circuit 8.

The sync generator (SG) 14 generates a horizontal synchronizing signal HD and a vertical synchronizing signal VD and a field discriminating signal FP, and the horizontal synchronizing signal HD and the vertical synchronizing signal VD are sent to the timing generator (TG) 10 as a field. The discrimination signal FP is sent to the image control circuit 8.

The image control circuit 8 has a CC
The electronic zoom control for enlarging the image picked up by the D image sensor 2 is performed, and when the camera control circuit 9 supplies a signal indicating that the camera shake correction is to be performed, the camera shake correction processing program stored therein is used. The camera shake correction amount calculation is performed based on the motion vector data from the motion detection circuit 18. That is,
The image control circuit 8 detects the shake component of the image from the supplied motion vector data, and calculates the shake correction amount corresponding to this shake component.

The correction value obtained by the camera shake correction amount calculation in the image control circuit 8 is transferred to the timing generator 10 and the linear interpolation calculation circuit 7 as serial data.

The serial data transferred from the image control circuit 8 to the linear interpolation calculation circuit 7 is as follows:
The horizontal enlargement / reduction ratio HMAG and the vertical enlargement / reduction ratio value VMAG, the horizontal interpolation offset value HOFF, and the even field vertical interpolation offset value V.
OFFE and vertical interpolation offset value V of odd field
OFFO, the write end address value HSTOP to the line memory arranged in the linear interpolation calculation circuit 7, the write start address value HSTART to the line memory, the rising / writing control rising phase value HCPS of the line memory, and the line Falling phase value HCPE for writing / reading control of memory, writing start phase value MWBS of line memory, writing end phase value MWBE of line memory, reading start phase value MRBS of line memory, and reading end of line memory Phase value MR
There is BE.

Here, in the present embodiment, the camera shake correction processing in the vertical (V) direction and the camera shake correction processing in the horizontal (H) direction are performed as follows.

First, the vertical shake correction processing will be described.

Regarding the initial value setting of each field, in the correction processing of the integer part, the image control circuit 8
From the field read control signal FLD of the CCD image sensor 2 and the vertical sweep pixel value VTB to the image output from the CCD image sensor 2 to the timing generator 10 so that the CCD image sensor 2 can perform offset reading. Control. Further, in the correction processing of the decimal part, the signal of the vertical interpolation offset value VOFFE of the even field and the vertical interpolation offset value VOFFO of the odd field is sent from the image control circuit 8 to the linear interpolation calculation circuit 7 to control the interpolation processing. .

Further, regarding the successive interpolation processing of each line, the image control circuit 8 enlarges / extends in the vertical direction.
By sending the signal of the reduction ratio value VMAG to the linear interpolation calculation circuit 7 and, in synchronization with the horizontal synchronization signal HD, the VGAT signal indicating the presence or absence of the carry-out of the fractional addition section to the timing generator 10. Control.

Next, the horizontal shake correction process will be described.

Regarding the initial value setting of each line, in the correction processing of the integer part, the write start address value HSTART from the image control circuit 8 to the line memory arranged in the linear interpolation calculation circuit 7 and the write to the line memory. Control is performed by sending a signal of the end address value HSTOP to the linear interpolation calculation circuit 7. Further, in the correction processing of the decimal part, the signal of the interpolation offset value HOFF in the horizontal direction is sent from the image control circuit 8 to the linear interpolation calculation circuit 7 to control the interpolation processing.

Regarding the successive interpolation processing of each pixel,
The image control circuit 8 sends a horizontal enlargement / reduction ratio HMAG signal to the linear interpolation calculation circuit 7 for control.

Regarding the interpolation processing of the decimal part,
Since both the horizontal and vertical directions are performed by the linear interpolation calculation circuit 7, the image control circuit 8 outputs the rising phase value HCPS for writing / reading control of the line memory.
, The falling phase value HCPE for writing / reading control of the line memory, the writing start phase value MWBS of the line memory, and the writing end phase value M of the line memory.
WBE and read start phase value MRBS of line memory
Then, each signal of the read end phase value MRBE of the line memory is transferred to the linear interpolation calculation circuit 7.

Further, the timing generator 10 has a C
V-drive 1 that drives the CD image sensor 2
5, the frame shift operation control signals XV1 to XV4 corresponding to the camera shake correction amount are transferred, and the high-speed sweep operation control signal XSUB is transferred.

Next, the structure of a specific example for generating a camera shake correction signal provided in the image control circuit 8 will be described with reference to FIG.

The structure for generating the shake correction signal in the image control circuit 8 is a monitor section (that is, shake state determination means) for analyzing the shake motion of the video camera using the motion vector data detected by the motion detection circuit 18. ) And a calculation unit (that is, a shake correction signal output unit) that calculates a shake correction amount according to the result of the operation analysis and outputs a shake correction signal. The monitor section is
It has a mode detection circuit 84 as a main component,
The arithmetic unit has a limiter 71, an attenuator (attenuator) 93, and a low-pass filter 94 as main components.

In FIG. 2, the terminal 70 is supplied with the motion vector data from the motion detection circuit 18 of FIG. This motion vector data is sent to the limiter 71 of the arithmetic unit and the mode detection circuit 84 of the monitor unit.

The limiter 71 of the arithmetic unit operates when the motion vector data corresponding to the shake of the video camera exceeds a predetermined limit value, in other words, on the CCD image sensor 2 caused by the shake of the video camera. When the moving speed amount of the image (moving speed amount corresponding to the number of pixels) is equal to or larger than a predetermined limit value, the low pass filter 94 in the subsequent stage is restricted from inputting a value equal to or larger than the limit value. Also, this limit value is
It is also set for the mode detection circuit 84 to detect whether or not the motion vector data is caused by panning or tilting. This limiter 7
The output data of 1 is sent to the multiplier 72 of the attenuator 93.

The attenuator 93 comprises a multiplier 72 and an attenuation coefficient generator 73 for generating an attenuation coefficient K _{3 by} which the output data of the limiter 71 is multiplied by the multiplier 72. It controls the gain of the data sent to the filter 94. The attenuation coefficient generator 73 outputs 0 ≦ K ₃ as an attenuation coefficient K ₃ according to the detection result of the mode detection circuit 84 of the monitor unit, as described later.
A value of ≤1 is output. The output data from the attenuator 93 is sent to the low pass filter 94.

The low-pass filter 94 includes an adder 74.
It has a register 77, a multiplier 75, and an integration coefficient generator 76, and integrates the data supplied from the previous stage attenuator 93 according to each mode detected by the mode detection circuit 84. That is, the output data from the attenuator 93 at the previous stage is supplied to the adder 74 as an addition signal, and the output data of the adder 74 is sent to the multiplier 75 via the register 77. A predetermined filter coefficient (integral coefficient K ₄ ) is supplied from the integral coefficient generator 76 to the multiplier 75, and the output data of the register 77 is multiplied by the integral coefficient K ₄ . The output data of the multiplier 75 is sent to the adder 74, and is added to the output data of the previous-stage attenuator 93 in the adder 74. Here, the integration coefficient generator 76 outputs, for example, 0 ..
The integral coefficient K _{4 of} 80 ≦ K ₄ ≦ 0.999 is output. The output data of the low-pass filter 94 is output from the terminal 85 as a camera shake correction signal.

On the other hand, the mode detection circuit 84 of the monitor section
The motion vector data supplied via the terminal 70 is received for a certain fixed time (for example, about 0.5 seconds), and the output data of the low-pass filter 94 of the arithmetic unit is received.
Then, the mode detection circuit 84 analyzes the shake state of the video camera (operation analysis) based on these data, and according to the shake state of the video camera obtained by the analysis, the attenuation coefficient is calculated. The subtraction coefficient K ₃ output from the generator 73 is controlled, and the integration coefficient K ₄ output from the integration coefficient generator 76 is controlled.

Here, in this embodiment, by controlling the subtraction coefficient K ₃ of the attenuation coefficient generator 73 and the integration coefficient K ₄ of the integration coefficient generator 76 as follows, the shake of the video camera can be suppressed. Make a correction.

First, as the shake state of the video camera, for example, when the photographer intentionally shakes the camera, it is regarded as a panning or tilting operation.
In this case, the camera shake correction processing is not performed so that the panning or tilting operation is accompanied by the movement of the image as much as possible. Specifically, as shown in FIG. 3, the damping coefficient K ₃ and the integration coefficient K ₄ are made as small as possible, and the output value of the low-pass filter 94 which is the camera shake correction output is set to zero. In this way, the mode of the processing that minimizes the correction amount, such as during the panning or tilting operation, is referred to as the convergence mode. It should be noted that, of the low-pass filter integrated values (LPF integrated values) shown in FIG. 3, the integrated value SH corresponds to half the number of pixels in the horizontal surplus area of the CCD image sensor 2, and the integrated value SV is the CCD image sensor 2. Corresponds to 1/2 the number of pixels in the vertical surplus area.

Next, in the case where the video camera shakes as described above, if the video camera shakes regardless of the intention of the photographer, it is considered that the video camera shakes due to camera shake.
In this case, camera shake correction processing is performed so as to eliminate image shake as much as possible. Specifically, as shown in FIG. 4, the damping coefficient K ₃ and the integration coefficient K ₄ are made as close to 1 as possible so that the calculated value for camera shake correction becomes the output value of the low-pass filter 94 as it is. The mode of the correction process that maximizes the correction amount (approaches 100% correction) such as when the video camera shakes regardless of the will of the photographer is hereinafter referred to as a correction mode. Of the LPF integral values shown in FIG. 4, the integral value SH corresponds to half the number of pixels in the horizontal surplus area of the CCD image sensor 2, and the integral value SV is C.
1/2 of the surplus area in the vertical direction of the CD image sensor 2
When the processing is performed in the correction mode in the steady state, within 1/2 of the number of pixels in the surplus area in the horizontal and vertical directions of the CCD image sensor 2 is used for camera shake correction. It becomes the usable area. However, if all the half of the surplus area of the number of pixels in the horizontal and vertical directions is used for the correction process in the steady state, the continuity of the image may be lost when the half of the surplus area is exceeded. Therefore, in practice, as shown in FIG. 4, the area (the LPF integrated value sh in the horizontal direction and the LPF integrated value sv in the vertical direction is smaller than half the number of pixels in the surplus area by a predetermined number of pixels. ) Is used for the correction process, and the integration coefficient K ₄ is made smaller so that the correction amount becomes smaller than that, and the convergence is made.

Further, as the shake condition of the video camera, there is an intermediate condition between the panning or tilting motion in which the photographer intentionally shakes the camera and the shake condition in which the video camera shakes regardless of the intention of the photographer. In some cases, processing for converging while performing camera shake correction is performed. Specifically, as shown in FIG. 5, by setting the damping coefficient K ₃ and the integration coefficient K ₄ to appropriate values, it is possible to converge while correcting. The mode of processing for performing correction and convergence in the case of an intermediate state between the panning or tilting operation and the camera shake state in this manner is hereinafter referred to as a quasi-correction mode. In the case of FIG. 5 as well, of the LPF integrated values, the integrated value SH corresponds to half the number of pixels in the horizontal surplus area of the CCD image sensor 2, and the integrated value SV is the vertical direction of the CCD image sensor 2. This corresponds to half the number of pixels in the surplus area, and when processing is performed in the quasi-correction mode in the steady state, the number of pixels in the surplus area in the horizontal and vertical directions of the CCD image sensor 2 is 1 The area within 1/2 is the area that can be used for camera shake correction. However, if all the half of the surplus area of the number of pixels in the horizontal and vertical directions is used for the correction process in the steady state, the continuity of the image may be lost when the half of the surplus area is exceeded. Therefore, in practice, as shown in FIG. 5, the area is smaller than half the number of pixels in the surplus area by a predetermined number of pixels (up to the LPF integral value sh in the horizontal direction and the LPF integral value sv in the vertical direction). ) Is used for the correction process, and the integration coefficient K
₄ is made small so that it converges.

The above-mentioned damping coefficient K ₃ is K _3X <K _3Y
<K _3Z , and the integration coefficient K ₄ is K _4X <K _4Y <K _4Z
Have a relationship. However, in the relational expression, X
Indicates a convergence mode, Y corresponds to a quasi-correction mode, and Z corresponds to a correction mode.

In this embodiment, any one of the convergence mode, the correction mode and the quasi-correction mode is used to correct the shake of the video camera. The mode detection circuit 84 analyzes whether the video camera shakes or not, to determine whether or not to use. That is, in the mode detection circuit 84, which one of the above processing modes should be used depending on whether the shake state of the video camera matches any one of the following seven kinds of determination conditions. Making a decision.

The reference (judgment condition) for the mode judgment in the mode detection circuit 84 and the correction processing in the image stabilization apparatus of the present embodiment according to the mode judgment result will be described.

Here, as shown in FIG. 6, the camera-shake correction apparatus of the present embodiment has a mode detecting circuit 84 which will be described later with reference to FIGS. 9 to 9 during a specified time t ₀ seconds (for example, 30 fields).
Mode judgment is performed with seven kinds of judgment conditions as shown in FIG.
At the next specified time t _{0 to} 2t ₀ , the correction processing is performed according to the determination result, and the mode is determined for the next specified time. In the case of the forced convergence mode described later, as shown in FIG. 7, the forced interruption is performed as soon as the judgment condition of the forced convergence mode is satisfied during the processing of any mode. Immediately execute the processing of the convergence mode.

Further, the number of zero crossings of the motion vector data used in the description of the mode judgment below is defined as shown in FIG. In FIG. 8, the number of samples (30 samples) in a specified time is Sm, the value within a predetermined threshold level (THL ₊ or THL ₋ ) is regarded as a 0 value, and the number of signs of motion vector inversion within a specified time is defined. Pn
And the motion vector is the limit value LIM of the limiter 71.
Let Sx be the number of samples while exceeding. However, the motion vector concerned has a limit value LI continuously for 10 samples.
When it exceeds M, the count of the zero-cross number is reset at that time, and the number of samples from the first zero-cross point after the count reset to the end of the specified time Sm is S.
n. Therefore, in FIG. 8, for example, Sx <
When 10, Pn = 9, the number of samples S = Sm, and Sx
When ≧ 10, Pn = 4 and the number of samples S = Sn. The calculation of the motion vector is 1 sample / field.

Under such a premise, the mode detection circuit 84 makes a mode decision according to the following mode decision conditions, and the image stabilization apparatus of this embodiment carries out a shake correction process according to the mode decision result. .

First, as shown in FIG. 9, as a determination condition, when the current mode is determined, the motion vector data which exceeds the predetermined limit value LIM of the limiter 71 is 10
When the number of zero crosses at which the motion vector data crosses a predetermined threshold level (THL ₊ or THL ₋ ) is 3 or more and 7 or less without being continuous in the field, the processing mode is the correction mode shown in FIG. And That is, in the processing in the steady state when this determination condition is satisfied, the 31st field after the specified time when the mode determination is performed (the first field of the next specified time)
Therefore, the damping coefficient K ₃ = 1 is set, and the integration coefficient is processed in the correction mode using the integration coefficient K ₄ of FIG.

Next, as shown in FIG. 10, as the judgment condition, the previous mode is the correction or quasi-correction mode, and the predetermined limit value L of the limiter 71 is judged when the current mode is judged.
When the motion vector data exceeding IM continues for 10 fields, the processing mode is forcibly set to the convergence mode shown in FIG. That is, in the processing in the steady state when this determination condition is satisfied, the processing in the convergence mode is forcibly started from t ₁ when the determination condition is satisfied, the damping coefficient K ₃ = 0, and the integration coefficient is as shown in FIG. Integration coefficient K ₄ (= 0.
Perform processing using 9). Even if the correction or quasi-correction mode exists in the first half of the current mode determination, when the determination condition is satisfied, the forced convergence mode is prioritized.

Next, as shown in FIG. 11, as the judgment condition, the previous mode is the forced convergence mode or the continuous convergence mode described here, and the limit of the limiter 71 at the time of judging the current mode. When the motion vector data exceeding the value LIM continues for 10 fields, the processing mode is continuously set to the convergence mode shown in FIG. That is, in the processing in the steady state when this determination condition is satisfied, the 31st field after the specified time when the mode determination is performed (the first field of the next specified time)
Thereafter, the damping coefficient K ₃ = 0 is set, and the integration coefficient K ₄ (= 0.9) in FIG. 3 is used as the integration coefficient to perform the continuous convergence mode processing. If the correction mode occurs in the latter half of the current mode determination, the correction mode is prioritized.

Next, as shown in FIG. 12, as a judgment condition, when the motion vector data does not exceed the predetermined threshold level (THL ₊ or THL ₋ ) for 30 fields, the processing mode is set to the above-mentioned. The convergence mode shown in FIG. 3 is set. That is, in the processing in the steady state when this determination condition is satisfied, the damping coefficient K ₃ is set to 0 from the 31st field (the first field of the next specified time) following the specified time when the mode determination is performed. , The integration coefficient is the static convergence mode processing using the integration coefficient K ₄ (= 0.9) in FIG. The setting of the threshold level is represented by the maximum value of motion vector data when the video camera is fixed and stationary on a tripod or a desk, for example.

Next, as shown in FIG. 13, as a determination condition, the motion vector data exceeding the limit value LIM of the limiter 71 does not continue for 10 fields at the time of determining the current mode, and the motion vector data has a predetermined threshold value. When the number of zero-crosses that cross the hold level (THL ₊ or THL ₋ ) is 2 or less and the static convergence mode is not set, the processing mode is set to the quasi-correction mode shown in FIG. That is, in the processing in the steady state when this determination condition is satisfied, the damping coefficient K _{3 is set} to K ₃ from the 31st field (the first field of the next specified time) following the specified time when the mode determination is performed. <1, and the integration coefficient is processed in the quasi-correction mode using the integration coefficient K ₄ shown in FIG.

Next, as shown in FIG. 14, as the determination condition, the previous mode is the correction mode, and the motion vector data exceeding the limit value LIM of the limiter 71 does not continue for 10 fields when the current mode is determined. , The motion vector data has a predetermined threshold level (TH
L ₊ or THL _-) when the number of zero crossings of the cross is present 8 or more, and to the correction mode showing the mode of processing in FIG. 4. That is, in the processing in the steady state when this determination condition is satisfied, the processing in the correction mode using the integration coefficient K ₄ in FIG. 4 is performed within the specified time for performing the mode determination. Note that the reason why the correction mode is used when this determination condition is satisfied is to prevent an erroneous determination from being caused by noise in the motion vector data rather than the natural vibration of the video camera.

Next, as shown in FIG. 15, as the determination condition, the previous mode is a mode other than the correction mode, and when the current mode is determined, motion vector data exceeding the limit value LIM of the limiter 71 continues for 10 fields. However, when there are eight or more zero crosses at which the motion vector data crosses a predetermined threshold level (THL ₊ or THL ₋ ), the processing mode is changed to that shown in FIG.
As described in 1, the continuous convergence mode is set. That is, in the processing in the steady state when this determination condition is satisfied, the continuous convergence mode using the integration coefficient K ₄ (= 0.9) in FIG. Process. Further, the reason why the continuous convergence mode is used when this determination condition is satisfied is that the motion vector data is based on the natural vibration of the video camera.

Here, the mode detection circuit 84 described above is
When determining the mode, the motion vector data is limited by the limiter 71.
The counting of the number of continuous fields exceeding the predetermined limit value LIM (count value is 00h to 0Fh) and the reset and recount (recount) operation of the count are performed according to the following conditions.

That is, in the following cases, the mode detection circuit 84 clears the count of the number of continuous fields in which the motion vector data exceeds the predetermined limit value LIM, and the re-count operation is started. For example, as shown in FIG. 16, when the final value (motion vector data) of the previous mode determination section (specified time) is smaller than the predetermined limit value LIM, the current mode determination section (specified time) is entered. clear. In addition, as shown in FIG. 17, the motion vector data has the above limit value LI within the determination section of the current mode.
It is cleared when the count number is smaller than 10 and the motion vector data takes a value smaller than the limit value LIM during the counting operation of the number of consecutive fields exceeding M. Further, as shown in FIG. 18, the forced convergence mode is established in the previous mode, and it is cleared when shifting to the next current mode determination section.

On the other hand, as shown in FIG. 19, the previous mode is not the forced convergence mode, and the last motion vector data of the previous mode takes a value larger than the limit value LIM (however, the count value of the number of consecutive fields is Is smaller than 10), when the previous mode determination section is shifted to the current mode determination section, the count of the number of fields is not cleared and the counting operation is continued. Also, FIG.
As shown in, the previous mode is the forced convergence mode or the continuous convergence mode, and when the count value of the number of continuous fields where the motion vector data exceeds the limit value LIM at the time of determining the current mode is 10 ( (At the time of crossing)
Then, the limit flag is set and the count value is set to 1.
Hold at 0. However, after the limit flag is set, the count is cleared when the motion vector data is zero-crossed in the same determination section (determination section in the current mode) or when the correction mode is determined. The flag is cleared at the time of shifting to the next determination section.

In the above example, the mode detection circuit 84
In the mode determination in (1), the determination is performed based on the above-described seven types of mode determination conditions, but it is also possible to perform the following. For example, it is possible to change the definition of the correction / convergence mode so that either one of the quasi-correction modes is fetched and processed. At this time, the damping coefficient K ₃ shown in FIGS. 3 to 5 and the correction coefficient shown as the integration coefficient K ₄ are changed at the same time. Also, for example, instead of defining the continuous convergence mode, performing the processing in the forced convergence mode, on the contrary, without defining the forced convergence mode,
It is also possible to substitute the processing in the continuous convergence mode. Furthermore, for example, it is possible to handle the modes other than the correction mode and the static convergence mode as the convergence mode or the quasi-correction mode without setting the judgment using the motion vector data exceeding the predetermined limit value in the limiter 71. As described above, many correction processing algorithms (e.g., eight types) can be considered in addition to the above-described embodiments.

By the way, in the case where the mode determination is performed as described above and the processing is performed according to each of the determined modes, the processing is also switched at the time of the mode transition in which the mode changes, so that the processing is smooth. It is possible that the movement of the image cannot be obtained. Therefore, in the present embodiment, in order to smooth the movement of the image at the time of mode transition, the attenuation coefficient K ₃ and the integration coefficient K ₄ are made continuous as follows. There are six cases of mode transition: correction mode to convergence mode, convergence mode to correction mode, correction mode to quasi-correction mode, quasi-correction mode to correction mode, convergence mode to quasi-correction mode, and quasi-correction mode to convergence mode. Although conceivable, since the processing method is the same in any case, the case of mode transition from the correction mode to the convergence mode will be described as an example, and other cases will be omitted.

First, as shown in FIGS. 3 to 5, the integration coefficient K ₄ in the horizontal (H) direction does not change depending on the mode in the section where the integrated value of the low-pass filter 94 is from SH to SM. It is not necessary to perform the continuity processing of the coefficient. Also, the integration coefficient K ₄ in the vertical (V) direction is
Since the integral value of the low-pass filter 94 does not change depending on the mode in the section from SV to SM, it is not necessary to perform the continuity processing of the coefficient in this section.

Next, the integrated value of the low-pass filter 94 is 0.
The horizontal processing in the section from SH to SH will be described with reference to FIG.

In FIG. 21, paying attention to the point of Q ₁ in the figure, in order to make the mode transition continuously from the correction mode to the convergence mode, the integration coefficient K ₄ is changed from 0.999 to 0.9 by 30 fields. Must be continuously changed during. Therefore, when transitioning from the correction mode to the convergence mode, the integration coefficient K in the correction mode
₄ of 0.999 and the integration coefficient K ₄ of 0.
9 (that is, 0.999-0.9) is divided by 30 field samples, and every 1 field, 0.0
The integration coefficient K ₄ is reduced by 99/30. In the section where the integrated value of the low-pass filter 94 is from 0 to sh, 0.09 is obtained for each one field as described above.
The integration coefficient K ₄ is reduced by 9/30.
On the other hand, in the section where the integral value of the low-pass filter 94 is from sh to SH, Q ₂ obtained by reducing the integral coefficient K ₄ at the Q ₁ point by 0.099 / 30 every one field, and a fixed point diagram The middle Q ₃ point is connected by a line segment, and the integration coefficient K ₄ corresponding to the current integral value of the low-pass filter 94 is obtained from the line segment.

As described above, in the mode transition from the correction mode to the convergence mode, the section from the integration value 0 to sh of the low-pass filter 94, the section from the integration value sh to SH, and the integration value SH to SM. By performing the above-described processing continuously for 30 fields for each section of, the continuity of the integration coefficient K ₄ can be maintained.

Regarding the continuity processing of the attenuation coefficient K ₃ , since it is not a function of the integrated value of the low pass filter 94,
Processing is performed so that the integration coefficient K ₄ is reduced by (1-0) / 30 every one field from the value 1 to the value 0 of the attenuation coefficient K ₃ .

As described above, in the present embodiment, the motion vector data obtained from the motion detection circuit 18 and the output data (integral value) of the low-pass filter 94 are used to set the shake condition of the video camera to several modes. By classifying and combining processing suitable for them, the overall performance of the camera shake correction performance and the panning / tilting tracking performance is improved. That is, according to the image stabilization apparatus and the video camera of the present embodiment, most of the panning and tilting components are deleted as the data that enters the low-pass filter, and the input data has a large periodicity, so the linear correction area is large. Can be secured (same CC
Even if the amount of D surplus area is large, a large correction capability can be provided. Further, according to the image stabilization apparatus and the video camera of the present embodiment, the convergence processing is performed at the time of panning and tilting, so that residual camera shake is less likely to occur than in the conventional image stabilization apparatus (the followability of panning and tilting is improved. good). Further, according to the image stabilization apparatus and the video camera of the present embodiment, the surplus pixels of the CCD image sensor can be effectively utilized, and a large correction capacity can be provided even with the same surplus pixel amount.

In the above-mentioned embodiment, the memory control method is taken as an example of the method for correcting the camera shake, but it is also possible to use a method for correcting the camera shake by optical processing. . As a method for correcting camera shake by the above optical processing, a gimbal mechanical method,
The active prism system is known. When the camera shake is detected, the gimbal mechanical system corrects the camera shake by moving the entire lens unit in a direction of canceling the camera shake by the optical system driving means. According to this method, since the entire lens unit is moved, the mechanism is large and power consumption is large, but there is no deterioration in resolution and the correction range is relatively wide, so high resolution can be obtained even if it is somewhat large. Suitable when you want. Further, in the active prism system, when a camera shake is detected, only a part of the lens unit is moved in a direction of canceling the camera shake by the optical system driving means to correct the camera shake. According to this method, compared to the gimbal mechanism method, the power consumption is small, the size can be easily reduced, the resolution is not deteriorated, and the correction range can be set relatively wide. Is suitable. That is, even when the method of correcting the camera shake by these optical processes is used, the mode determination of the camera shake is performed as described above, and the correction is performed according to the mode determination result, so that the camera can follow the shake caused by the panning or the tilting. In addition, it is possible to effectively correct camera shake.

Further, in the above-described embodiment of the present invention, the motion vector detecting method is adopted as the camera shake detecting method, but other than this, for example, the above-mentioned angular velocity detecting method may be used. According to this angular velocity detection method, it is possible to detect in real time without malfunctioning under low illuminance conditions. Therefore, according to this method, it is possible to accurately perform camera shake correction.

[0078]

As is apparent from the above description, according to the present invention, the shake state determination means determines the shake state, and the shake correction means determines the shake correction amount according to the shake state. I am calculating. At this time, if it is determined whether the shake state is an artificial shake such as panning or tilting and a non-artificial shake, the correction amount according to the shake state, that is, the correction amount is zero when panning or tilting is performed. However, 100% correction can be used during camera shake, and therefore, according to the present invention, it is possible to effectively correct camera shake and improve followability to camera shake due to panning or tilting. Further, it becomes possible to effectively use the surplus pixels of the CCD image sensor. In addition,
In the conventional device, when the correction button is turned on, there are side effects such as panning on a tripod, and the button is turned off. On the other hand, in the case of the present invention, since processing suitable for camera operation is performed each time, camera shake correction is turned on / off.
The OFF button is unnecessary.

[Brief description of drawings]

FIG. 1 is a block circuit diagram showing a schematic configuration of a video camera incorporating an image stabilizing apparatus according to an embodiment of the present invention.

FIG. 2 is a block circuit diagram showing a configuration for generating an image stabilization signal provided in an image control circuit of a video camera according to an embodiment of the present invention.

FIG. 3 is a diagram showing a relationship between an integration coefficient and a low-pass filter integration value in a convergence mode.

FIG. 4 is a diagram showing a relationship between an integration coefficient and a low-pass filter integration value in a correction mode.

FIG. 5 is a diagram showing a relationship between an integration coefficient and a low-pass filter integration value in the quasi-correction mode.

FIG. 6 is a diagram for explaining a flow of a mode determination and correction execution process.

FIG. 7 is a diagram for explaining a flow of execution processing of correction of mode determination, by giving a specific mode name.

FIG. 8 is a diagram for explaining the definition of each term.

FIG. 9 is a diagram for explaining a determination condition of a correction mode.

FIG. 10 is a diagram for explaining a determination condition of a forced convergence mode.

FIG. 11 is a diagram for explaining a determination condition of a continuous convergence mode.

FIG. 12 is a diagram for explaining a determination condition of a static convergence mode.

FIG. 13 is a diagram for explaining a determination condition of a semi-correction mode.

FIG. 14 is a diagram for explaining the condition of the mode determination for not performing the mode determination due to the noise of the motion vector data instead of the natural vibration of the video camera.

[Fig. 15] Fig. 15 is a diagram for describing a condition for mode determination for preventing mode determination due to natural vibration of a video camera.

FIG. 16 describes conditions for count clearing and recounting of the number of consecutive fields in which motion vector data exceeds a predetermined limit value (when shifting to the current mode determination section when the final value of the previous mode is smaller than the limit value). FIG.

FIG. 17 describes conditions for count clearing and recounting the number of consecutive fields in which motion vector data exceeds a predetermined limit value (when the motion vector data value is smaller than the limiter value during the counting operation). FIG.

FIG. 18 is a condition for count clearing and recounting of the number of continuous fields in which the motion vector data exceeds a predetermined limit value (when a forced convergence mode is established in the previous mode and a transition is made to the next mode determination section). It is a figure for explaining.

FIG. 19 is a condition for continuing the count operation without clearing the count of the number of continuous fields in which the motion vector data exceeds a predetermined limit value (while the motion vector data counting operation exceeding the limit value is being performed, the following operation is performed from the current mode determination section). FIG. 7 is a diagram for explaining a case of shifting to the determination section of FIG.

FIG. 20 is a condition for holding the count value of the number of consecutive fields in which the motion vector data exceeds a predetermined limit value (the previous mode is a forced convergence mode or a continuous convergence mode, and the limit value is exceeded when the current mode is determined. It is a figure for demonstrating when the count value of motion vector data became a predetermined value.

FIG. 21 is a diagram for explaining mode switching at the time of mode transition.

FIG. 22 is a block circuit diagram showing a schematic configuration of a conventional configuration for generating a camera shake correction signal.

FIG. 23 is a diagram for explaining a common integration coefficient for performing conventional camera shake correction and convergence processing.

[Explanation of symbols]

 5 camera signal processing circuit 7 linear interpolation calculation circuit 8 image control circuit 9 camera control circuit 10 timing generator 16 camera shake correction instruction means 17 field or frame memory 18 motion detection circuit 71 limiter 84 mode detection circuit 93 attenuator 94 low-pass filter

Claims (8)

[Claims] 1. A motion detecting means for detecting a motion of an image from a video signal, and a shake state determination capable of determining at least an intentional shake and a shake as a shake state based on a motion detection signal from the motion detecting means. And a shake correction signal output means for calculating a shake correction amount from the motion detection signal based on the shake state determination signal from the shake state determination means and outputting a shake correction signal. Correction device. 2. The shake correction signal output means sets the shake correction amount to zero when the shake state determination signal indicates the artificial shake state, and sets the shake correction amount to the calculated value as it is when indicating the shake state. The image stabilization device according to claim 1, wherein 3. The shake correction signal output means includes at least an attenuating means for attenuating the motion detection signal and a low-pass filter for integrating an output signal of the attenuating means, and the shake correcting signal output means responds to the shake state determination signal. 2. The image stabilizing apparatus according to claim 1, wherein the damping coefficient of the damping means and the integration coefficient of the low pass filter are controlled. 4. An image pickup means for generating an electric signal according to light incident on the image pickup surface, an optical system for forming an incident light image on the image pickup surface of the image pickup means, and an image from the electric signal of the image pickup means. A video signal generating means for generating a signal, a motion detecting means for detecting a motion of an image from the video signal, and a shake condition based on the motion detecting signal from the motion detecting means, at least an intentional shake and a shake. And a shake correction signal output for calculating a shake correction amount from the motion detection signal based on the shake state judgment signal from the shake state judgment means and outputting the shake correction signal. A video camera, comprising: a device and a shake correction device that performs a shake correction according to the shake correction signal. 5. The shake correction signal output means sets the shake correction amount to zero when the shake state determination signal indicates the artificial shake state, and sets the shake correction amount as the calculated value as it is when indicating the shake state. 5. The video camera according to claim 4, wherein 6. The shake correction signal output means includes at least an attenuating means for attenuating the motion detection signal and a low-pass filter for integrating an output signal of the attenuating means, and the shake correcting signal output means responds to the shake state determination signal. 5. The video camera according to claim 4, wherein the attenuation coefficient of the attenuation means and the integration coefficient of the low pass filter are controlled. 7. The shake correcting means extracts a part of the video signal as an image frame, moves the image frame of the previous field and the image frame of the current field so as to match each other according to the shake correction amount, and The video camera according to claim 4, wherein the shake is corrected by matching both image frames with each other. 8. The shake correction means has an optical system drive means for driving the optical system to shift the position of an incident light image formed on the image pickup surface of the image pickup means, and according to the shake correction amount. 5. The video camera according to claim 4, wherein the shake is corrected by shifting the position of the incident light image formed on the image pickup surface of the image pickup means.