JP3460382B2 - Image stabilizer and video camera - Google Patents

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a shake correction device and a video camera for correcting a shake component of an image.

[0002]

2. Description of the Related Art In recent years, so-called CCD (charge coupl)
ed device, solid-state image sensor) A handy type video camera equipped with an image sensor is widely used.

Since the above-mentioned video camera is often used for hand-held photographing, there is a problem that camera shake is likely to occur during photographing. In this way, if a camera shake occurs during shooting, for example, when a zoomed-up image is reproduced, the image quality deteriorates due to the camera shake, and the reproduced image becomes very difficult to see.

For this reason, in recent years, a video camera has been commercialized, which is equipped with a shake correction device capable of correcting the above-mentioned camera shake, thereby correcting the camera shake during photographing.

In addition, with a video camera, panning (a method of shooting by swinging the camera left and right), tilting (a method of shooting by moving the camera from top to bottom or from bottom to top), etc. are sometimes performed at the time of shooting. Many.

Here, a conventional structure of the shake correction device mounted on the video camera will be described with reference to FIGS. Although there are various shake correction methods, an example using a so-called memory control method will be described here. When the shake is detected, the memory control method extracts a part of a video signal obtained by capturing a video with the CCD image sensor of the camera as an image frame, and according to the shake amount, the image frame of the previous field and the image of the current field. This is a method of correcting the shake by moving the frames so that they are aligned with each other, and matching both the image frames with each other. Further, here, as an example of a method for detecting the amount of shake, an angular velocity detection method is adopted. The angular velocity detection method is a method in which an angular velocity sensor such as a piezoelectric vibrating gyro is used to detect the angular velocity caused by the shake and obtain the amount of shake according to the detected angular velocity.

In FIG. 4, terminal 120 is supplied with angular velocity data from an angular velocity sensor. This angular velocity data is sent to the high pass filter 121. The high-pass filter 121 is a filter that mainly removes low-frequency components caused by panning and tilting of the video camera from the angular velocity data, and allows hand-shake components to pass through as they are.

The output data from the high-pass filter 121 is supplied to the multiplier 127, and the total gain adjuster 12 is supplied.
8 is multiplied by a predetermined multiplication coefficient from
After being multiplied by the multiplication coefficient according to the zoom magnification in the optical zoom in 9, the data is sent to the low-pass filter 154.
In the total gain adjuster 128, the gain of the correction signal obtained by the optical system of the video camera and the angular velocity sensor is not necessarily the design center value.
It is provided for the purpose of generating a multiplication coefficient for correcting the variation of the gain. In addition, the zoom gain table 130 stores a plurality of multiplication coefficients for gain correction according to the zoom magnification in the optical zoom of the video camera.
The multiplication coefficient corresponding to the current zoom magnification of the optical zoom is read from and is sent to the multiplier 129. The output data from the multiplier 129 is sent to the low pass filter 154.

The low-pass filter 154 integrates the data input from the multiplier 129 at the preceding stage using the integration coefficient from the integration coefficient table 136.

Here, the coefficients stored in the integration coefficient table 136 have a relationship with the integration value (LPF integration value) of the low-pass filter 154 as shown in FIG. 5, for example. An integration coefficient corresponding to the integrated value of the low-pass filter 154 is extracted from the integration coefficient table 136, and the low-pass filter 154 uses this integration coefficient to integrate the data supplied from the multiplier 129. The curve showing the relationship between the integration coefficient and the low-pass filter integration value in FIG. 5 shows both the horizontal (H) direction and the vertical (V) direction. Further, of the low-pass filter integrated values (LPF integrated values) in FIG. 5, the integrated value SH corresponds to the number of pixels of 1/2 of the horizontal surplus area of the CCD image sensor, and the integrated value SV is the CCD image sensor. Corresponds to 1/2 the number of pixels in the vertical surplus area. That is, as shown in FIG. 5, in the existing shake correction apparatus, the correction processing for camera shake and the convergence processing at the time of panning and tilting are performed using a common integral coefficient.

The output data of the low pass filter 154 comes to be output from the terminal 145 as a shake correction signal. The video camera performs shake correction processing for correcting the shake component of the image based on the shake correction signal.

[0012]

As described above, the conventional shake correction device is designed in consideration of all shake motions such as camera shake, panning and tilting in view of the use condition as a video camera.

That is, in image stabilization in a general video camera, the photographer performs hand-held still shooting while performing panning and tilting operations. Therefore, as the shake amount of panning and tilting increases, the integrated value becomes an integral value. It is necessary to have a large convergence performance in proportion. For this reason, the integration coefficient table for camera shake correction is as shown in FIG.

However, a video camera designed in consideration of all the shaking motions such as hand shaking, panning, and tilting motions is used, for example, for monitoring or vehicle-mounted (for vehicles, where the installation position and angle are fixed). When the video camera is used as fixed), panning and tilting operations are not performed as a usage condition of the video camera, but the convergence function for these is also included in the correction function. Therefore, the correction performance required for the fixed video camera is deteriorated. In other words, in other words, when the video camera using the integration coefficient table of FIG. 5 is used as a fixed monitor or a fixed camera mounted on a vehicle, there is no panning or tilting (the photographer intentionally shakes the camera) state. However, when the shake amplitude is near SV (or SH), the convergence performance becomes dominant and the correction capability cannot be fully utilized.

Therefore, the present invention has been made in view of such circumstances, and an object thereof is to provide a shake correction device and a video camera in which the correction performance does not deteriorate even when the device is fixed. And

[0016]

A shake correction apparatus of the present invention integrates a shake detection signal for detecting a shake of an image and a shake detection signal from the shake detection means to generate and output a shake correction signal. The above problem is solved by having the shake correction signal output means and the fixed integration coefficient supply means for supplying the shake correction signal output means with a fixed integration coefficient (for example, 0.999) having a sufficient correction capability.

Further, the video camera of the present invention comprises an image pickup means for generating an electric signal according to the light incident on the image pickup surface,
An optical system for forming an incident light image on the image pickup surface of the image pickup means, a video signal generation means for generating a video signal from an electric signal of the image pickup means, a shake detection means for detecting a shake of an image, and A shake correction signal output means for integrating a shake detection signal from the shake detection means to generate and output a shake correction signal, a fixed integration coefficient supply means for supplying a fixed integration coefficient to the shake correction signal output means, and the shake correction The above problem is solved by having a shake correction unit that performs a shake correction of an image according to a signal.

[0018]

According to the present invention, it is close to 1 (for example, 0.99).
9) Since the shake detection signal is integrated using a fixed integration coefficient, for example, the entire surplus area of the image pickup means can be used as the shake correction area.

[0019]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A preferred embodiment of the present invention will be described below with reference to the drawings.

The shake correction apparatus of this embodiment is used for monitoring or vehicle mounting (the video camera is fixed to the vehicle) with its installation position and angle fixed, and is not further fixed. It is installed in a video camera that can be used by hand. Further, there are various shake correction methods as described later, but here, an example using the memory control method will be described. The size of the CCD image sensor used in this embodiment is ± 48 pixels larger than the image frame in the vertical direction and ± 63 pixels larger in the horizontal direction than the image frame. ing. Therefore, the maximum shake correction amount is ± 48 pixels in the vertical direction and ± 63 pixels in the horizontal direction. Further, in the embodiment of the present invention, the angular velocity detection method is adopted as the method for detecting the amount of shake adopted in the shake detecting means for detecting shake of an image.

The configuration of a video camera equipped with the shake correction apparatus according to the embodiment of the present invention will be described with reference to FIG.

In FIG. 1, light from a subject or the like that has entered through the optical system 1 enters the CCD image sensor 2 and is converted into an electric signal by the CCD image sensor 2. The optical system 1 may, for example,
A lens system for forming an image on the D image sensor 2,
An optical zoom mechanism when moving the lens system for zooming and a drive system for the optical zoom mechanism, a focusing mechanism when moving the lens system for focusing, and a drive system for the focusing mechanism, an iris mechanism and the iris mechanism It consists of a drive system. Further, the CCD image sensor 2 in this case is composed of an optical filter and three CCD image sensors for receiving the respective colored lights.

The image pickup signal from the CCD image sensor 2 is sampled and held by an automatic gain control for automatically adjusting the signal gain in a signal adjusting circuit 3 and then digitalized by an analog / digital (A / D) converter 4. It is converted into an image pickup signal. This digital image pickup signal is sent to the camera signal processing circuit 5.

The camera signal processing circuit 5 digitally performs signal processing of the CCD color camera such as generation of a luminance (Y) signal and a chroma (C) signal from a digital image pickup signal. The camera signal obtained by the camera signal processing circuit 5 is output from the output terminal 6.

The camera control circuit 9 controls the drive of the optical zoom mechanism, the drive control for performing the autofocus control in the focusing mechanism, the drive control for performing the automatic iris control in the iris mechanism, and the entire system of the video camera. The control other than the shake correction processing described later, such as the control of the timing, is performed.

The switching input means 16 is composed of, for example, a button provided on the casing of the video camera, and when the video camera is fixed, shake correction is performed in the fixed state, or it is fixed by hand. This is for the user to switch and set whether or not to correct the camera shake or the like when there is no such error. The user operates this switching input means 16,
When instructing whether to perform correction processing such as fixing or holding, the camera control circuit 9 sends an instruction signal for the correction processing to the image control circuit 8.

Further, the angular velocity sensors 12 and 13 are both configured by a piezoelectric vibrating gyro, and the angular velocity detection surface of the angular velocity sensor 12 corresponds to the horizontal direction of the CCD image sensor 2, and the angular velocity detection surface of the angular velocity sensor 13 is the same. C
The video camera body is arranged so as to correspond to the vertical direction of the CD image sensor 2. That is, the angular velocity sensor 12 detects the angular velocity when the video camera body swings in the yawing direction, and the angular velocity sensor 13 detects the angular velocity when the video camera body swings in the pitching direction. These angular velocity sensors 12, 1
The angular velocity detection signal from 3 is converted into a digital signal by the A / D converter 11 and sent to the image control circuit 8 as angular velocity data.

The sync generator (SG) 14 generates a horizontal synchronizing signal HD and a vertical synchronizing signal VD and a field discriminating signal FP, and the horizontal synchronizing signal HD and the vertical synchronizing signal VD are sent to the timing generator (TG) 10 as a field. The discrimination signal FP is sent to the image control circuit 8.

The image control circuit 8 has a CC
The electronic zoom control for enlarging the image picked up by the D image sensor 2 is performed, and when the camera control circuit 9 supplies a signal for performing a shake correction process such as fixed or hand-held operation, the internal According to the shake correction processing program stored in
The shake correction amount calculation is performed based on the angular velocity data from the angular velocity sensors 12 and 13 via the converter 11. In addition,
The camera control circuit 9 controls the optical zoom by the optical system 1, which is different from the electronic zoom in the image control circuit 9. Further, when zooming is performed by the optical zoom, the correspondence between the shake angle when the video camera shakes and the shake amount on the CCD image sensor 2 changes depending on the zoom magnification. The zoom magnification information in the optical zoom is also sent to the image control circuit 8. Therefore, the image control circuit 8 is designed to take into account the zoom magnification information when calculating the shake correction amount.

The correction value obtained by the shake correction amount calculation in the image control circuit 8 is transferred to the timing generator 10 and the linear interpolation calculation circuit 7 as serial data.

Here, the image control circuit 8
As the serial data transferred from the linear interpolation calculation circuit 7 to the linear interpolation calculation circuit 7, the horizontal enlargement / reduction ratio HMAG and the vertical enlargement / reduction ratio value VMAG, the horizontal interpolation offset value HOFF, and the even field vertical interpolation offset are used. The value VOFFE and the vertical interpolation offset value VOFFO of the odd field, and the write end address value HSTOP to the line memory arranged in the linear interpolation calculation circuit 7.
And the write start address value HSTA to the line memory
RT, rising / falling phase value HCPS for line memory writing / reading control, falling phase value HCPE for line memory writing / reading control, writing start phase value MWBS for line memory, and ending writing phase value for line memory There are MWBE, the read start phase value MRBS of the line memory, and the read end phase value MRBE of the line memory.

Here, in the present embodiment, the camera shake correction processing in the vertical (V) direction and the camera shake correction processing in the horizontal (H) direction are performed as follows.

First, the vertical shake correction processing will be described.

Regarding the initial value setting of each field, in the correction processing of the integer part, the image control circuit 8
From the field read control signal FLD of the CCD image sensor 2 and the vertical sweep pixel value VTB to the image output from the CCD image sensor 2 to the timing generator 10 so that the CCD image sensor 2 can perform offset reading. Control. Further, in the correction processing of the decimal part, the signal of the vertical interpolation offset value VOFFE of the even field and the vertical interpolation offset value VOFFO of the odd field is sent from the image control circuit 8 to the linear interpolation calculation circuit 7 to control the interpolation processing. .

Further, regarding the successive interpolation processing of each line, the image control circuit 8 enlarges / extends in the vertical direction.
By sending the signal of the reduction ratio value VMAG to the linear interpolation calculation circuit 7 and, in synchronization with the horizontal synchronization signal HD, the VGAT signal indicating the presence or absence of the carry-out of the fractional addition section to the timing generator 10. Control.

Next, the horizontal shake correction process will be described.

Regarding the initial value setting of each line, in the correction processing of the integer part, the write start address value HSTART from the image control circuit 8 to the line memory arranged in the linear interpolation calculation circuit 7 and the write to the line memory. Control is performed by sending a signal of the end address value HSTOP to the linear interpolation calculation circuit 7. Further, in the correction processing of the decimal part, the signal of the interpolation offset value HOFF in the horizontal direction is sent from the image control circuit 8 to the linear interpolation calculation circuit 7 to control the interpolation processing.

Regarding the successive interpolation processing of each pixel,
The image control circuit 8 sends a horizontal enlargement / reduction ratio HMAG signal to the linear interpolation calculation circuit 7 for control.

Regarding the interpolation processing of the decimal part,
Since both the horizontal and vertical directions are performed by the linear interpolation calculation circuit 7, the image control circuit 8 outputs the rising phase value HCPS for writing / reading control of the line memory.
, The falling phase value HCPE for writing / reading control of the line memory, the writing start phase value MWBS of the line memory, and the writing end phase value M of the line memory.
WBE and read start phase value MRBS of line memory
Then, each signal of the read end phase value MRBE of the line memory is transferred to the linear interpolation calculation circuit 7.

Further, the timing generator 10 has a C
V-drive 1 that drives the CD image sensor 2
5, the frame shift operation control signals XV1 to XV4 corresponding to the camera shake correction amount are transferred, and the high-speed sweep operation control signal XSUB is transferred.

Next, a specific example of the configuration for generating the shake correction signal provided in the image control circuit 8 will be described with reference to FIG.

In FIG. 2, angular velocity data from the angular velocity sensors 12 and 13 converted into digital signals by the A / D converter 11 of FIG. 1 is supplied to the terminal 20 as a shake detection signal, and the angular velocity data is supplied. It is sent to the high pass filter 21.

The high-pass filter 21 serves as a filter that allows the shake component of the video camera, when the video camera is fixed, to pass from the supplied angular velocity data as it is and removes the DC component of the angular velocity data. ing. It should be noted that the high-pass filter 21 when the video camera is used by hand, etc. mainly removes low-frequency components due to panning and tilting, and allows hand-shake components other than panning and tilting to pass through as they are. .

The output data from the high-pass filter 21 is multiplied by a predetermined multiplication coefficient from the total gain adjuster 28 in the multiplier 27, and further in the multiplier 27 according to the zoom magnification in the optical zoom. Is multiplied and then sent to the adder 34 of the low-pass filter 54. The total gain adjuster 28 is used in the optical system 1
The gain of the correction signal obtained by each of the speed sensors 12 and 13 is not necessarily the design center value, and is provided for the purpose of generating the multiplication coefficient for correcting the variation in the gain. Further, the zoom gain table 30 stores a plurality of multiplication coefficient for gain correction according to the zoom magnification in the optical zoom, and the multiplication coefficient according to the current zoom magnification of the optical zoom is stored from the zoom gain table 30. Read out,
It is sent to the multiplier 29.

The low pass filter 54 includes an adder 34.
The register 37 and the multiplier 35 are included, and the data supplied from the multiplier 29 at the preceding stage is integrated using the integration coefficient supplied via the changeover switch 56. That is, the adder 34 includes the multiplier 29 of the preceding stage.
Is supplied as an addition signal, and the output data of the adder 34 is sent to the multiplier 35 via the register 37. The multiplier 35 includes a changeover switch 5
The integration coefficient via 6 is supplied to the register 3 above.
The output data of 7 is multiplied by the integration coefficient. The output data of the multiplier 35 is sent to the adder 34, and is added to the output data from the multiplier 29 of the previous stage in the adder 34. The output data of the adder 34 becomes a low-pass filter integration output, the integration output is output from the terminal 45 as a shake correction signal, and an integration coefficient table that stores the same integration coefficient as the integration coefficient table 136 of FIG. Also sent to 36.

The changeover switch 56 has a terminal 60.
One of the terminals to be switched a or b is selected in accordance with the switching control signal supplied via. Above terminal 6
The switching control signal supplied to the camera control circuit 9 is supplied to the camera control circuit 9 according to the switching input of the switching input means 16.
Is output from. Specifically, when the installation position and angle of the video camera are fixed and the user operates the switching input means 16 to switch and set shake correction in a fixed state, the switched terminal is switched. It is switched to the a side. Further, when the user operates the switching input means 16 when the video camera is not fixed because it is handheld or the like, when the user performs switching setting to correct the camera shake or the like, the terminal to be switched b is set. Can be switched.

A fixed correction coefficient (integration coefficient) as shown in FIG. 3 stored in the correction coefficient table 55 is supplied to the switched terminal a of the changeover switch 56.
Further, the switched terminal b is supplied with the integral coefficient as shown in FIG. 5 which is extracted from the integral coefficient table 36 according to the integral output of the low-pass filter 54.
That is, in this embodiment, the low-pass filter 54 when the video camera is not fixed because it is handheld, etc., the integration coefficient as shown in FIG. 5 that realizes correction for camera shake and convergence during panning or tilting. However, when the video camera is fixed, a fixed correction coefficient as shown in FIG. 3 is supplied to the low-pass filter 54 as an integration coefficient.

When the video camera is fixed, the fixed correction coefficient table shown in FIG. 3 stored in the correction coefficient table 55 for the low pass filter 54 is used as follows. It depends on the reason. That is, in a usage condition in which the video camera is fixed such as fixed monitoring or vehicle mounting, it is not necessary to take into consideration operations such as artificial panning and tilting, so (angular velocity data value 0) = ( The original position or angle) will be satisfied. Therefore, when the angular velocity integration is performed for one loop of the video camera stationary → vibration → stationary, theoretically it becomes 0, and it is not necessary to intentionally introduce a convergence coefficient into the integration coefficient. On the contrary, when the video camera is fixed as described above, if the integration coefficient including the convergence coefficient is used, the correction performance is deteriorated as described above. Therefore, the integration coefficient in FIG. 3 may be used. Will be needed.

From the above, according to the shake correcting apparatus and the video camera of the embodiment of the present invention, the shake can be effectively corrected when the video camera is fixed,
Furthermore, even when the video camera is not held by holding it, shakes such as camera shake can be effectively corrected. In addition, when used for fixed monitoring or in-vehicle, it is not necessary to take into consideration operations such as panning and tilting, so the linear correction area of the low-pass filter 54 is CC.
It is possible to expand to the maximum value of the surplus area of the D image sensor 2, the image blur correction performance can be maximized, and the dynamic range of the blur correction can be made larger than the conventional one.

In the above example, the memory control method is taken as an example of the shake correction method, but the present invention can also be applied to a method for correcting shake by an optical process. Is. As a method of correcting the shake by the optical processing, a gimbal mechanical method and an active prism method are known. In the gimbal mechanical system, when shake is detected, the entire lens unit is moved in a direction of canceling the shake by the optical system driving means to correct the shake. According to this method, since the entire lens unit is moved, the mechanism is large and power consumption is large, but there is no deterioration in resolution and the correction range is relatively wide, so high resolution can be obtained even if it is somewhat large. Suitable when you want. Further, in the active prism system, when a shake is detected, the shake is corrected by moving only a part of the lens unit in a direction of canceling the shake by the optical system driving means. According to this method, compared to the gimbal mechanism method, the power consumption is small, the size can be easily reduced, the resolution is not deteriorated, and the correction range can be set relatively wide. Are suitable.

Further, in the above-described embodiment of the present invention, the angular velocity detecting method is adopted as the shake detecting method adopted in the shake detecting means for detecting the shake of the image. It is also possible to use a vector detection scheme. The motion vector detection method detects the amount of movement and the direction of the subject by obtaining the difference between the image signals of the subject in the current field and the previous field stored in the semiconductor memory by image processing. According to this method, it is possible to reduce the size and cost because it can be configured only with an IC. When this motion vector detecting method is used, if the detected motion vector data is directly supplied to the low-pass filter 54 of FIG. 2, the same thing as the above-mentioned embodiment is established.

[0052]

As is apparent from the above description, according to the present invention, when the shake detection signal is integrated to generate the shake correction signal, 1 is set as the integration coefficient for integrating the shake detection signal.
Since a fixed integration coefficient close to (for example, 0.999) is used to integrate the shake detection signal, it is possible to use, for example, the entire surplus area of the image pickup means as the shake correction area, and thus the apparatus can be used. If is fixed, the correction performance does not deteriorate.

[Brief description of drawings]

FIG. 1 is a block circuit diagram showing a schematic configuration of a video camera incorporating a shake correction apparatus according to an embodiment of the present invention.

FIG. 2 is a block circuit diagram showing a configuration for generating a shake correction signal in the image control circuit of the video camera according to the embodiment of the present invention.

FIG. 3 is a diagram showing a relationship between a fixed integration coefficient (correction coefficient) and a low-pass filter integration value according to the present embodiment.

FIG. 4 is a block circuit diagram showing a configuration for generating a conventional shake correction signal.

FIG. 5 is a diagram showing a relationship between an integration coefficient and an integrated value of a low pass filter.

[Explanation of symbols]

5 Camera signal processing circuit 7 Linear interpolation calculation circuit 8 Image control circuit 9 Camera control circuit 10 Timing generator 12, 13 Angular velocity sensor 16 Switching input means 21 High-pass filter 36 Integration coefficient table 54 Low-pass filter 55 Correction coefficient table 56 Changeover switch

Claims (6)

(57) [Claims] 1. A shake detection unit for detecting a shake of an image, a shake correction signal output unit for integrating a shake detection signal from the shake detection unit to generate and output a shake correction signal, and the shake correction signal output. A shake correction apparatus comprising: a fixed integral coefficient supply means for supplying a fixed integral coefficient having sufficient correction capability to the means. 2. An integral coefficient output means for outputting an integral coefficient that changes according to the integral output of the shake detection signal of the shake correction signal output means, a fixed integral coefficient from the fixed integral coefficient supplying means, and the integral coefficient. 2. The shake according to claim 1, further comprising: an integral coefficient switching means for switching between the integral coefficient from the output means, and the shake correction signal output means is supplied with the integral coefficient switched by the integral coefficient switching means. Correction device. 3. An image pickup means for generating an electric signal according to light incident on the image pickup surface, an optical system for forming an incident light image on the image pickup surface of the image pickup means, and an image from the electric signal of the image pickup means. A video signal generating means for generating a signal, a shake detecting means for detecting a shake of an image, a shake correction signal output means for integrating a shake detection signal from the shake detecting means and generating and outputting a shake correction signal, A fixed integration coefficient supply means for supplying a fixed integration coefficient having sufficient correction capability to the shake correction signal output means, and a shake correction means for correcting a shake of an image in accordance with the shake correction signal are provided. Video camera. 4. An integral coefficient output means for outputting an integral coefficient that changes according to the integral output of the shake detection signal of the shake correction signal output means, a fixed integral coefficient from the fixed integral coefficient supplying means, and the integral coefficient. 4. The video according to claim 3, further comprising: an integral coefficient switching means for switching the integral coefficient from the output means, and the shake correction signal output means is supplied with the integral coefficient switched by the integral coefficient switching means. camera. 5. The shake correcting means extracts a part of the video signal as an image frame, moves the image frame of the previous field and the image frame of the current field so as to match each other according to the shake correction amount, and moves the image frame. 4. The video camera according to claim 3, wherein the shake is corrected by matching both image frames with each other. 6. The shake correction means has an optical system drive means for driving the optical system to shift a position of an incident light image formed on an image pickup surface of the image pickup means, and according to the shake correction amount. 4. The video camera according to claim 3, wherein the shake is corrected by shifting the position of the incident light image formed on the image pickup surface of the image pickup means.