JPH0846856A - Hand shake correcting device - Google Patents

Abstract

PURPOSE:To provide a hand shake correcting device capable of performing the hand shake correction of an image without reducing the number of effective picture elements of an image pickup element by detecting the moving vector of a photographed image, recording the image photographed based on the moving vector on image memory by moving and synthesizing the image with image data stored in the image memory. CONSTITUTION:An analog video signal outputted sequentially from the image pickup part of a camera is converted to digital image data by an A/D converter 16, and the image data is stored in the image memory 18 capable of recording one screenful of image data. At this time, the moving quantity of the image due to hand shake is detected by angle sensors 26, 27 and a zoom (magnification), and the image data is recorded by moving the address of the image memory 18 based on the moving quantity. In this way, the image data is moved by the moving vector of the image by the hand shake, and it is stored in the image memory 18. The image data stored in the image memory 18 is converted to the analog video signal by a D/A converter 20.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image stabilizer,
In particular, the present invention relates to a camera shake correction device that forms a camera shake correction image by correcting an image obtained from a camera.

[0002]

2. Description of the Related Art Conventionally, an image stabilizer for electronically correcting an image is designed so that the size of the screen cut out from the image pickup surface of an image sensor is smaller than the input screen of the entire image pickup surface, and the direction and amount of movement caused by camera shake are adjusted. In addition, the image signal is read by changing the cut-out position of the screen, and then electronically zoomed to create an image signal equivalent to the image signal read from the input screen.

[0003]

However, according to this method, the entire image pickup surface of the image pickup device cannot be used while the image stabilizing device is operating, so that the number of effective pixels of the image pickup device is reduced. In addition, it is almost impossible to incorporate a device for preventing camera shake into a normal camera system with a major modification.

The present invention has been made in view of such circumstances, and it is possible to reduce the number of effective pixels of an image pickup device without decreasing the number of effective pixels.
An object of the present invention is to provide a camera shake correction device capable of electronically correcting camera shake.

[0005]

In order to achieve the above object, the present invention is an A / D for converting analog video signals sequentially output from an image pickup section of a camera into digital image data.
A converter, an image memory for storing image data for one screen output from the A / D converter, a D / A converter for converting image data read from the image memory into an analog video signal, Detecting means for detecting a movement vector of a subsequent image with respect to a reference image serving as an arbitrary reference; and detecting means for updating the stored contents of the image memory with image data sequentially output from the A / D converter. Based on the movement vector detected by
And a control means for shifting the address to be stored in the image data output from the / D converter or the address set in the image memory and storing the image data in the image memory. .

[0006]

According to the present invention, analog video signals sequentially output from a camera are converted into digital image data, and this image data is stored in an image memory capable of recording one screen of image data. The old image data recorded in the image memory is sequentially converted into an analog video signal.

Then, when the moving vector of the subsequent image with respect to the reference image is detected by the detecting means and the image data is stored in the image memory, the address to store the image data and the address to store the image data based on the detected moving vector. The address set in the image memory is shifted, and the relative shift amount of the address set in the image memory with respect to the address to store the image data is made equal to the detected movement amount.

[0008]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT A camera shake compensator according to the present invention will be described below with reference to the accompanying drawings.
A preferred embodiment of the positive device will be described in detail. Image stabilization
FIG. 3 is an explanatory diagram showing the principle of image formation. Around in time
Each of the two frame images₀, L₁And
L₀Is L ₀The base that becomes the reference for the images that will be taken afterwards.
Quasi-image and L₁Is L₀Is the image that follows.

In FIG. 1A, the image L ₁ following the reference image L ₀ in the frame shown by the solid line moves to the image in the frame shown by the dotted line in FIG. There is. At this time, if the image L ₁ is displayed on the screen as it is,
It is displayed like the image in the frame shown by the solid line in FIG. In addition, it is assumed that the image in the frame indicated by the solid line in FIGS. 9A, 9B, 9C and 9D is displayed on the screen.

Here, the horizontal movement amount of the subsequent image L _{1 with} respect to the reference image L ₀ due to camera shake is dm, and the vertical movement amount thereof is dn (hereinafter referred to as (dm, dn). Movement vector). At this time, the image L ₁
Is moved by the movement vector (dm, dn) to output the image L ₁ ′ in the solid line frame in FIG. However, if the image L ₁ 'is output to the screen as it is, the range shown by the solid diagonal lines in FIG. Therefore, the blank portion is supplemented with the corresponding image portion of the reference image L ₀ . That is, only the overlapping portion of the reference image L ₀ and the subsequent image L _{1 of} FIG. 9A is extracted from the image L _1, and only the overlapping portion of the reference image L ₀ is overwritten. .

The image synthesized in this way becomes an image L ₁ ″ shown by a solid line frame in FIG. 2D. This image L ₁ ″ is an image equal to the reference image L _0, and hand shake is caused. It can be seen that the movement of the image due to is corrected.
In the above description, when forming the image L ₁ ″, the image L
_Although the blank part of ₁ 'was supplemented by the reference image L ₀ ,
In reality, the subject itself may change while camera shake occurs, and the blank portion of the image L ₁ 'is the reference image L.
_If supplemented with an image of _0, an unnatural composite image may result. Accordingly, the image L ₁ 'blank part of the image L _1' to compensate the correction image before a single (reference image in the case of the photographed image image L ₁ is the next reference image L ₀ Corrected by L ₀ ).

FIG. 2 is a block diagram showing the internal structure of the image stabilizing apparatus according to the present invention. In the figure, the subject is imaged on the surface of a solid-state image sensor (CCD) 12 by the photographing optical system 10 and converted into an electric signal by the CCD 12. The electric signal output from the CCD 12 is the image pickup circuit 14
Is converted into a video signal. The video signal output from the image pickup circuit 14 goes through different processes depending on whether or not the image stabilization is performed. That is, when outputting the captured image as it is, the terminal 32 side of the selection switch 22 is connected to the terminal 36, and the video signal from the imaging circuit 14 is directly output to the output terminal 30. It should be noted that the camera shake correction mode for camera shake correction and the normal mode for outputting the captured image as they are are selected by operating the switch 28. If the switch 28 is operated in the OFF state (normal mode). In this case, the shake correction of the captured image is not performed, and the selection switch 22 is always connected to the terminal 32.

On the other hand, when camera shake correction is performed, the terminal 34 side of the selection switch 22 is connected to the terminal 36, and the video signal from the image pickup circuit 14 is output to the A / D converter 16 side. When the switch 28 is operated to the ON state and the camera shake correction mode is selected, the determination as to whether or not to perform the camera shake correction is made based on the image fluctuation state. The fluctuation state of this image is determined by the angular velocity sensors 26, 27.
The image indicated by the video signal output from the image pickup circuit 14 is horizontal with respect to the reference image based on the voltage signal proportional to the angular velocity output from the It is detected by calculating the amount of movement in the direction and the vertical direction (movement amount) (hereinafter, the direction and amount of movement of the image will be generically referred to as a movement vector). If the absolute value of the movement vector is equal to or larger than a certain value and fluctuates with a predetermined period, it is determined that camera shake occurs, and camera shake correction is performed.

The angular velocity sensors 26 and 27 are installed so as to detect the horizontal and vertical angular velocities, respectively, and output a voltage signal proportional to the horizontal and vertical angular velocities of the camera to the CPU 20. . In addition, the reference image is set at the start of shooting, and when it is determined that camera shake occurs and camera shake correction is performed, the movement vector of the subsequent image is calculated using the image shot in the shooting direction immediately before this determination as a reference. To do.

Next, the process of the video signal processing when the camera shake is corrected will be described. The video signal output from the image pickup circuit 14 is converted into digital image data by the A / D converter 16. Then, this image data is stored in a predetermined address of the image memory 18. However,
The address set in the image memory 18 changes based on the movement vector of the image, and the image data output from the A / D converter 16 is moved in the same direction as the movement due to camera shake by the movement amount. The image memory 18 is overwritten.

The image memory 18 has a memory of the same size as the number of pixels of the image and stores image data for one frame. The address of the image memory 18 is moved in the moving direction of the image by the moving amount. Let the A / D converter 16
The image data output from is recorded. As a result, the image data outside the address range set in the image memory 18 is removed, and the originally recorded image data remains at the address range where new image data was not recorded.

For example, the method of setting the address of the image memory will be described with reference to FIG. 1. First, it is assumed that the image L ₀ shown by the solid line frame in FIG. 1A is recorded on the image memory 18. At this time, assuming that the address of the image memory 18 corresponding to the point a in the upper left corner of FIG. 1A is 0, for example, when recording the subsequent image L ₁ , the point b in the upper left of FIG. 1A (the upper left corner of the dotted frame) ), The address of the image memory 18 corresponding to
To record as. As a result, the image L ₁ ′ is overwritten on the image memory 18 and an L ₁ ″ image is formed.

The image stabilization images thus formed on the image memory 18 are sequentially output to the D / A converter 20,
After being converted into an analog video signal, the selection switch 22
Is output from the output terminal 30 via. FIG. 3 is a flowchart showing a processing procedure of the CPU in the image stabilizing apparatus according to the present invention.

First, in step S10, initialization is performed. Here, the shooting direction currently taken by the television camera is set as the reference direction, and the image taken at this time is set as the reference image. In step S12, the angular velocities sequentially output from the angular velocity sensor 22 are integrated, and the shake amounts in the horizontal direction and the vertical direction from the reference direction are calculated (S12). Further, the angle of view is obtained from the position of the zoom lens 10A of the image pickup optical system 10 (S14). Then, the amount of movement (movement vector) (dm, dn) of the image in the horizontal and vertical directions with respect to the reference image is obtained from the shake amount and the angle of view of the television camera (S16).

In step S18, it is determined whether or not the movement vector periodically fluctuates with an amplitude of a certain value or more. That is, it is determined whether the movement vector is due to camera shake or panning or tilting motion. If it is determined in step S18 that the movement vector is not due to camera shake, the switch of the selection switch 22 is connected to the terminal 32 side and the video signal is output to the output terminal 30 as it is (S2).
0).

On the other hand, if it is determined in step S18 that the movement vector is due to camera shake, the switch of the selection switch 22 is connected to the terminal 34 side, and the video signal is output to the image memory 18 side (S24). Then, interrupt processing is performed (S26). When it is determined that the movement vector is due to camera shake, the movement vector (dm, dn) is calculated using the image captured immediately before this determination as the reference image.

Next, the interrupt processing shown in the flowchart of FIG. 4 will be described. First, the movement vector (d
The address of the image memory 18 is set based on m, dn) (S40). Then, in step S42, A
The image data output from the / D converter 16 is recorded in the image memory 18. That is, the movement vector (dm, d
By parallelly moving the address set in the image memory by the amount corresponding to (n), the image data output from the A / D converter 16 is apparently moved (dm, dn) to the previously output frame image. I am trying to overwrite it. Then, the corrected image recorded in the image memory 18 is output to the D / A converter 20 (S44).

As described above, the image blurring due to camera shake can be stopped by sequentially performing the processing described above. Next, another embodiment different from the above embodiment will be described. In the above embodiment, the movement vector of the image is detected by using the angular velocity sensor which is a mechanical sensor. However, the invention is not limited to this, and it may be detected by an image sensor.

The image sensor calculates the correlation between the preceding image and the succeeding image and detects the movement vector of the image based on this correlation. Since the correlation takes the maximum value when the preceding image and the succeeding image completely match, the camera shake can be corrected by moving the succeeding image and obtaining the movement vector when the correlation becomes maximum. However, when the image sensor is used for image stabilization, it is necessary to distinguish the image change caused by the camera shake from the image change caused by the panning operation, the tilting operation, and the change of the subject.
Therefore, a plurality of sections are provided on the screen, the correlation calculation is performed in each section, and the movement vector of the image in each section is calculated. Then, of the movement vectors of these images, the one with a small movement amount is selected (if the movement amount is large only for a section having a movement amount, it is considered to be due to a panning operation, a tilting operation, or a change in the object, and The movement amount of the section is omitted), and this movement vector is added (vector operation). This makes it possible to determine the movement vector of the image due to camera shake.

In the above embodiment, if the direction of the camera shake is limited to correct the image, the camera shake can be corrected more effectively. For example, the shake correction is performed only in the vertical direction during the panning operation, and the shake correction is performed only in the horizontal direction during the tilting operation.

[0026]

As described above, according to the image stabilization apparatus of the present invention, the movement vector moved by the image shake is detected, the image is moved based on this movement vector, and the screen generated by this movement is detected. Since the blank portion of is compensated by the preceding image, camera shake correction can be performed without reducing the number of effective pixels of the image sensor.

[Brief description of drawings]

FIG. 1 is an explanatory diagram showing the principle of camera shake correction.

FIG. 2 is a block diagram showing an internal configuration of a camera shake correction device according to the present invention.

FIG. 3 is a flowchart showing a processing procedure of the image stabilizing apparatus shown in FIG.

FIG. 4 is a flowchart showing the interrupt processing of FIG.

[Explanation of symbols]

 10 ... Shooting optical system 12 ... CCD 16 ... A / D converter 18 ... Image memory 20 ... D / A converter 22 ... Selection switch 24 ... CPU 26, 27 ... Angular velocity sensor

Claims (3)

[Claims] 1. A / A for converting analog video signals sequentially output from an image pickup section of a camera into digital image data.
A D converter, an image memory for storing one screen of image data output from the A / D converter, and a D / A converter for converting the image data read from the image memory into an analog video signal Detecting means for detecting a movement vector of a subsequent image with respect to a reference image serving as an arbitrary reference, and the detecting when updating the storage content of the image memory with image data sequentially output from the A / D converter. Based on the movement vector detected by the means, the address to store the image data output from the A / D converter or the address set in the image memory is shifted, and the image data is stored in the image memory. A camera shake correction device, comprising: 2. A camera shake according to claim 1, further comprising a selection switch for selecting and outputting one of a video signal from the image pickup section of the camera and a video signal from the D / A converter. Correction device. 3. A recognition unit for recognizing a camera shake when the absolute value of the movement vector detected by the detection unit is a certain value or more and has a predetermined cycle, and the camera shake is recognized by the recognition unit. Only when there is D /
3. The image stabilizing device according to claim 2, wherein the image signal from the A converter is selected by the selection switch and output.