JP3216820B2 - Image stabilization device - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shake correction apparatus, and more particularly to a camera shake correction apparatus for correcting a blur caused by so-called hand-held photographing of a small video camera.

[0002]

2. Description of the Related Art In general, when a small video camera is photographed by hand, or when an image of a microscope is observed on a monitor, ambient vibrations are transmitted to the microscope. And it is hard to see. 2. Description of the Related Art Conventionally, as an image pickup apparatus that performs this blur correction, for example, there is an image pickup apparatus disclosed in Japanese Patent Application Laid-Open No. 61-1988879.

FIG. 6 is a schematic block diagram of a conventional image pickup apparatus. In FIG. 1, reference numeral 1 denotes an image pickup tube, and an output signal of the image pickup tube 1 is output to a frame memory 2 for storing an obtained image. Then, the motion detection circuit 3 detects a motion amount from the image signal from the image pickup tube 1 and the image signal delayed by one frame from the frame memory 2. The synchronization signal generator 4 outputs the synchronization signal SY1 to the frame memory 2, and outputs the synchronization signal SY2 to the image pickup tube 1 and the frame memory 2.

[0004] In the image pickup apparatus constructed as described above, the motion detection circuit 3 detects the amount of motion from the current frame and the image one frame before, and furthermore, detects the motion amount.
Is determined. The range read from the frame memory 2 will be described with reference to FIG.

In FIG. 7, a range indicated by reference numeral 5 is the entire screen of the entire frame memory, and corresponds to the size of the imaging surface of the imaging tube 1. The synchronizing signal SY2 output from the synchronizing signal generator 4 in FIG. 6 is for scanning the entire screen 5 in the above range when an image is input from the imaging tube 1. On the other hand, the synchronization signal SY1 is a signal for outputting an image signal by scanning a range (a range indicated by a reference numeral 6 in FIG. 7) smaller than the entire full screen 5. That is, when capturing an image, the entire screen 5 is scanned as shown in FIG. 7, and when outputting, a part of the image is output. At that time,
The motion detection circuit 3 detects the amount of screen shake caused by hand shake or the like. Then, the range of the image output from the frame memory 2 is determined according to the direction and the size.

In FIG. 7, when the apparent position of a subject moves as shown by an arrow A in the figure due to camera shake,
The output range 6 is determined as a range 7 indicated by a dotted line in the figure. As a result, it is possible to obtain an output as if no blurring occurred.

[0007]

However, according to such prior art, the output image signal is a part of the range that can be picked up by the pick-up tube, and has a higher resolution than the case where the entire picked-up image is output. It was decreasing. That is, there is a problem that the resolution inherent in the image pickup tube cannot be used effectively.

SUMMARY OF THE INVENTION The present invention has been made in consideration of the above problems, and has as its object to provide a camera shake correction apparatus capable of effectively using the resolution inherent in an image sensor and preventing a reduction in resolution. And

[0009]

That is, according to the present invention, there is provided an image pickup means for continuously picking up images at a predetermined time interval, and a first means for storing image data over the entire image picked up image picked up by the image pickup means. Image storage means, and the first image
And image data stored in the image storage means, this images data
A motion detecting means for detecting the motion of the screen between the image data preceding in time, and removing the motion of the screen.
And stores the corrected image data.
A second image storage means to be output as data, in accordance with the output signal from the detecting means-out above kidou <br/>, read address when reading out the first image storing means or al picture image data When,
By controlling the write address when writing the upper Kiga image data into the second image storing means, the corrected image data
Correction address generating means for storing the data in the second image storage means . Also this invention is to capture images sequentially at predetermined time intervals, and the non-destructive readable imaging means for storing the latest image data over the entire imaged image pickup plane, the imaging means
Record image data over the entire imaged screen
The corrected image data from which the screen motion has been removed
The image storage means output as a
The latest image data stored and the image taken before the latest image data are stored in the image storage means.
By comparing the previous image data are a motion detection means for detecting a motion of the screen, in accordance with the output signal from the upper Symbol motion detecting means, a read address when reading the image data from the imaging means, by controlling the write address when writing the image data into the image storage means, the
And a correction address generation means for storing the correction image data in the image storage means .

[0010]

According to the camera shake correction apparatus of the present invention, the A / D
The converted and continuous images are taken at predetermined time intervals by the imaging means, and image data over the entire imaging screen taken by the imaging means is stored in the first image storage means. Further, the image data stored in the first image storing means, between the image data to be temporally precedes the images data of this, movement of the hand screen is detected in the motion detecting means. The corrected image data from which the screen motion has been removed is
The corrected image data is stored in the image storage
Output as image data. Then, in response to the output signal from the detecting means-out above kidou, from the first image storing means
A read address for reading image data and a second image
Write address for writing the image data to the storage means
Is controlled by the correction address generating means.
The corrected image data is stored in the second image storage means.
You. Also, in the camera shake correction apparatus of the present invention, the predetermined
A continuous image at the time interval is displayed on the entire captured screen.
Non-destructive readout of latest image data over the entire range
Therefore, the image is taken, and the image is taken by this imaging means.
Image data over the entire screen is stored in the image storage means,
The corrected image data from which the motion of the image has been removed is used as the image data.
Output. The latest image stored in the imaging means
Image data and images taken before this latest image data
And the previous image data stored in the image storage means.
Motion detection means for detecting the motion of the screen by comparing
According to an output signal from the motion detecting means.
A read address for reading image data from the image means;
Writing when writing the image data to the image storage means
Address is controlled by the correction address generation means.
The corrected image data is stored in the image storage means.
You.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a basic block diagram of a camera shake correction apparatus according to a first embodiment of the present invention. In FIG. 1, reference numeral 11 denotes an image pickup device for forming an image of a subject by an optical system (not shown), and an image signal from the image pickup device 11 is A / A.
The digital signal is converted by the D converter 12. The frame memory 13 stores the image signal converted into the digital signal. The motion vector of the image is detected by the motion detection circuit 14 from the image signal of the previous frame stored in the frame memory 13 and the image signal of the current frame from the A / D converter 12.

The motion vector obtained by the motion detection circuit 14 is sent to a correction address generator 15 where the addresses of the frame memories 13 and 16 when writing the image signal from the frame memory 13 to the frame memory 16 are stored. Controlled. In addition,
The frame memory 16 is a frame memory in which an image signal read from the frame memory 13 is written according to a write address output from the correction address generator 15, and outputs an image signal after blur correction. Next, a process of correcting a blur by a combination of the motion detection circuit 14, the correction address generator 15, and the frame memories 13 and 16 will be described.

In the motion detecting circuit 14, the current frame signal and
A correlation operation is performed with the image of the previous frame stored in the frame memory 13, a motion vector is obtained, and the motion vector is sent to the correction address generator 15. Correction address generator
In 15, the frame memory is set in the direction to cancel the above motion vector.
The read address of the frame memory 13 and the write address of the frame memory 16 are associated with each other so that the contents stored in the frame memory 13 are written in the frame memory 16 as if they were moved in the direction to cancel the motion vector.

This process will be described with reference to images stored in the frame memories 13 and 16. 2 (a) to 2 (d)
3 shows an example of an image stored in a frame memory. FIG. 2A shows an image stored in the frame memory 13, and FIG. 2B shows an image stored in the frame memory 16.
2 (c) shows the image of the next frame in FIG. 2 (a), and FIG. 2 (d) shows the image of the next frame in FIG. 2 (b).

Now, it is assumed that the motion vector from the previous frame of the image shown in FIG. In this case, the data is read from the frame memory 13 as it is, and exactly the same contents are written to the frame memory 16 as shown in FIG. Also, the image in FIG. 3C is an image in which the image of the previous frame (FIG. 2A) is shifted to the upper left due to camera shake. The direction and magnitude of this motion are detected by the motion detection circuit 14.

In order to correct this camera shake, when the image shown in FIG. 1C is read from the frame memory 13 and written into the frame memory 16, the image is shifted to the lower right as shown in FIG. Put in. The above-mentioned correction address generator associates the addresses of the read and write frame memories.
It is 15. In this case, the top and left sides of the screen,
Since the image data of the portion indicated by the oblique lines in (d) is lost, the data of the previous frame is left as it is in this portion. Thus, the image in the frame memory 16 is output as it is. As described above, an image signal corrected for camera shake is obtained.

Next, a second embodiment of the present invention will be described with reference to FIG. In the following embodiments, the same parts as those in the above-described first embodiment are denoted by the same reference numerals, and description thereof will be omitted to avoid duplication.

In FIG. 3, image data (i-frame) formed on the image sensor 11 is converted into a digital signal by an A / D converter 12, and then a motion detection circuit 14 and a frame memory 13 are provided. Sent to The motion detection circuit 14 includes A
The image data of the (i-1) th frame stored in the frame memory 13 is sent at the same time as the signal sent from the / D converter 12. In the motion detection circuit 14, the i-th frame and the i-th frame
The correlation calculation of the image of the first frame is performed. Then, a motion vector is calculated and output to the motion vector adder 17.

On the other hand, the motion vector adder 17 sequentially accumulates and adds the motion vectors sent for each frame from the motion detection circuit 14 until the reset signal is sent from the reset circuit 18. Then, a motion vector (hereinafter referred to as a cumulative motion vector) between the image at the time of reset and the current image is obtained, and is transmitted to the address generator 19 and the reset circuit 18.
The address generator 19 outputs the read address of the frame memory 13 and the write address of the frame memory 16 in association with each other so as to move the image of the i-th frame so as to cancel the transmitted motion vector. As a result, the image of the i-th frame is shifted by a predetermined amount and
Written to 16. At this time, the image data of the (i-1) th frame is left as it is in the frame memory 16 for the missing part of the image caused by the shift.

The reset circuit 18 receives the cumulative motion vector calculated by the motion vector adder 17 and a vertical synchronization signal from a synchronization signal generation circuit (not shown). The reset circuit 18 outputs a reset signal when these inputs satisfy any of the following conditions. (1) When the magnitude of the cumulative motion vector exceeds a predetermined threshold. (2) Immediately after outputting the reset signal, the direction of the accumulated motion vector does not change and its magnitude continues to increase. (3) When the number of frames is counted by the vertical synchronization signal and the number of frames after outputting the reset signal exceeds a predetermined threshold. The reset circuit 18 does not perform shake correction for camera movement other than camera shake based on the above three conditions.

A motion vector adder is provided by a reset circuit 18.
When 17 is reset, the output cumulative motion vector is 0
Becomes Then, the same address is output from the address generator 19 for reading and writing, and the contents of the frame memory 13 are written to the frame memory 16 as they are. That is,
No blur correction is performed.

As described above, when the camera is moving due to an operation such as panning or tilting, the blur correction is not performed.
When camera shake occurs, image shake-free image signals can be obtained by camera shake correction. Next, C
A third embodiment using an MD (Charge Modulation Device) will be described with reference to FIG.

Referring to FIG. 2, a camera shake correction apparatus according to a third embodiment includes a CMD 20, a frame memory 21 and a motion detection circuit 14 via the CMD 20, and a camera shake via the motion detection circuit 14. It comprises a decision circuit 22 and a correction address generator 23.

That is, the image data obtained by the CMD 20 is converted into a digital signal by the A / D converter 12 and sent to the motion detection circuit 14. In the motion detection circuit 14, the image data already stored in the frame memory 21 and the A /
From the image data from the D converter 12, a motion vector is obtained. The obtained motion vector is sent to the camera shake determination circuit 22.

In the camera shake determination circuit 22, the input motion vector is compared with a predetermined value to make a determination. If the input motion vector is determined to be larger than the predetermined value, the vector of size 0 is determined, and if the input motion vector is determined to be equal to or smaller than the predetermined value, the input motion vector is directly transmitted to the correction address generator 23. Sent. The correction address generator 23 converts the A / D-converted C into a direction for canceling the input vector.
The read address of the CDM 20 and the frame memory are shifted so that the image signal of the MD 20 is shifted and written into the frame memory 21.
Generates 21 write addresses. By the above series of operations,
An image whose camera shake has been corrected is always written in the frame memory 21.

In this embodiment, the image pickup device CMD
In step 20, two readings must be performed within one frame period, for detecting motion and for writing to the frame memory 21. In addition, when writing to the frame memory 21, an address on the imaging surface must be specified and read.

Here, the CMD can be read (i) without destroying the pixel data. (ii) High-speed reading is possible. (iii) Random access of pixels is possible because of the XY address system. It is suitable for use in such an embodiment.

In the motion detecting circuit 14 of the embodiment,
Instead of detecting the motion between two consecutive frames of images, the motion between the output image and the input image is always detected, so that the accumulation described in the second embodiment described above is performed. You are outputting a vector. Therefore, the motion vector adder 17 used in the second embodiment becomes unnecessary.

As described above, in the third embodiment, the frame memory is sufficient for one screen and the motion vector adder is not required as compared with the second embodiment, so that the camera shake can be corrected with a simpler circuit configuration. The obtained image signal can be obtained. next,
Another example using CMD as an image sensor will be described.

FIG. 5 shows the configuration of a camera shake correction apparatus according to a fourth embodiment of the present invention. In this embodiment, a detection address generator 24 for designating a range in which motion is detected in a screen is added to the configuration of the apparatus shown in FIG.

The detection address generator 24 controls the CMD 20 so as to read out a predetermined range of image data in the screen.
And the address of the frame memory 21 are designated. The motion detection circuit 14 performs a correlation operation on the images in the set range in the CMD 20 and the frame memory 21 to obtain a motion vector. When the magnitude of the motion vector is equal to or smaller than a predetermined value, the camera shake determination circuit 22 determines that the motion is a camera shake and sends the motion vector to the correction address generator 23.
On the other hand, when the magnitude of the motion vector is equal to or larger than a predetermined value,
It is determined that the movement is a camera movement other than camera shake, and a vector of size 0 is sent to the correction address generator 23.

The correction address generator 23 shifts the image from the CMD 20 in the direction of canceling the motion vector based on the input motion vector and writes the image into the frame memory 21.
And the corresponding write address of the frame memory 21 are designated. In this way, the frame memory 21 records the image data whose camera shake has been corrected. In the fourth embodiment, if the range in which motion is detected by the detection address generator is set to be small, the circuit size of the motion detection circuit can be reduced. The range in which the motion detection specified by the detection address generator is performed may be arbitrarily set from the outside.

[0034]

As described above, according to the present invention, it is possible to effectively use the resolution inherent in the image pickup device, to prevent the resolution from being lowered, and to obtain a high quality image output. An apparatus can be provided.

[Brief description of the drawings]

FIG. 1 is a basic block configuration diagram of a camera shake correction apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an example of an image stored in a frame memory of FIG. 1;

FIG. 3 is a basic block diagram of a camera shake correction apparatus according to a second embodiment of the present invention.

FIG. 4 is a diagram showing a CMD according to a third embodiment of the present invention;
FIG. 2 is a basic block configuration diagram of a camera shake correction device using (Charge Modulation Device).

FIG. 5 is a diagram showing a configuration of a camera shake correction device according to a fourth embodiment of the present invention.

FIG. 6 is a schematic block diagram of a conventional imaging device.

FIG. 7 is a diagram illustrating a range of a screen read from the frame memory of FIG. 6;

[Explanation of symbols]

11: image sensor, 12: A / D converter, 13, 16, 21: frame memory, 14: motion detection circuit, 15, 23: correction address generator, 17: motion vector adder, 18: reset circuit, 19 …
Address generator, 20 ... CMD (Charge Modulation Devi)
ce), 22: camera shake determination circuit, 24: address generator for detection.

──────────────────────────────────────────────────続 き Continued on the front page (58) Field surveyed (Int.Cl. ⁷ , DB name) H04N 5/222-5/257 G03B 17/00

Claims (6)

(57) [Claims] 1. A imaging means for capturing an image continuously at predetermined time intervals, a first image storage means for storing image data over the entire imaging screen captured by the imaging means, the first Image data stored in the image storage means ;
During than the images data of a temporally preceding image data
In a motion detection means for detecting a motion of the screen, and stores the corrected image data obtained by removing the motion of the screen, this
Of the second image storage means for correcting the image data is outputted as image data, in accordance with the output signal from the detecting means-out above kidou, when reading out the first image storing means or al picture image data by controlling the read address and the write address when writing the upper Kiga image data into the second image storing means, the correction-image
And a correction address generation means for storing image data in the second image storage means . 2. A non-destructively readable image capturing means for continuously capturing images at predetermined time intervals and storing the latest image data over the entire captured image capturing screen, and an image captured by the image capturing means. Image over the entire imaging screen
The image data is stored and the corrected image data
Image storage means for outputting the image data as image data, and the latest image data stored in the image pickup means.
Than the most recent image data captured previously, the image memory
By comparing the previous image data stored in the unit, a motion detection means for detecting a motion of the screen, in accordance with the output signal from the upper Symbol movement detector, when reading the image data from said image pickup means By controlling the read address of the image data and the write address when the image data is written to the image storage means, the corrected image data is updated.
And a correction address generating means to be stored in the image storage means . 3. The correction address generation means includes a motion vector addition means for adding a motion vector detected by the motion detection means, and an address for generating the read address and the write address according to the added motion vector. 2. The camera shake correction apparatus according to claim 1, further comprising: a generation unit; and a reset unit configured to reset the motion vector addition in response to an input of a vertical synchronization signal. 4. An output signal of the motion detecting means is compared with a predetermined value. If the value of the output signal is larger than a predetermined value, the value of the output signal is set to 0, and 3. The camera shake correction apparatus according to claim 2, further comprising a camera shake determination unit that outputs the value of the output signal as it is when the value is smaller. 5. The correction address generating means according to claim 2, wherein:
Of the temporally preceding image stored in the image storage means of
Of the image data, the output signal from the
Controlling only the address of the corresponding area
The camera shake correction device according to claim 1. 6. The correction address generating means according to claim 1 ,
The temporally preceding image data stored in the storage means
Data corresponding to the output signal from the motion detecting means.
Claims: Only the address of the area is controlled.
3. The camera shake correction device according to 2.