JPH05110931A - Camera shake corrector - Google Patents

Abstract

PURPOSE:To write the contents of a first memory to a second memory as if the contents are shifted in a direction to cancel a motion vector so as to correct camera shake without lowering the resolution of an image pickup device. CONSTITUTION:A/D converted image signals from an imaging device 11 area stored in a frame memory 13. According to the image signal of a preceding frame from the frame memory 13 and the current image signal from the imaging device 11, a motion detecting circuit 14 detects the motion vector of the image, correlation is calculated between the current image signal and the image signal of the preceding frame, and the motion vector is calculated. The read address of the frame memory 13 and the write address of a frame memory 16 are made correspondent by a correcting address generator 15 just like the stored contents of the frame memory 13 is shifted in the direction to cancel the motion vector so as to write the stored contents of the frame memory 13 in the frame memory 16.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shake correction device, and more particularly to a camera shake correction device for correcting camera shake caused by so-called hand-held photography of a small video camera.

[0002]

2. Description of the Related Art Generally, when a small video camera is used for hand-held photography, or when ambient vibrations are transmitted to the microscope when observing a microscope image on a monitor, the output image signal is a so-called blur. It becomes difficult to see because it includes. Conventionally, as an image pickup apparatus for correcting this blur, there is one disclosed in, for example, Japanese Patent Laid-Open No. 61-198879.

FIG. 6 is a schematic block diagram of a conventional image pickup apparatus. In the figure, reference numeral 1 denotes an image pickup tube, and an output signal of the image pickup tube 1 is output to a frame memory 2 which stores the obtained image. Then, the motion detection circuit 3 detects the amount of motion from the image signal from the image pickup tube 1 and the image signal delayed by one frame from the frame memory 2. The sync signal generator 4 outputs the sync signal SY1 to the frame memory 2 and the sync signal SY2 to the image pickup tube 1 and the frame memory 2.

In the image pickup apparatus having such a configuration, the motion detection circuit 3 detects the amount of motion from the image of the current frame and the image one frame before, and further the frame memory 2
The range to be read is determined. The range read from the frame memory 2 will be described with reference to FIG.

In FIG. 7, the range indicated by reference numeral 5 is the entire screen of the entire frame memory and corresponds to the size of the image pickup surface of the image pickup tube 1. The synchronization signal SY2 output from the synchronization signal generator 4 of FIG. 6 is for scanning the entire screen 5 in the above range when an image is input from the image pickup tube 1. On the other hand, the synchronization signal SY1 is a signal for scanning an area narrower than the entire screen 5 (area indicated by reference numeral 6 in FIG. 7) and outputting an image signal. That is, when capturing an image, the entire screen 5 is scanned as shown in FIG. 7, and when outputting, a part of it is output. At that time,
The motion detection circuit 3 detects the amount of screen blur caused by camera shake or the like. Then, the range of the image output from the frame memory 2 is determined according to the direction and size.

In FIG. 7, when the apparent position of the subject moves due to camera shake, as shown by arrow A in the figure,
From the output range 6, the range 7 shown by the dotted line in the figure is determined. As a result, it is possible to obtain an output as if no blur occurs.

[0007]

However, according to such a conventional technique, the output image signal is a part of the range that can be picked up by the pick-up tube, and has a resolution higher than that in the case where the picked-up full screen is output. It was something that declined. That is, there is a problem that the resolution originally possessed by the image pickup tube cannot be effectively used.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image stabilization apparatus capable of preventing the deterioration of the resolution by effectively using the resolution originally possessed by the image pickup device. And

[0009]

That is, according to the present invention, an image pickup means for continuously picking up an image at a predetermined time interval, and image data for the entire image pickup screen picked up by the image pickup means are stored. No. 1 image storage means, the above-mentioned image data, and the motion detection means for detecting the motion of the screen by comparing the above-mentioned image data with the image data after being imaged after the above-mentioned predetermined time interval, Second image storage means for storing the corrected image data corrected so as to eliminate the movement of the screen, and the corrected image data corrected so as to eliminate the movement of the screen are stored in the second image storage means. As described above, according to the output signal from the motion detecting means, the read address for reading the previous image data from the first image storage means and the previous image data to the second image storage means are written. Included Characterized by comprising a correction address generating means for controlling the write address when.

[0010]

In the image stabilizer of the present invention, the A / D
Images that have been converted and are continuous at predetermined time intervals are captured by the image capturing unit, and the image data of the entire image captured by the image capturing unit is stored in the first image storage unit. In addition, the previous image data and the image data captured after the predetermined time interval of the previous image data are compared by the motion detecting means, and the motion of the screen is detected. The corrected image data corrected to eliminate the movement of the screen is stored in the second image storage means. And
According to the output signal from the motion detecting means, the correction address generating means may cause the corrected image data to be stored in the second image storing means so that the corrected image data corrected to eliminate the movement of the screen may be stored in the first image storing means. The read address for reading the previous image data from the image storage means and the write address for writing the previous image data in the second image storage means are controlled.

[0011]

Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a basic block diagram of an image stabilization apparatus according to the first embodiment of the present invention. In the figure, 11 is an image pickup device for forming an image of a subject by an optical system (not shown), and the image signal from the image pickup device 11 is A /
The digital signal is converted by the D converter 12. The frame memory 13 stores the image signal converted into the digital signal. From the image signal of one frame before stored in the frame memory 13 and the image signal of the current frame from the A / D converter 12, the motion vector of the image is detected by the motion detection circuit 14.

The motion vector obtained by the motion detection circuit 14 is sent to the correction address generator 15, where the addresses of the frame memories 13 and 16 at the time of writing the image signal from the frame memory 13 to the frame memory 16 are changed. Controlled. In addition,
The frame memory 16 is a frame memory in which the image signal read from the frame memory 13 is written according to the write address output from the correction address generator 15, and outputs the image signal after blur correction. Next, a process of correcting the blur by the combination of the motion detection circuit 14, the correction address generator 15, and the frame memories 13 and 16 will be described.

In the motion detection circuit 14, the current frame signal,
Correlation calculation is performed with the image one frame before stored in the frame memory 13, and a motion vector is obtained and sent to the correction address generator 15. Correction address generator
In 15, the frame memory is in the direction of canceling the above motion vector.
The read address of the frame memory 13 and the write address of the frame memory 16 are associated with each other so that the stored contents of 13 are written in the frame memory 16 as if they were moved in the direction of canceling the motion vector.

This process will be described by taking images stored in the frame memories 13 and 16 as an example. 2 (a)-(d),
6 is a diagram showing an example of an image stored in a frame memory. FIG. 2A shows an image stored in the frame memory 13, and FIG. 2B shows the image stored in the frame memory 16.
2 (c) shows the image of the next frame in FIG. 2 (a), and FIG. 2 (d) shows the image of the next frame in FIG. 2 (b).

Now, it is assumed that the motion vector from the previous frame of the image of FIG. 2A is 0. In that case, the data is read from the frame memory 13 as it is, the exact same contents are written in the frame memory 16, and the data shown in FIG. In addition, the image of FIG. 2C is an image in which the image of the previous frame (FIG. 2A) is shifted in the upper left direction due to camera shake. The direction and magnitude of this movement is detected by the movement detection circuit 14.

In order to correct this camera shake, when the image shown in FIG. 7C is read from the frame memory 13 and written in the frame memory 16, the image is shifted downward to the lower right as shown in FIG. To be crowded. The correction address generator correlates the read and write frame memory addresses.
Is 15. In this case, the upper part and the left part of the screen, that is, FIG.
Since the image data of the shaded portion in (d) will be missing, the data of the previous frame is left unchanged for this portion. As a result, the image in the frame memory 16 is output as it is. As described above, the image signal with the camera shake corrected can be obtained.

Next, a second embodiment of the present invention will be described with reference to FIG. In the following embodiments, the same parts as those in the first embodiment described above are designated by the same reference numerals, and the description thereof will be omitted to avoid duplication.

In FIG. 3, the image data (i-frame) formed on the image pickup device 11 is converted into a digital signal by the A / D converter 12, and then the motion detection circuit 14 and the frame memory 13 are used. Sent to. The motion detection circuit 14 has an A
At the same time as the signal sent from the / D converter 12, the image data of the (i-1) th frame stored in the frame memory 13 is sent. In the motion detection circuit 14, the i-th frame and i-
The correlation calculation of the image of the first frame is performed. Then, the motion vector is calculated and output to the motion vector adder 17.

On the other hand, the motion vector adder 17 sequentially cumulatively adds the motion vectors sent from the motion detection circuit 14 for each frame until the reset signal is sent from the reset circuit 18. Then, the motion vector of the image at the time of reset and the current image (hereinafter referred to as the cumulative motion vector) is obtained and sent to the address generator 19 and the reset circuit 18.
The address generator 19 outputs the read address of the frame memory 13 and the write address of the frame memory 16 in association with each other in order to move the image of the i-th frame so as to cancel the transmitted motion vector. As a result, the image of the i-th frame is shifted by a predetermined amount and the frame memory
Written on 16. At this time, the image data of the (i-1) th frame is left as it is in the frame memory 16 for the missing portion of the image caused by the shift.

Further, the reset circuit 18 is supplied with the cumulative motion vector calculated by the motion vector adder 17 and the vertical sync signal from a sync signal generation circuit (not shown). The reset circuit 18 outputs a reset signal when these inputs satisfy any of the following conditions. (1) When the magnitude of the cumulative motion vector exceeds a predetermined threshold. (2) When the direction of the cumulative motion vector does not change and its magnitude continues to increase immediately after the reset signal is output. (3) When the number of frames is counted by the vertical synchronization signal and the number of frames after the reset signal is output exceeds a predetermined threshold value. The reset circuit 18 does not perform camera shake correction for camera movements other than camera shake due to the above three conditions.

A motion vector adder is provided by the reset circuit 18.
When 17 is reset, the output cumulative motion vector is 0
Becomes The address generator 19 outputs the same address for reading and writing, and the contents of the frame memory 13 are written to the frame memory 16 as they are. That is,
No blur correction is performed.

As described above, when the camera is moving due to the pan or tilt operation, the blur correction is not performed,
When the camera shake occurs, the camera shake correction makes it possible to obtain a shake-free image signal. Next, C on the image sensor
A third embodiment using an MD (Charge Modulation Device) will be described with reference to FIG.

In the figure, the image stabilization apparatus according to the third embodiment has a CMD20, a frame memory 21 and a motion detection circuit 14 via the CMD20, and an image stabilization via the motion detection circuit 14. It comprises a decision circuit 22 and a correction address generator 23.

That is, the image data obtained by the CMD 20 is converted into a digital signal by the A / D converter 12 and sent to the motion detecting circuit 14. In the motion detection circuit 14, the image data already stored in the frame memory 21 and the A /
The motion vector is obtained from the image data from the D converter 12. The obtained motion vector is sent to the camera shake determination circuit 22.

In the camera shake determination circuit 22, the input motion vector is compared with a predetermined value to make a determination. If it is determined that the input motion vector is larger than a predetermined value, the vector of size 0 is output. If it is determined that the input motion vector is less than the predetermined value, the input motion vector is directly input to the correction address generator 23. Sent. In the correction address generator 23, the A / D-converted C is used in the direction of canceling the input vector.
The read address of the CDM 20 and the frame memory so that the image signal of the MD 20 is shifted and written in the frame memory 21.
Generates 21 write addresses. By the above series of operations,
In the frame memory 21, an image in which camera shake is corrected is always written.

By the way, in this embodiment, the image pickup device CMD
In 20, the reading must be performed twice within one frame period for motion detection and for writing to the frame memory 21. In addition, when writing to the frame memory 21, it is necessary to specify and read an address on the imaging surface.

Here, the CMD can be read out without destroying (i) pixel data. (ii) High-speed reading is possible. (iii) Random access of pixels is possible due to the XY address system. Therefore, it is suitable for use in such an embodiment.

Further, in the motion detection circuit 14 of the embodiment,
Since the motion between the images being output and the image being input is always detected instead of detecting the motion between the images of two consecutive frames, the accumulation described in the second embodiment is performed. You are outputting a vector. Therefore, the motion vector adder 17 used in the second embodiment is unnecessary.

As described above, in the third embodiment, the frame memory for one screen is sufficient and the motion vector adder is not required in comparison with the second embodiment, so that the shake correction can be performed with a simpler circuit configuration. The obtained image signal can be obtained. next,
Another example using a CMD as an image sensor will be described.

FIG. 5 shows the structure of an image stabilization apparatus according to the fourth embodiment of the present invention. In this embodiment, a detection address generator 24 for designating a range of motion detection on the screen is added to the configuration of the apparatus shown in FIG.

The detecting address generator 24 reads the image data in a preset range on the screen so that the CMD20
And specify the address of the frame memory 21. The motion detection circuit 14 performs a correlation calculation on the images in the above-mentioned set range in the CMD 20 and the frame memory 21 to obtain a motion vector. When the magnitude of the motion vector is equal to or smaller than a predetermined value, the camera shake determination circuit 22 determines that there is a camera shake and sends the motion vector to the correction address generator 23.
On the other hand, when the magnitude of the motion vector is equal to or larger than a predetermined value,
It is determined that the movement of the camera is other than camera shake, and a vector of size 0 is sent to the correction address generator 23.

In the correction address generator 23, the image from the CMD 20 is shifted from the input motion vector in the direction of canceling the motion vector and written in the frame memory 21 so as to be written in the frame memory 21.
And the corresponding write address of the frame memory 21 are designated. In this way, the frame memory 21 records the image data with the camera shake corrected. In the fourth embodiment, the circuit scale of the motion detecting circuit can be reduced by setting the range for motion detection by the detecting address generator to be small. The range of motion detection designated by the detection address generator may be set externally.

[0034]

As described above, according to the present invention, the image stabilization can effectively prevent the deterioration of the resolution by effectively using the resolution originally possessed by the image pickup device and obtain a high quality image output. A device can be provided.

[Brief description of drawings]

FIG. 1 is a basic block configuration diagram of an image stabilization apparatus according to a first embodiment of the present invention.

FIG. 2 is a diagram showing an example of an image stored in a frame memory of FIG.

FIG. 3 is a basic block configuration diagram of a camera shake correction device according to a second embodiment of the present invention.

FIG. 4 is a third embodiment of the present invention in which a CMD is used as an image sensor.
It is a basic block block diagram of the camera-shake correction apparatus using (Charge Modulation Device).

FIG. 5 is a diagram showing a configuration of an image stabilization apparatus according to a fourth embodiment of the present invention.

FIG. 6 is a schematic block diagram of a conventional imaging device.

7 is a diagram illustrating a range of a screen read from the frame memory of FIG.

[Explanation of symbols]

11 ... Image sensor, 12 ... A / D converter, 13, 16, 21 ... Frame memory, 14 ... Motion detection circuit, 15, 23 ... Correction address generator, 17 ... Motion vector adder, 18 ... Reset circuit, 19 …
Address generator, 20 ... CMD (Charge Modulation Devi
ce), 22 ... Shake detection circuit, 24 ... Address generator for detection.

Claims (4)

[Claims] 1. An image pickup unit for continuously picking up images at a predetermined time interval, a first image storage unit for storing image data of the entire image pickup screen imaged by the image pickup unit, and the above-mentioned unit. Image data and the image data after the image data captured after the predetermined time interval of the image data of the future are compared to detect the movement of the screen, and the movement of the screen is corrected to eliminate the movement. Second image storing means for storing the corrected image data, and the motion detecting means for storing the corrected image data corrected to eliminate the movement of the screen in the second image storing means. In accordance with the output signal, a read address for reading the previous image data from the first image storage means and a write address for writing the previous image data in the second image storage means are controlled. Camera shake correction apparatus characterized by comprising a positive address generating means. 2. A non-destructive readable image pickup means for continuously picking up images at predetermined time intervals and storing the latest image data over the entire picked-up image screen, the latest image data, The motion detection means for detecting the motion of the screen is compared with the corrected image data captured before the latest image data and corrected so that the motion disappears, and the motion is corrected to eliminate the motion of the screen. Image storage means for storing the corrected image data, and the latest image data stored in the image pickup means according to the output signal from the motion detection means so that the corrected image data is stored in the image storage means. A correction address generating means for controlling a read address for reading the image data and a write address for writing the latest image data in the image storage means. Characteristic image stabilization device. 3. The correction address generating means includes a motion vector adding means for adding the motion vectors detected by the motion detecting means, and an address for generating the read address and the write address according to the added motion vector. The image stabilization apparatus according to claim 1, further comprising: a generation unit and a reset unit that resets the motion vector addition in response to an input of a vertical synchronization signal. 4. The output signal of the motion detecting means is compared with a predetermined value, and when the value of the output signal is larger than the predetermined value, the value of the output signal is set to 0, and the value is larger than the predetermined value. The image blur correction device according to claim 2, further comprising an image blur determination unit that outputs the value of the output signal as it is when the value is small.