JPH07298127A - Image stabilizing device - Google Patents

Abstract

PURPOSE:To prevent deterioration in the resolution by segmenting an area of an image signal depending on hand-shake and implementing interpolation while using preceding image pickup data for a peripheral image. CONSTITUTION:An object image by a lens 19 is given to a process 11 as an image pickup signal from a CCD 10 being an image pickup element and processed for CDS, AGC, and gamma correction or the like and a video signal is outputted. The signal is converted into a digital signal by an A/D converter 12 and stored in an image memory 13 and converted into an analog signal by a D/A converter 14. Hand-shake sensors 15a, 15b in the X and Y directions generate an output in response to the acceleration by hand-shake and the output is converted into a proper signal at an IF section 16 to be sent to a system control section 17, a hand-shake quantity is obtained from the received acceleration signal in the X and Y directions through the arithmetic operation by a video signal correction means and data for image memory control in response to the hand-shake are outputted to a memory control section 18. Thus, read control of output data is implemented. Thus, the deterioration in the resolution is prevented.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image stabilizing device, and more particularly to a technique for improving the difficulty in viewing an image when a screen shake such as a camera shake occurs in a camera for obtaining an image pickup signal output from an image pickup device.

[0002]

2. Description of the Related Art As a camera shake correction in a conventional video camera or the like, a camera shake correction apparatus disclosed in Japanese Patent Laid-Open No. 4-88761 discloses a video signal from a result of detecting a camera shake amount in a vertical direction or a horizontal direction using an acceleration sensor. Was a device for changing the area taken out as an output from the memory circuit storing the, and expanding the taken out area.

Further, the shake correction apparatus for shake correction disclosed in Japanese Patent Laid-Open No. 2-250568 calculates a shake vector by calculating an image shake vector from a video signal photographed by a CCD which is an image pickup device, and shakes the shake. Is a method of enlarging a partial area of the video signal to obtain the image data of the output screen.

JP-A-4-88761 and JP-A-2-88761.
All of the conventional examples of 250568 are CCDs.
A part of the image pickup area is cut out, and the cutout part is enlarged and the output screen is enlarged while changing the cutout position according to the shake amount.

FIGS. 6 and 7 show images of the image data transfer state of the above-mentioned conventional image stabilization processing. FIG. 6 shows a state before the camera shake occurs (becomes a reference state), and the central cutout area 1a 'on the CCD area 1a.
Image data of the image memory 1
It is expanded to b and stored. On the other hand, FIG. 7 shows a state in which camera shake in the direction of the arrow has occurred. The image pickup data of the cut-out area 2a ′ at a position shifted by the camera shake amount on the CCD area 2a is transferred to the image memory 2b. And store it to obtain image data without shake.

[0006]

In the above-mentioned conventional example, as shown in FIGS. 6 and 7, it is necessary to cut out and enlarge a part of the image data picked up by the CCD. On the other hand, there is a problem that the resolution is lowered, and there is also a problem that the size becomes larger than the intended size of the subject, resulting in unsatisfactory performance.

The present invention has been made in order to solve the above-mentioned inconvenience, and even if the correction processing for camera shake is performed,
An object of the present invention is to provide an image stabilization device capable of preventing a reduction in the resolution of a screen imaged by an image sensor, obtaining a subject size that is the same as in the case where image stabilization is not performed, and providing a screen that is not unsightly. To do.

[0008]

One image stabilizing device of the present invention comprises an image sensor, a memory for storing a video signal, and a screen shake detecting means for detecting a screen shake and outputting a shake amount. A memory control means for reading out the video signal stored in the memory, and a video signal correction means for correcting the video signal from the image sensor by capturing the video signal before the video signal according to the output from the screen shake detecting means. To do.

In the image stabilizing device, the video signal from the image sensor is corrected in the partial area corresponding to the output from the screen shake detecting means by capturing the video signal before that.

Further, another image stabilizing device of the present invention is the image stabilizing device according to the above-mentioned one, in which the video signal from the image pickup device at the reference time is stored in a memory in advance for the correction of the video signal. After that, the new video signal from the image pickup device is stored in the memory while moving the writing area on the memory according to the output from the screen shake detecting means.

Still another image stabilizing device of the present invention is configured such that, in the other image stabilizing device, the memory can store image information in a region wider than one screen. However, the video signal from the image sensor at the reference time is stored in a predetermined area, and the output video signal can be output from an arbitrary area.

[0012]

Embodiments of the present invention will be described below with reference to the drawings. Prior to the detailed description of the above embodiment, the outline of the image stabilizing apparatus of the present invention will be described with reference to the data transfer state diagram in the image stabilization processing of FIGS. In the image stabilizing device of the present invention, the image signal from the CCD which is the image pickup device is converted into a digital signal, and the image pickup signal of the CCD is further stored in the image memory. Then, in a state in which there is no camera shake during the camera shake correction reference time, FIG.
As shown in, the video signal 3a picked up by the CCD is transferred to the image memory 3c as it is, and the area is the transfer area 3c.
It becomes equal to the CCD area 3a as shown in b.

Next, when a shake occurs, the shake amount is detected by the shake detecting means, and the screen input to the CCD image pickup area moves from the screen of the previous time in accordance with the shake amount. As shown in, the transfer area 4b is cut out from the imaged area 4a and transferred to the image memory 4c for the same screen range as the previous time. At this time, the transfer area 4b has a screen smaller than the CCD image pickup area by the shake amount, and the image pickup memory area 4e up to the previous time is added to the area 4d on the image memory 4c other than the newly transferred image data. The video data of the upper area 4f is left as it is as unmodified area data. And
The data of the image memory is always output as a video signal in a fixed area or without clipping the image.

As described above, in the present image stabilizing device, the newly rewritten area on the image memory and the area stored up to the previous time can be output as one screen, so that the cut-out CCD is cut out. It is not necessary to enlarge the image pickup area, and there is no change in resolution or change in size of the subject, and it is possible to capture a subject image without camera shake.

Next, the structure of an image pickup device which is an image stabilizing device according to an embodiment of the present invention will be described with reference to the block diagram of FIG. The subject image formed by the lens 19 is taken out as an image pickup signal from the CCD 10 which is an image pickup element and is input to the process processing 11. This process processing 11 outputs a video signal by performing CDS, AGC, gamma correction, etc. in the generally performed image pickup signal processing, but details thereof are omitted here. The video output from this process processing 11 is A / D
The data is converted into a digital signal by the conversion 12 and temporarily stored in the image memory 13, and the stored data is converted into an analog video signal by the D / A conversion 14 and output.

Further, an X-axis camera shake sensor 15a and a Y-axis camera shake sensor 1 as screen shake detecting means.
5b, an output corresponding to the acceleration due to camera shake is generated, and the output of each sensor is I.D. F. The signal is converted into an appropriate signal for transmission to the system control section 17 in the section 16, and is input to the system control section 17 having a built-in video signal correction means. In the system control unit 17, the amount of camera shake is calculated from the input acceleration signals in the X-axis and Y-axis directions, and the memory control unit 18 serving as a memory control unit receives control data for image memory control according to the amount of camera shake. Output. Memory control unit 18
Sets the address of the video data to be written in the image memory by this control data to control writing, and
It also controls the reading of output data.

Next, the operation sequence of the system control unit in the image pickup apparatus of this embodiment will be described with reference to the conceptual diagram of the storage area of the image memory in FIG. 3 and the flowcharts in FIGS. The above-mentioned FIG. 4 shows the main processing routine "MAIN" of the system control unit.
FIG. 5 shows a flowchart of “START”, and FIG. 5 is a subroutine for camera shake calculation processing “SUB S”.
The flowchart of "TART" is shown.

First, starting from MAIN START (step S20) in FIG. 4, V-SYN is performed in step S21.
C (vertical synchronizing signal), and after synchronizing, the X-axis. Sensor outputs from the Y-axis camera shake sensors 15a and 15b are captured as a camera shake signal via an A / D conversion input unit inside the system control unit 17.

Next, in step S24, the camera shake calculation processing sub-routine is called and the SUB STA of FIG.
RT processing is executed. After that, MAIN START
Returning to (step S20), the process is repeated.

Referring to the flow chart of the camera shake calculation processing subroutine of FIG. 5, the SUB STAR will be described.
The process starts from T (step S26), and then step S2
In the X-axis input processing of 7, the average shake amount in one vertical synchronization period in the horizontal direction is calculated. In the next Y-axis input process in step S28, the average shake amount in one vertical synchronization period in the vertical direction is calculated.

Then, in step S29, the memory control unit 18 creates control data for setting the address for writing the video data in the image memory 13 from the respective camera shake amounts. In step S30, the control data is transferred from the system control unit 17 to the memory control unit 18, and the present subroutine is terminated by the declaration in step S31.

The process of writing the image data to the image memory 13 at this time will be described. As shown in the conceptual diagram of the storage area of the image memory in FIG. 3, the memory area is temporarily m in the horizontal direction and n pixels in the vertical direction. If so, the address value changes from (0, 0) to (m-1, n11), but the video signal is present in the image area 40 of all the pixel areas by the previous imaging field or imaging frame. It is assumed to be filled with data. It should be noted that the previous time is the timing of the imaging field or the imaging frame before the time when the camera shake is detected, and it may be immediately before the occurrence of the camera shake. Further, the timing becomes a reference time for occurrence of camera shake.

Then, the current imaging field, or
If there is a camera shake amount indicated by an arrow S in the frame, if the vector is decomposed, the vertical direction moves downward by y pixels, and the horizontal direction moves left by x pixels. X pixels on the left and y on the bottom
The pixel region Rb is discarded as a region outside the image memory 40 at the reference time. In addition, new image data is recorded in the remaining area Ro. At this time, the image data up to the previous time is left as it is in the area Ra for the upper y pixels and the right x pixels of the image memory (40), and the image data of the area Ra and the image data of the area Ro is shaken. Image data of.

As described above, in the image pickup apparatus of this embodiment, X
A state in which the image output is not enlarged and the resolution is not changed only by recalculating the sensor output of the Y-axis and the Y-axis sensor as the amount of camera shake to the moving amount of the image memory and controlling the write address of the image memory as it is. Then, you can easily get the image with the image stabilization. In addition, compared to the case of cropping by enlarging the screen like the conventional image stabilization,
Since the screen can be cut out without limiting the cutout area, it is possible to perform correction in a wider range.

In the above embodiment, the camera shake detection is detected by using the acceleration sensor, but a device to which another camera shake detection method is applied can be proposed as a modification. For example, even if the camera shake detection method disclosed in Japanese Patent Laid-Open No. 3-3571, that is, the method of calculating the camera shake amount and the moving direction from the change of the image data to calculate the camera shake amount is applied to the camera shake detection. Good.

Further, even if the image memory is attached, the image pickup device C
Keeping a wider area against the CD, in accordance with the movement of the center of gravity of the camera shake, that is, the camera shake movement considering the rotation component,
A method of moving the output area can be easily considered. Further, in the above-described embodiment, the composition is performed with the data unmodified area on the memory, but the unmodified area from the previous data on the memory is cut out from the result of camera shake detection, and the signal from the image sensor is processed on the signal processing. It is also possible to synthesize and capture the video signal.

Needless to say, the effect of the apparatus of this embodiment is effective not only for camera shake but also for screen shake due to other factors.

(Appendix) Based on the embodiment described above, an image stabilizing device having the following configuration can be proposed.

(1) Means for detecting the amount of camera shake by using two acceleration sensors of the vertical axis and the horizontal axis and outputting the detection result, an image memory in which a video signal for one field can be written, and the image memory Means for writing a video signal,
The image memory has means for taking out and outputting a video signal, taking in a video signal obtained from a CCD at a reference time into the image memory, and from the reference time onward, the image signal is output according to the camera shake detection output. An image stabilizing apparatus, wherein a video signal is written in the image memory while moving a writing area on the image memory. According to this apparatus, since the writing area is moved according to the amount of camera shake, it is possible to perform camera shake correction without enlarging the image data.

(2) A means for detecting the amount of camera shake using the amount of motion vector obtained from the video signal and outputting the detection result, an image memory in which a video signal for one field can be written, and an image in the image memory. A means for writing a signal and a means for taking out and outputting a video signal from the image memory are provided, a video signal obtained from a CCD at a reference time is taken into the image memory, and after the reference time, the camera shake detection output The image stabilization device writes a video signal in the image memory while moving a writing area on the image memory according to the above. According to the present device, it is possible to provide a cost-effective image stabilizing device that does not require a sensor for detecting camera shake.

[0031]

The image stabilizing device according to the first and second aspects of the present invention cuts out the area of the video signal according to the camera shake and complements the photographing data before that as the peripheral image. Since it is created as a video signal, it is not necessary to enlarge the cut-out screen unlike the above-described conventional image stabilization, and the resolution is not deteriorated. Further, since the size of the subject intended by the photographer can be maintained, there is no sense of discomfort.

In the image stabilizing device according to the third aspect of the present invention, the video signal from the image pickup device at the reference time is stored in a predetermined area, and the output video signal can be output from an arbitrary area. However, it is possible to correct image data corresponding to a wide range of camera shake.

[Brief description of drawings]

FIG. 1 is a data transfer state diagram in a camera shake correction process in an image stabilizing apparatus according to the present invention. FIG. 1 (A) shows a state before a camera shake, and FIG. 1 (B) shows a data transfer after a camera shake. The figure which shows a state.

FIG. 2 is a block configuration diagram of an image pickup apparatus that is an image stabilizing apparatus according to an embodiment of the present invention.

3 is a conceptual diagram of a storage area of an image memory in the image pickup apparatus of FIG.

4 is a main processing routine "MAIN START" of the system control unit in the image pickup apparatus of FIG. 2;
Flow chart.

5 is a camera shake calculation processing subroutine "SUB STA" of the system control unit in the image pickup apparatus of FIG. 2;
RT "flow chart.

FIG. 6 is an image of an image data transfer state of a conventional image stabilization process, showing a state before the occurrence of an image blur.

FIG. 7 is an image of an image data transfer state of a conventional image stabilization process, showing a state after the occurrence of an image blur.

[Explanation of Codes] 10 ………… CCD (imaging device) 13 ………… Image memory (memory) 15a ………… Shake sensor in the X-axis direction (screen shake detection means) 15b ………… Y axis Direction shake sensor (screen shake detection means) 17 ……………… System control unit (video signal correction means) 18 ……………… Memory control unit (memory control means) S ……………… Shake amount (shake amount) )

Claims (3)

[Claims] 1. An image sensor, a memory for storing a video signal, a screen shake detecting means for detecting a screen shake and outputting a shake amount, a memory control means for reading out a video signal stored in the memory, and an image sensor. An image stabilizing device, comprising: a video signal correcting unit that captures and corrects a video signal from the image shake detecting unit according to an output from the screen shake detecting unit. 2. The image signal is corrected by pre-storing the image signal from the image sensor at the reference time in a memory and then moving the writing area on the memory according to the output from the screen shake detecting means. The image stabilizing device according to claim 1, wherein the image stabilizing device is performed by storing a new video signal from the image sensor in a memory. 3. The memory is configured so as to be able to store image information in an area wider than one screen, the video signal from the image sensor at the reference time is stored in a predetermined area, and the output video signal is 3. The image stabilizing device according to claim 2, wherein the image stabilizing device is capable of outputting from any area.