JPH0595502A - Blurring correction device - Google Patents

Abstract

PURPOSE:To realize the blurring correction function of the device performing the precise blurring correction through the interpolation processing by controlling the position and method for reading an imaging device. CONSTITUTION:The blurring correction device is provided with an imaging device 31 with an effective scanning area P1 and a correction area P2, a scanning area control circuit 35 enabling the movement of the effective scanning area P1, a motion vector detection circuit 35, and an interpolation circuit 34. By the blurring motion vector, the scanning area control circuit 35 moves the effective scanning area P1 and changes the combination of adjacent horizontal pixel train read out from the imaging device 31 in the horizontal scanning up and down. The interpolation circuit 34 performs the interpolation processing according to the motion vector, outputting a picture signal without blurring.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shake correction device for an image pickup device.

[0002]

2. Description of the Related Art In recent years, along with the widespread use of a camera-integrated VTR, its size, weight and zoom ratio have been increased.
Along with these improvements, image shake due to hand-held photography increases, so various image stabilization methods have been proposed.

A few examples of the conventional camera shake correction device used in an image pickup device such as a camera-integrated VTR will be briefly described. FIG. 6 is a block diagram showing a first example of a conventional image stabilization apparatus for an image pickup apparatus. In the figure, first, the image pickup device 1 converts an object in the image pickup field into electric signals pixel by pixel, and its output signal is given to the motion vector detection circuit 2 and the memory circuit 3. The motion vector detection circuit 2 processes the image information for each field or frame obtained from the image sensor 1 and determines whether the motion of the image in the screen is due to the motion of the subject itself or the motion due to camera shake. Then, in the case of movement due to camera shake, the movement amount is output as vector information.

The memory circuit 3 stores an image signal for each field. The memory control circuit 4 changes the read position of the image signal from the memory circuit 3 according to the vector information, cuts out a part of the image signal and reads it. Then, the signal processing circuit 5 performs signal processing such as a luminance signal and a color signal from the position-corrected image signal, and outputs the output to the electronic zoom circuit 6. The electronic zoom circuit 6 electrically enlarges the image, the position of which has been corrected by clipping, to the display area of the display screen and outputs the image. Image stabilization is performed by repeating this operation for each frame.

Next, FIG. 7 is a block diagram showing a second example of a conventional image stabilizing apparatus for an image pickup apparatus. This camera shake correction device does not perform position correction by electronic zoom, but performs camera shake correction by the method described below. In FIG. 7, the image sensor 11
Is, for example, a CCD image sensor in which a plurality of photoelectric conversion elements are two-dimensionally arranged, and the entire area thereof is set as an imaging area P0. The effective scanning area P1 is a display area for outputting an image in the imaging field of view as an image signal, and occupies most of the imaging area. The correction area P2 is a portion obtained by removing the effective scanning area P1 from the imaging area P0, and generally has a width that covers a motion vector amount due to camera shake. Motion vector detection circuit 12 and signal processing circuit 13
Respectively operate in the same manner as the motion vector detection circuit 2 and the signal processing circuit 5 shown in FIG. The output of the motion vector detection circuit 12 is given to the scanning area control circuit 14, and the image pickup device 11 is controlled by the drive circuit 15.

The operation will be described with reference to this figure. First, the image signal output from the image pickup device 11 is processed by the signal processing circuit 1.
At 3, the luminance signal and the color signal are processed. In addition, the image pickup device 11
Is also given to the motion vector detection circuit 12, and the motion vector detection circuit 12 detects a motion vector due to camera shake. The scanning area control circuit 14 positions the effective scanning area P1 within the imaging area P0 so that the motion vector becomes zero. The result is instructed to the drive circuit 15, and the effective scanning area P1 is moved by this instruction to move the image sensor 11
To drive. At this time, the image signal obtained from the effective scanning area P1 has been subjected to camera shake correction, and camera shake correction is performed by repeating this operation for each frame. Then, this image signal is given to a magnetic recording / reproducing unit (not shown) via the signal processing circuit 13.

Further, FIG. 8 is a block diagram showing a third example of a conventional image stabilizing apparatus used in an image pickup apparatus. In the figure, the image pickup device 21, the drive circuit 22, the motion vector detection circuit 23, and the signal processing circuit 26 have the same operations as the image pickup device 11, the drive circuit 14, the motion vector detection circuit 12, and the signal processing circuit 13 shown in FIG. 7, respectively. Is to do. Here, the memory circuit 24 temporarily holds the video signal of the image pickup device 21. The memory control circuit 25 moves the effective scanning area P1 according to the vector information detected by the motion vector detection circuit 23, and outputs the position-corrected image signal from the memory circuit 24.

The operation will be described with reference to this drawing. First, the image signal output from the image pickup device 21 is stored in the memory circuit 24.
To the motion vector detection circuit 23. Memory circuit 24
Holds an image signal of one field, and the motion vector detection circuit 23 detects a motion vector due to camera shake. The memory control circuit 25 moves the effective scanning area P1 from the imaging area P0 held in the memory circuit 24 in the direction in which camera shake can be corrected by the detected motion vector amount, and reads the image signal. Then, the position-corrected image signal is given to the signal processing circuit 26 to perform signal processing, and this operation is repeated for each frame to perform camera shake correction.
Then, this image signal is given to a magnetic recording / reproducing unit (not shown), and an image for which image stabilization has been performed is recorded.

[0009]

However, such a conventional structure has the following problems. That is, in the first conventional example, the image signal position-corrected by the electronic zoom circuit 6 is expanded to the display area. Therefore, the pixels of the image signal obtained by the image pickup device 1 become rough, and there is a problem that the quality of the corrected image deteriorates significantly.

Further, in the second and third conventional examples, the position is corrected by moving the effective scanning area P1 within the image pickup area P0 in order to correct the camera shake. However, the position correction amount is an integral multiple of the horizontal scanning line interval of the imaging region P0, and the vertical reading accuracy cannot be kept within one horizontal scanning accuracy. For this reason, it was not possible to correct the camera shake accurately.

The present invention has been made in view of the above-mentioned conventional problems, and it is possible to secure an accuracy of reading in the vertical direction of one line or less in the horizontal scanning, and to have stable image quality with little deterioration in image quality. An object of the present invention is to provide a camera shake correction device for an imaging device.

[0012]

The present invention converts an image of a subject into an image signal by interlaced scanning, and
An image sensor that outputs an image signal from a movable effective scanning area within an image capturing area of an image, a motion vector detection circuit that detects an image motion vector due to camera shake of an image capturing apparatus including the image sensor, and a motion vector detection circuit The effective scanning area is moved in the vertical direction by the integer part of the vertical motion vector in units of horizontal scanning lines, and the combination of the adjacent horizontal pixel rows read out simultaneously from the image sensor by the value of the decimal part of the motion vector is set. A scanning area control circuit that drives the image sensor by switching up and down, and an interpolation processing circuit that performs interpolation processing of the image signal with an interpolation coefficient according to the value of the fractional part of the vertical motion vector detected by the motion vector detection circuit. , Is provided.

[0013]

According to the present invention having such a feature, the effective scanning area is moved by the integer part of the motion vector detected by the motion vector detection circuit, and the camera shake is corrected with the accuracy of one line. Further, depending on the value of the fractional part of the vertical motion vector detected by the motion vector detection circuit, the combination of adjacent horizontal pixel rows obtained by simultaneous reading in horizontal scanning is switched up and down, and the interpolation circuit detects the motion vector detection circuit. The interpolation process is performed with the interpolation coefficient according to the value of the decimal part of the vertical motion vector.

[0014]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS An embodiment of the present invention will be described below with reference to the drawings. FIG. 1 is a block diagram showing the arrangement of a camera shake correction device for an image pickup apparatus according to this embodiment. It should be noted that the same parts as those in FIGS. 6 to 8 showing the conventional example are given the same names and detailed description thereof will be omitted. In FIG. 1, the image pickup device 31 is a solid-state image pickup device in which a plurality of photoelectric conversion elements are two-dimensionally arranged, such as a CCD image sensor. Image sensor 3
Reference numeral 1 denotes an imaging area P0 that covers the entire area of a subject that can be imaged via an optical system, an effective scanning area P1 that gives an image in the imaging field of view to a display device, and a correction area obtained by removing the effective scanning area P1 from the imaging area P0. And P2. The correction area P2 is an area in which the effective scanning area P1 can be moved within the imaging visual field, and generally has a width that covers a motion vector amount due to camera shake. The output of the image sensor 31 is given to the motion vector detection circuit 32 and the signal processing circuit 33.

The motion vector detection circuit 32 sets a plurality of cutout frames in the image pickup area P0 of the image pickup device 31, for example,
The motion vector of the image obtained from the representative point in each cutout frame is detected for each field to detect the motion vector.
The motion vector detection circuit 32 processes the image information for each field by a known method to determine whether the motion of the image on the screen is the motion of the subject itself or the motion of the camera shake. Then, in the case of movement due to camera shake, the movement amount is output as vector information. This motion vector has horizontal and vertical components with respect to the imaging region P0, but horizontal position correction can be easily performed by delaying the scanned image signal using a delay circuit or the like. In the present embodiment, a detection circuit for performing vertical position correction, which is said to be difficult to perform highly accurate correction, is used.

The signal processing circuit 33 is a circuit for generating a luminance signal, a color signal, etc. from the output of the image pickup device 31, and the output thereof is given to an interpolation circuit 34 described later. The scanning area control circuit 35 is a circuit for driving the image pickup device 31, and controls the effective scanning area P1 so as to be movable within the image pickup area P0. The control unit 36 calculates from the vector information obtained from the motion vector detection circuit 32 how many lines the component in the vertical scanning direction is with the pitch of the scanning line as a basic unit up to a decimal digit, and controls based on the result. A signal is generated, and its output is given to the scanning area control circuit 35 and the interpolation count generation circuit 37.

The interpolation circuit 34 performs interpolation processing on the output signal from the signal processing circuit 33, and multiplies an image signal pair of adjacent scan lines in an odd field or an even field by an interpolation weight coefficient (hereinafter referred to as an interpolation coefficient). It is a circuit that adds and adds. The interpolation coefficient generation circuit 37 uses an interpolation coefficient w for two adjacent scan line signals in each field.
It is a circuit that calculates (1-w) based on a command from the control unit 36, and its output is given to the interpolation circuit 34. Here, the interpolation circuit 34 and the interpolation count generation circuit 37 constitute an interpolation processing circuit for performing the interpolation processing of the image signal with the interpolation coefficient according to the value of the fractional part of the vertical direction motion vector detected by the motion vector detection circuit 32. I shall.

The operation of the image stabilizing apparatus of the present embodiment having the above-mentioned structure will be described with reference to FIGS.
FIG. 2 is a schematic diagram illustrating a method of reading a signal from the image sensor 31. In FIG. 2, the photoelectric conversion elements of the image pickup element 31 are two-dimensionally arranged, and the line C includes the elements Ck, 1
, Ck, 2, ..., Ck, m, and line n has elements Cn, 1,
Cn, 2, ... Cn, m, the next line k + 1 has an element Ck +
.1, Ck + 1,2, ... Ck + 1, m, and elements n + 1,1, Cn + 1,2 ,. And

The reading of the normal image pickup device 31 has two modes. That is, as shown in the pattern A, the outputs of the elements Ck, 1, Ck, 2, ..., Ck, m and the elements Cn, 1,
Each output of Cn, 2, ..., Cn, m is added, and elements Ck + 1,1,
Each output of Ck + 1,2, ..., Ck + 1, m and elements Cn + 1,1, C
There is a method of adding the respective outputs of n + 1,1, ..., Cn + 1, m and taking out the average of the addition outputs. Further, as shown in the pattern B, each output of the elements Cn, 1, Cn, 2, ..., Cn, m and the element Ck
There is a method in which the outputs of +1,1, Ck + 1,2, ..., Ck + 1, m are added, the outputs of the following two lines are also added, and the average of the added outputs is taken out. As described above, in the pattern A and pattern B reading methods, a combination of two horizontal pixel columns adjacent to each other in one horizontal scanning from the image pickup device 31 is vertically shifted by one pixel and simultaneously read.

The signal from the image pickup device 31 is output as an image signal by the signal processing circuit 33. Further, the motion vector detection circuit 32 detects a motion vector due to camera shake,
This motion vector information is given to the control unit 36. Control unit 3
Reference numeral 6 gives an instruction to the scanning area control circuit 35, and first, according to the size of the integer part of the detected vertical direction motion vector amount, the camera shake correction is first performed with the accuracy of one line unit. The scanning area control circuit 35 moves the effective scanning area P1 upward or downward by a predetermined line in the vertical scanning direction. Further, the control unit 36 switches the combination of two adjacent horizontal pixel rows read simultaneously in one horizontal scan up and down according to the value of the fractional part of the vector amount detected by the motion vector detection circuit 32, and drives the image sensor 31. To do. The control unit 36 sends a signal of the fractional part to the interpolation coefficient generating circuit 37 in order to perform the interpolation process for correcting the shake of less than one line by the fractional part of the motion vector detected at the same time. In the interpolation circuit 34, the signal processing circuit 33
The interpolation processing for correcting the shake of less than one line is performed on the output of. This operation is repeated for each frame to output a shake-corrected image signal.

Here, the image pickup device 31 according to the motion vector.
The operation principle of the scanning area control circuit 35 for driving the will be described in detail. FIG. 3 is a schematic diagram showing the configuration of the image pickup area P0 by the reading of the image pickup device 31 in the present embodiment,
An arrow V represents a motion vector due to camera shake. Here, depending on the presence / absence of the fractional part of the motion vector amount, FIG.
Two types of position correction shown in (a) and (b) are executed.

(1) When no camera shake occurs, the motion vector amount detected by the motion vector detection circuit 32 is zero. The pattern A and the pattern B shown in FIG. 2 are alternately read in the field cycle without moving the effective scanning region P1 from the previous field, and the result is directly output through the signal processing circuit 33 and the interpolation circuit 34.

(2) When camera shake occurs and the fractional part of the vertical motion vector amount is zero, the scanning area control circuit 35 moves the effective scanning area P1 by an integral multiple of the horizontal scanning line. In this case, since the fractional part of the vertical direction motion vector is zero, the boundary between the scanning line of the imaging region P0 and the effective scanning region P1 coincides with each other as shown in FIG. Therefore, after the effective scanning area P1 is moved, the pattern A and the pattern B are alternately read in the field cycle as in the normal reading. That is, in FIG. 3A, if the reading method in the previous field is pattern A, the reading start point of the current field is position PB, and the reading method is pattern B. If the reading method in the previous field is pattern B, the reading start point of the current field is position PA and the reading method is pattern A.

(3) When camera shake occurs and the fractional part of the vertical motion vector is not zero, the effective scanning area P1 is moved in accordance with the detected motion vector amount, and in the moved effective scanning area P1. Reading is performed in a pattern different from that of the previous field. Since the decimal part is not zero, as shown in FIG.
The boundary of 1 and the scanning line of the imaging region P0 do not match. For this reason, the scanning line is moved to a position represented by the fractional part of the vertical motion vector, and interpolation is performed by the following calculation.

FIG. 4 is a schematic diagram showing a method of vertically switching the combination of two adjacent horizontal pixel columns when the image signals of the image pickup device 31 are simultaneously read out by one horizontal scan. That is, the position of the first scanning line of the effective scanning area to be obtained by interpolation is included in any of the areas a, b, c, and d of FIG. 4 according to the value of the fractional part of the vertical motion vector. The read pattern and the read start position are controlled according to which region the first scan line is included in. When the first scanning line to be obtained by interpolation is included in the region a or d, the reading is performed with the pattern A, and the image signal is read from the k line in the drawing. Further, when the first scanning line to be obtained by interpolation is included in the area b or c, the reading is set to the pattern B, and when it is included in the area b, the reading of the image signal is started from the n−1 line, and the area c is read. , The reading of the image signal is started from the n-th line. Then, the interpolation coefficient of the image signal to be interpolated is determined based on the distance between the read scanning line and the interpolation line.

When the reading line and the reading pattern are switched in this way to perform the reading, the interpolation circuit 34 outputs 2
When interpolation is performed from one input line, the interpolation position is always located within 25% of the line interval from either line, and the image signal on the side closer in the vertical scanning direction can be used for interpolation. Therefore, unlike the first conventional example, the image signal is not interpolated by the electronic zoom circuit in the vicinity of the center of the two input lines, and the vertical frequency response characteristic of the interpolated line is not deteriorated and good image quality is obtained. . In addition, 1
Since the camera shake less than the line is also corrected, it is possible to perform the stable camera shake correction with little image quality deterioration.

Although the motion vector detection circuit 32 obtains the motion vector information from the image pickup device 31 in FIG. 1, another camera shake motion vector detector for detecting the vibration of the image pickup device itself is provided, and the motion vector detection circuit 32 detects the vector. You may input the amount. In addition, the signal processing circuit 33 is replaced by the interpolation circuit 34.
The signal processing circuit 33 may be provided after the interpolation circuit 34 as shown in FIG. 5 to perform the luminance signal processing and the color signal processing on the corrected image signal as shown in FIG. Therefore, the description of the operation of each circuit is the same as that described above except that the order thereof is different, and therefore will be omitted. Further, the image pickup device 31 is, for example, a CCD image sensor, but it is needless to say that the present invention can be applied as long as the image pickup device 31 can be switched to the above-mentioned two scanning methods by using any image pickup device.

[0028]

As described above, according to the present invention, the effective scanning area is moved so as to correct the camera shake with the accuracy of one line unit by the integer part of the motion vector detected by the motion vector detection circuit. .. Further, depending on the value of the fractional part of the vertical motion vector detected by the motion vector detection circuit, the combination of adjacent horizontal pixel columns read simultaneously from the image sensor in one horizontal scan is switched up and down, and the interpolation circuit causes the motion vector to move. The interpolation processing is performed by the interpolation coefficient according to the value of the decimal part of the vertical motion vector detected by the detection circuit. Therefore, it is possible to secure the vertical reading accuracy of 1 line or less in the horizontal scanning, and it is possible to realize the image stabilization apparatus of the image pickup apparatus having stable image quality with little image quality deterioration.

[Brief description of drawings]

FIG. 1 is a block diagram showing the configuration of a camera shake correction device according to an embodiment of the present invention.

FIG. 2 is a schematic diagram showing a method of reading a signal from an image sensor according to the present embodiment.

FIG. 3 is an explanatory diagram showing a configuration of an image pickup area in reading according to the present embodiment.

FIG. 4 is a schematic diagram illustrating a method of vertically switching the combination of horizontal pixel rows that are simultaneously read by horizontal scanning in the present embodiment.

FIG. 5 is a block diagram showing a configuration of an image stabilization apparatus according to another embodiment of the present invention.

FIG. 6 is a block diagram showing a configuration of a first conventional camera shake correction device.

FIG. 7 is a block diagram showing a configuration of a second conventional camera shake correction device.

FIG. 8 is a block diagram showing a configuration of a third conventional camera shake correction device.

[Explanation of symbols]

 31 image sensor 32 motion vector detection circuit 33 signal processing circuit 34 interpolation circuit 35 scanning area control circuit 36 control unit 37 interpolation coefficient generating circuit P0 imaging area P1 effective scanning area P2 correction area

Claims (1)

[Claims] 1. An image sensor for converting an image of a subject into an image signal by interlaced scanning and outputting an image signal from an effective scanning area that is movable within an image capturing area of the image, and an image capturing apparatus including the image sensor. A motion vector detection circuit that detects an image motion vector due to camera shake, and the effective scanning area is moved up and down by an integer part in units of horizontal scanning lines of the motion vector in the vertical direction detected by the motion vector detection circuit. A scan area control circuit for driving the image sensor by vertically switching the combination of adjacent horizontal pixel rows read simultaneously from the image sensor according to the value of the fractional part of the motion vector, and the vertical direction detected by the motion vector detection circuit. An interpolation processing circuit for performing an interpolation process of an image signal with an interpolation coefficient according to the value of the fractional part of the motion vector in the direction; A camera shake correction device comprising: