JPH07177417A - Image blur preventing device - Google Patents

Abstract

PURPOSE:To minimize image degradation after correcting an image blur by setting a correcting step at the time of correcting the image blur corresponding to the degree of the image blur to an optimum state. CONSTITUTION:Output signal (the respective components in a horizontal direction and a vertical direction of a motion vector at a prescribed screen position) from a motion vector detection circuit 15 are read for respective fields. Then, prescribed processings are performed to the read motion vectors at the plural screen positions and a certain representative motion vector is computed. The prescribed processings are the reliability evaluation processing of the respective motion vector and a controlled system area deciding processing, etc. The representative motion vector is integrated, a deviation from the reference position of a screen is obtained and image blur correction signal are obtained. Then, the correction step of a memory read control circuit 18 is set to the optimim state corresponding to the absolute value of the calculated representative motion vector. Then, picture are moved by the set correction step, image blur correction is executed and then, this processing operation is ended.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image shake preventing device for preventing an image of an image pickup means from shaking due to camera shake, etc., and particularly to an image pickup having a magnetic recording / reproducing function represented by a portable video camera or the like. The present invention relates to an image blur prevention device suitable for the means.

[0002]

2. Description of the Related Art Generally, in image pickup means such as a video camera,
Whether it is for industrial use or consumer use, the camera shake makes it difficult to see the image and causes various malfunctions. In particular,
Image blurring tends to occur when photographing from a moving vehicle while walking, or in a place where there is a lot of vibration.Therefore, various image blurring prevention devices have been conventionally proposed to correct image blurring as described below. There is.

(A) An inertial pendulum type anti-shake lens having a biaxial gimbal structure is arranged around the master lens, and the anti-shake lens cancels camera shake.
Inertial pendulum type image blur prevention device for correcting image blur (US Pat. No. 2,959,088 and US Pat. No. 2,822,955).
No. 7).

(B) A variable apex angle prism for changing the optical axis of the lens (front lens) is provided at the tip of the lens (front lens) to detect the motion from the image signal output from the image pickup device, or the motion is detected by the acceleration sensor. A variable apex prism type image shake preventing device for detecting an image shake by driving the variable apex angle prism according to the detected signal.

(C) The image signal output from the image pickup device (CCD) is stored in an image memory or the like, the image shake is detected from the information, the amount of displacement of the image is obtained, and the read address of the image memory is correspondingly obtained. Of a pure electronic image blur correction device for correcting image blur by shifting
3-166370).

Of the above-mentioned conventional apparatuses, the pure electronic image blur prevention apparatus of (c) in particular does not require a special mechanical mechanism for correcting the image blur and is large scale due to the rapid progress of semiconductor technology. Since such electric circuits can be packaged in an extremely small package and can be made compact, lightweight, and low cost, they have been receiving attention in recent years.

[0007]

However, in the conventional pure electronic image blur prevention device, the correction step (when shifting the image when the image is electronically enlarged is smaller than the minimum pixel of the image sensor). When the image is moved (shifted) in the minimum shift unit, and corrected, there is a difference depending on the enlargement ratio and the image, but due to the movement of the position of resolution unevenness at the time of enlargement, moiré-like noise occurs and the image deteriorates. May occur, and even if the image is simply moved by a correction step that is less than or equal to the minimum pixel of the image sensor without performing expansion,
There is a problem that resolution deterioration occurs and the image deterioration after image shake correction is large.

The present invention has been made in view of the above-mentioned problems of the above-mentioned conventional technique, and an object of the present invention is to eliminate moire noise caused by deterioration of resolution and movement of positions of uneven resolution. It is an object of the present invention to provide an image blur prevention device capable of minimizing the image blur and minimizing the image deterioration after the image blur correction.

[0009]

In order to achieve the above object, a first invention of the present invention provides an image blur prevention apparatus for detecting a movement of a target image pickup means and correcting the shake in real time. A motion vector detecting means for detecting a motion vector between the two, an absolute deviation calculating means for calculating an absolute deviation from the reference point of the current image based on the motion vector detected by the motion vector detecting means, and the absolute deviation calculation Electronic correction means for electrically correcting the image shake based on the absolute deviation value calculated by the means, and control means for controlling the correction step of the electronic correction means and the image enlargement ratio to the optimum state according to the motion vector value. And is provided.

In order to achieve the same object, a second invention of the present invention detects a motion vector between images in an image shake preventing apparatus which detects a motion of a target image pickup means and corrects the shake in real time. Motion vector detecting means, an absolute deviation calculating means for calculating an absolute deviation from the reference point of the current image based on the motion vector detected by the motion vector detecting means, and an absolute deviation calculated by the absolute deviation calculating means. An electronic correction means for electrically correcting the image shake on the basis of the deviation value; and a control means for controlling the correction step of the electronic correction means and the image enlargement ratio to an optimum state according to the absolute deviation value. It is a feature.

[0011]

In the image blur preventing device of the first aspect of the invention, the motion vector detecting means detects the motion vector between the images, and the absolute deviation calculating means calculates the absolute deviation from the reference point of the current image based on the detected motion vector. Is calculated, the electronic correction means electrically corrects the image shake based on the calculated absolute deviation value,
The control means controls the correction step and the image enlargement ratio of the electronic correction means to the optimum state according to the motion vector value.

In the image blur preventing apparatus of the second invention, the motion vector detecting means detects the motion vector between the images, and the absolute deviation calculating means calculates the absolute deviation from the reference point of the current image based on the detected motion vector. The electronic correction means electrically corrects the image shake based on the calculated absolute deviation value, and the control means sets the correction step and the image enlargement ratio of the electronic correction means to the optimum state according to the absolute deviation value. Control.

[0013]

Embodiments of the present invention will be described below with reference to the drawings.

(First Embodiment) First, a first embodiment of the present invention will be described with reference to FIGS. FIG. 1 is a block diagram showing the configuration of a video camera which is an image pickup means provided with an image blur prevention device according to the first embodiment of the present invention. In the figure, 1 is a focus lens group for normal focusing, 2 is a zoom lens group for changing the focal length, and 3 is a zoom lens group.
Is a lens group of the correction system, 4 is an aperture for adjusting the amount of light, 5
Is an image sensor such as a two-dimensional CCD, which converts an optical signal input through each of the lens groups 1 to 3 and the diaphragm 4 into an electric signal and outputs the electric signal. 6 is a sample hold (S /
H) circuit holds an electric signal output from the image pickup device 5. 7 is the automatic gain control (A
The GC) circuit automatically controls the gain of the signal output from the S / H circuit 6.

An analog / digital (A / D) converter 8 converts the analog signal output from the AGC circuit 7 into a digital signal. 9 is a delay circuit (2HDL
The Y) circuit receives the output signal of the A / D converter 8 and delays the color difference line sequential signal from the image sensor 5 by two horizontal scanning periods. 10 is a color signal generation (C process) circuit, and 2
The output signal of the HDLY circuit 9 is input and a color (C) signal is generated. 11 is a low-pass filter (LPF),
The output signal of the HDLY circuit 9 is input and the color signal mixed in the luminance signal is removed. 12 is an enhancer, LPF
The output signal 11 is input and the high frequency component is enhanced.

A gamma (γ) correction circuit 13 receives the output signal of the enhancer 12 and performs gamma correction processing.
Reference numeral 14 is a two-dimensional band pass filter (BPF) which is a spatial frequency filter, to which the output signal of the gamma correction circuit 13 is input and which removes a signal in a predetermined band. A motion vector detection circuit 15 receives an output signal from the BPF 14 and a first field memory 16 described later, and detects a motion vector of an image. The motion vector detection circuit 15 is a circuit based on a matching calculation, and in this embodiment, it needs to be a detection method capable of real-time processing. Reference numeral 16 is a first field memory to which the output signal from the BPF 14 is input. The first field memory 16 is a delay circuit that delays the luminance signal for a predetermined time (1 field time in this embodiment), stores the luminance signal of one field before, and enables matching calculation with the current field. To do.

Reference numeral 17 is a logical operation circuit for performing various kinds of signal processing to control the entire image pickup means, and the output signals of the motion vector detection circuit 15 and an image enlargement rate input switch (image enlargement rate changing means) described later are inputted. . In the present embodiment, the logical operation circuit 17 has an image enlargement ratio changing means for changing the image enlargement ratio based on the input output signal of the image enlargement ratio input switch.

A memory read control circuit 18 controls a read position of a second field memory 19, which will be described later, based on a control signal from the logical operation circuit 17. The memory read control circuit 18 uses the first field memory 1
The second correction means for correcting the image blur is constituted by using the image having the number 6 and delayed by the first field memory 16. A second field memory 19 receives the output signals of the C process circuit 10 and the gamma correction circuit 13.

An electronic zoom circuit 20 receives the output signals of the second field memory 19 and the logical operation circuit 17, and converts the image into a desired size. A digital / analog (D / A) converter 21 converts a digital signal output from the electronic zoom circuit 20 into an analog signal. 22 is a signal output terminal for the electronic zoom circuit 2
The corrected image signal output from 0 is output. 23
Is an image magnification input switch to input the image magnification,
The output signal is input to the logical operation circuit 17.

Next, the operation of the image pickup means having the above construction will be described.

The subject 23 includes lens groups 1 to 3 and a diaphragm 4
The image is formed on the image pickup surface of the image pickup element 5 and photoelectrically converted. The S / H circuit 6 holds the output signal of the image sensor 5, and the AGC circuit 7 that follows the S / H circuit 6 automatically executes a gain control process. The A / D converter 8 A / D converts the output signal of the AGC circuit 7. The 2HDLY circuit 9 converts the color difference line sequential signal from the image sensor 5 into a 1H delay signal and (0H + 2
H) Delay signal is separated into a luminance signal processing unit side (LPF 11 side) and a color signal processing unit side (C process circuit 1).
0 side). The signal sent to the color signal processing unit side is input to the C process circuit 10, and the C process circuit 1
0 generates a color signal C, and the generated color signal C is written in the second field memory 19.

On the other hand, the signal sent to the luminance signal processing section side is input to the LPF 11, which removes the carrier component from the color difference line sequential signal and performs luminance signal separation. The enhancer 12 performs processing for enhancing high frequency components such as edges of the subject 23 in order to improve image quality. Normally,
The second derivative of the video signal is added to the original signal. Subsequently, the gamma correction circuit 13 prevents saturation in the highlight portion and widens the dynamic range. The BPF 14 extracts a spatial frequency component effective for detecting a motion vector.

Generally, the low-frequency component and the high-frequency component of the image signal are not suitable for detecting the motion vector, so that the BP
Preliminarily removed by F14. In this embodiment,
It is assumed that only the sign bit of the BPF 14 is output. This means that the luminance signal is binarized using the DC level as a threshold. Therefore, the luminance signal after BPF14 is
It is 1 bit.

The motion vector detection circuit 15 includes a BPF 14
Also, the motion vector of the image is detected based on the signal input from the first field memory 16, and the detected motion vector signal is input to the logical operation circuit 17. Also,
The logical operation circuit 17 includes an image enlargement ratio input switch 2
An enlargement ratio signal indicating the image enlargement ratio at the time of image blur correction is input from 3. The logical operation circuit 17 calculates the deviation from the reference position of the image at that moment based on the motion vector signal (horizontal and vertical components of the motion vector at the predetermined screen position) according to the flowchart shown in FIG. To do. Then, based on this deviation, the memory read control circuit 1
In step 8, the read position of the second field memory 19 is controlled by a correction step according to the image enlargement ratio, and the image is converted at the desired enlargement ratio previously input in the electronic zoom circuit 20. In this way, an image with the image shake corrected is finally obtained. The corrected image signal is D / A converted by the D / A converter 21 and then output from the signal output terminal 22.

Next, the operation of the logical operation circuit 17 in the image blur prevention device according to this embodiment will be described with reference to FIGS. FIG. 2 is a flowchart showing the operation of the logical operation circuit 17. First, step S201 in FIG.
Then, the output signal from the motion vector detection circuit 15 (each component in the horizontal direction and the vertical direction of the motion vector at the predetermined screen position) is read for each field. Then, step S2
In step 02, the motion vector at the plurality of screen positions read in step S201 is subjected to predetermined processing to calculate one representative motion vector. The predetermined processing includes reliability evaluation processing of individual motion vectors, control target area determination processing, and the like.

Next, in step S203, the representative motion vector is integrated to obtain a deviation (image displacement amount) from the reference position on the screen, and an image shake correction signal is obtained. Next, in step S204, the correction step of the memory read control circuit 18 is set to the optimum state according to the absolute value of the representative motion vector calculated in step S202. Next, the process proceeds to step S205, the image is moved in the correction step set in step S204 to execute the image shake correction, and then the present processing operation is ended.

Since the processing routine of step S204 of FIG. 2 is a part of the present invention, it will be described below with reference to FIGS. 1 and 3.

FIG. 3 is a diagram showing the relationship between the representative motion vector value and the optimum correction step, in which the vertical axis represents the correction step and the horizontal axis represents the representative motion vector. Here, the representative motion vector represents the relative displacement of the image one field before and the current image. Also,
The correction step is a minimum shift unit when shifting an image, and normally, the correction step is set according to the detection accuracy of the motion vector. However, it has been found that the movement of the position of the resolution unevenness due to the image enlargement becomes conspicuous when the correction step is reduced to move the image finely.

Therefore, as shown in FIG. 3, when the representative motion vector is large, that is, when the image shake is severe, the fineness of the wood grain when correcting the image is not so important. The unevenness of the movement of the position is less noticeable, and the awkwardness of the image correction operation is more noticeable in a region where the representative motion vector is small, that is, when the image shake is small. It is possible to reduce noise due to movement of positions of uneven resolution.

According to the image blur prevention apparatus of the present embodiment, in the image blur prevention apparatus for electronically correcting the image blur,
By setting the correction value (shift step) when correcting the image shake according to the absolute value of the representative motion vector, that is, the degree of the image shake to the optimum state, noise due to resolution deterioration and movement of the position of resolution unevenness is eliminated. There is an effect that it can be minimized, and image deterioration after shake correction can be minimized.

(Second Embodiment) Next, a second embodiment of the present invention will be described with reference to FIG. FIG. 4 is a diagram showing the relationship between the representative motion vector value and the optimum correction step in the image blur prevention device according to the second embodiment of the present invention.
In the present embodiment, the relationship between the representative motion vector value and the optimum correction step is not changed stepwise as in the first embodiment, but is changed exponentially.

The rest of the configuration and basic operation of this embodiment are the same as those of the first embodiment described above,
The description is omitted.

(Third Embodiment) Next, a third embodiment of the present invention will be described with reference to FIGS. Since the structure of the video camera, which is the image pickup means having the image blur prevention device according to this embodiment, is the same as that of FIG. 1 in the above-described first embodiment, the description will be made by diverting the drawing. FIG. 5 is a flowchart showing the operation of the logical operation circuit 17 in the image blur prevention apparatus according to this embodiment. First, in step S501 of FIG. 5, the output signal (horizontal and vertical components of the motion vector at a predetermined screen position) from the motion vector detection circuit 15 is read field by field. Next, in step S502, a predetermined representative motion vector is calculated by performing a predetermined process on the read motion vector at the plurality of screen positions. The predetermined processing includes reliability evaluation processing of individual motion vectors, control target area determination processing, and the like.

Next, in step S503, the representative motion vector is integrated to obtain a deviation (image displacement amount) from the reference position on the screen, and an image shake correction signal is obtained. Next, in step S504, the correction step of the memory read control circuit 18 is set to the optimum state according to the absolute value of the displacement amount of the image calculated in step S503.
Then, the process proceeds to step S505, and the step S5 is performed.
After the image is moved and the image shake correction is executed in the correction step set in 04, this processing operation is ended.

Since the processing routine of step S504 in FIG. 5 is a part of the present invention, it will be described below with reference to FIGS. 1 and 6.

FIG. 6 is a diagram showing the relationship between the representative motion vector value and the optimum correction step. In this figure, the vertical axis represents the correction step and the horizontal axis represents the image displacement amount. Here, the image displacement amount is a relative position of the current image with respect to a predetermined reference position. Further, the correction step is a minimum shift unit when shifting an image, and normally, the correction step is set according to the detection accuracy of the motion vector. However, it has been found that the movement of the position of the resolution unevenness due to the image enlargement becomes conspicuous when the correction step is reduced to move the image finely.

Therefore, as shown in FIG. 6, when the image displacement is large, that is, when the image shake is severe, the fineness of the wood grain when correcting the image is not so important. The uneven movement of the image is less noticeable, and when the image displacement amount is small, that is, when the image shake is small, the awkwardness of the image correction operation is more noticeable. It is possible to reduce noise due to movement of positions of uneven resolution.

According to the image blur prevention apparatus of the present embodiment, in the image blur prevention apparatus for electronically correcting the image blur,
By setting the correction step when correcting the image shake according to the image displacement amount, that is, the degree of the image shake to the optimum state, it is possible to minimize the noise due to movement of the position of resolution deterioration and resolution unevenness. The effect is that image deterioration after shake correction can be minimized.

(Fifth Embodiment) Next, a fifth embodiment of the present invention will be described with reference to FIG. FIG. 7 is a diagram showing a relationship between an image displacement amount and an optimum correction step in the image blur preventing device according to the fifth embodiment of the present invention. In the present embodiment, the relation between the representative motion vector value and the optimum correction step is not changed stepwise as in the fourth embodiment, but is changed exponentially.

The rest of the configuration, operation, and effect of this embodiment are the same as those of the above-mentioned fourth embodiment, so a description thereof will be omitted.

(Sixth Embodiment) In the above-described first to fifth embodiments, the case where the present invention is applied to the image blur preventing apparatus for performing the image blur correction using the field memory has been described, but the present invention is not limited to this. Instead of the field memory, it may be applied to an image blur prevention device that performs image blur correction by using a large-area image sensor having an area larger than usual.

FIG. 8 is a block diagram showing the structure of a video camera which is an image pickup means having an image blur prevention device according to the sixth embodiment of the present invention. The same parts as 1 are assigned the same reference numerals. 8 is different from FIG. 1 in that the image sensor 5, the memory read control circuit 18 and the second field memory 19 are removed from the configuration of FIG. ′ And the image sensor read circuit 24 are provided respectively. The large area image pickup device 5'and the image pickup device readout circuit 24 constitute a first correction means having a feedback loop and correcting image shake. Then, by changing the read address in the image sensor read circuit 24, the large area image sensor 5 '
The image blur correction is performed by cutting out an arbitrary place of.

[0043]

As described in detail above, according to the image blur prevention apparatus of the present invention, in the image blur prevention apparatus for electronically correcting the image blur, when the image blur is corrected according to the degree of the image blur. By setting the correction step of (1) to the optimum state, it is possible to minimize the noise caused by the resolution deterioration and the movement of the position of the resolution unevenness, and it is possible to minimize the image deterioration after the image shake correction. Play.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a video camera which is an image pickup unit including an image blur prevention device according to a first embodiment of the present invention.

FIG. 2 is a flowchart showing a control operation of a logical operation circuit in the image blur prevention device shown in FIG.

FIG. 3 is a diagram showing a setting example of a correction step for a change in the representative motion vector in the image blur prevention device shown in FIG.

FIG. 4 is a diagram showing a setting example of a correction step with respect to a change in a representative motion vector in the image blur prevention apparatus according to the second embodiment of the present invention.

FIG. 5 is a flowchart showing a control operation of a logical operation circuit in an image blur prevention device according to a third embodiment of the present invention.

FIG. 6 is a diagram showing a setting example of a correction step with respect to a change in image displacement amount in the image shake preventing apparatus according to the embodiment.

FIG. 7 is a diagram showing a setting example of a correction step with respect to a change in an image displacement amount in an image shake preventing apparatus according to a fourth embodiment of the present invention.

FIG. 8 is a block diagram showing a configuration of a video camera which is an image pickup means including an image blur prevention device according to a fifth embodiment of the present invention.

[Explanation of symbols]

 15 motion vector detection circuit (motion vector detection means) 17 logical operation circuit (absolute deviation operation means, control means) 18 memory read control circuit (electronic correction means)

Claims (2)

[Claims] 1. An image blur prevention device for detecting movement of a target image pickup means and correcting the blur in real time,
Motion vector detecting means for detecting a motion vector between images, absolute deviation calculating means for calculating an absolute deviation from a reference point of the current image based on the motion vector detected by the motion vector detecting means, and the absolute deviation Electronic correction means for electrically correcting image shake based on the absolute deviation value calculated by the calculation means, and control for controlling the correction step of the electronic correction means and the image enlargement ratio to an optimum state according to the motion vector value. An image blur preventing device, comprising: 2. An image blur prevention device for detecting the movement of a target imaging means and correcting the blur in real time,
Motion vector detecting means for detecting a motion vector between images, absolute deviation calculating means for calculating an absolute deviation from a reference point of the current image based on the motion vector detected by the motion vector detecting means, and the absolute deviation Electronic correction means for electrically correcting the image shake based on the absolute deviation value calculated by the calculation means, and control for controlling the correction step and the image enlargement ratio of the electronic correction means to the optimum state according to the absolute deviation value. An image blur preventing device, comprising: