JP2944369B2 - Video camera shake correction device and video camera - Google Patents

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a camera shake correction apparatus for a video camera and a video camera having the same, and more particularly to, for example, evaluating whether or not a motion vector has been correctly detected based on a change in coordinates of a correlation value in a representative point matching method. for, the slope indicating the sharpness of the minimum correlation values
There was to determine whether greater than the threshold value, regarding image stabilizer Oyo Bibi camcorders.

[0002]

2. Description of the Related Art An example of a method for detecting a shake component of an image pickup apparatus has been disclosed by Matsushita Electric Industrial Co., Ltd. at the 20th Conference on Image Engineering in 1989. This method
JP-A-61-201381 [H04N 7/137]
This technique detects a shake component of an imaging device from image information using a motion vector obtained by a representative point matching method described in the publication. In this presentation, camera shake correction is performed based on motion vectors obtained from image information. In this case, four detection areas are arranged on the screen, and four motion vectors are obtained from one screen.

However, in the above-described method of detecting a shake component of an image pickup device from image information, the accuracy greatly changes depending on the state of an input image. For this reason, the validity of the motion vector detected according to the state of the image is evaluated by the microcomputer. FIG. 9 is a graph showing a change in the correlation value with respect to the coordinate position when the representative point matching method is used.
Normally, as shown by a solid line a in FIG. 9, a sharp minimum point indicating the minimum correlation value is provided, and a motion vector is given based on the position of the minimum correlation value. On the other hand, when the contrast of the screen is small or when there is a moving object in the image, the result is as shown by the one-dot chain line b and the broken line c in FIG. 9, and the detection accuracy of the motion vector is deteriorated. Further, even when there is a subject (such as a stripe image) having a repetitive pattern in an image, it is impossible to detect a motion vector. In order to determine the above state, the condition of Equation 3 is provided for the correlation value. However, “slope” is the minimum correlation as shown in FIG.
Indicates the sharpness of the value.

[0004]

(A) average correlation value> α (B) (value obtained by dividing the average correlation value by the minimum correlation value)> β (C) slope> γ where α, β, and γ are constant threshold values, For example, α
= 36, β = 7, γ = 8. This number 3
(A) shows that the average value, that is, the image contrast is sufficiently large.
(B) shows that the minimum value is the average value
(C) is the minimum value
Has a sharp peak.

The condition of equation (3) is examined for each of the four detection areas. If the condition of equation (3) is not satisfied, the detection area is regarded as an invalid area. Is an effective area. Then, a motion vector due to camera shake is determined using only the motion vector obtained from the effective area. As a method of obtaining a motion vector due to camera shake, for example, an average of motion vectors of an effective area is used.

[0006]

However, in the condition for judging whether the area is an effective area or an invalid area as shown in the equation (3), if the screen contrast is slightly deteriorated (lower), the condition (A) of the equation (3) is preceded. Since the condition (C) is no longer satisfied, the threshold value γ of the condition (C) needs to be reduced. However, regardless of the contrast of the screen, if the threshold value γ is uniquely reduced, it is easy to make a mistake in determining whether or not there is a subject (such as a stripe image) having a repetitive pattern in the image, and the accuracy of detecting a motion vector due to camera shake is reduced. There was a problem of falling.

SUMMARY OF THE INVENTION Therefore, a main object of the present invention is to provide a video camera having a camera shake correction device capable of improving the accuracy of detecting a motion vector.

[0008]

A first invention is a video camera.
Mera's image stabilization device, using the representative point matching method
The motion detection means.
Detect pixel correlation value, average correlation value and minimum correlation value,
In order to evaluate whether the motion vector was correctly detected based on the change in the coordinates of the correlation value , the minimum phase
In that slope indicating the sharpness of the function value has a tilt determining means for determining whether greater than the threshold value, the average correlation value
A determination means for determining whether the average correlation value is larger than the predetermined value is further provided .
It is determined whether the average correlation value is smaller than the first threshold and the slope is larger than the second threshold smaller than the first threshold when the average correlation value is smaller.
A camera shake correction device characterized in that it is determined whether or not the image is shaken.

[0009]

A second invention is a video camera according to the first invention.
5 is a video camera having a camera shake correction device.

[0011]

[0012]

The threshold value in the inclination judging means for judging the inclination is changed according to the average correlation value. That is, the inclination judgment
When the contrast of the screen is large, that is, when the average correlation value is large, the inclination is larger than the first threshold.
When the contrast of the screen is small, that is, when the average correlation value is small, it is smaller than the first threshold.
The inclination is determined based on the second threshold value . Therefore, the threshold value and the inclination adapted to the change in the contrast of the screen can be compared, and the determination as to whether or not there is a subject having a repetitive pattern in the image becomes more accurate.

[0013]

According to the present invention, it is possible to more accurately determine whether or not there is a subject having a repetitive pattern (such as a stripe image) in an image, and it is invalid even if the contrast of the screen is slightly deteriorated. Since no judgment is made, the detection accuracy of the motion vector is improved.

The above objects, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

[0015]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS A video camera 10 of this embodiment shown in FIG.
Includes a solid-state imaging device 12 such as a CCD that converts a light signal from a subject (not shown) input from a lens 14 into an electric signal. An electric signal from the solid-state imaging device 12 is input to a camera circuit 16. As is well known, the camera circuit 16 includes a sample and hold circuit, and the solid-state imaging device 1
The sample and hold of the electric signal from 2 is performed. The level of the sampled and held electric signal is adjusted by the AGC, and a synchronizing signal is further added by a synchronizing signal adding circuit. Thus, the camera circuit 16 converts the image signal from the solid-state imaging device 12 into an analog video signal. This analog video signal further comprises A
The digital video signal is converted by the / D converter 18. The digital video signal is provided to the motion detection circuit 20. As the motion detection circuit 20, for example, an LSI “L7A0948” manufactured by Sanyo Electric Co., Ltd. is used.
Under the control of a memory control circuit 22 included in the same LSI constituting the motion detection circuit 20, digital video signals are written in a field memory 24 in a field sequence.

The motion detection circuit 20 uses the well-known representative point matching method, for example, to obtain the highest degree of correlation (minimum correlation value) for each of the four detection areas A, B, C and D shown in FIG. The position of one point and four points around the point and the respective correlation values are calculated. The position data and the correlation value data from the motion detection circuit 20
Given to.

That is, referring to FIG. 2, motion detecting circuit 20 includes an input terminal 28 for receiving a digital video signal from A / D converter 18, and the digital video signal input from input terminal 28 is filtered. The signal is supplied to a representative point memory 32 and a subtraction circuit 34 through 30. The filter 30 is a kind of digital low-pass filter,
It is used to improve the S / N ratio and ensure sufficient detection accuracy with a small number of representative points. The representative point memory 32 extracts a plurality of representative points in each of the detection areas A to D shown in FIG.
Then, 30 representative points are extracted), and the position data and the luminance data are stored. Each detection area 42 (FIG. 4) formed by dividing into 30 is composed of, for example, 32 pixels × 16 rows.

The subtraction circuit 34 calculates the luminance data of the representative point of the previous field supplied from the representative point memory 32 and the input terminal 2.
8 is subtracted from the luminance data of all the pixels in the current field given by 8 and the absolute value is obtained. That is, a luminance difference is obtained between the luminance data of the current field and the luminance data of the previous field. The obtained luminance difference is given to the accumulation circuit 36. The accumulative addition circuit 36 accumulatively adds (30 in this embodiment) the luminance differences obtained for the pixels at the same position in each detection area 42 in the same detection area, and outputs correlation value data. The correlation value data is supplied to an arithmetic circuit 38, which calculates a minimum correlation value and an average correlation value for each of the detection areas A to D, and calculates the position data of the pixel indicating the minimum correlation value in each of the detection areas. Determined for each AD. The minimum correlation value, average correlation value and position data obtained in this way are output from the output terminal 40 to the microcomputer 2 described above.
6 given. However, the calculation of such a correlation value is
This is executed by the aforementioned LSI “L7A0948”.

In the microcomputer 26,
Based on the position data and the correlation value data, a motion vector (hereinafter, simply referred to as “overall motion vector”) due to camera shake of the entire screen, that is, the image field 44 (FIG. 3) is calculated. First, a shift of the pixel having the minimum correlation value with respect to the representative point is obtained based on the position data of the pixel having the minimum correlation value, and the shift is set as a partial motion vector.
In order to improve the detection accuracy of the partial motion vector,
Interpolation is performed using the correlation values of the four pixels surrounding the pixel having the minimum correlation value, and the position data of the pixel having the minimum correlation value is calculated.

The microcomputer 26 evaluates the validity of the partial motion vector detected from each detection area, that is, whether the detection area is an effective area or an invalid area, based on the state of the image. In this embodiment, since the representative point matching method is used, the change in the correlation value with respect to the coordinate position is:
It is the same as FIG. Here, in order to determine whether the contrast of the screen is low, whether there is a moving object in the detection area, and whether there is a subject (for example, a stripe image or the like) having a repetitive pattern in the detection area, Equation 6 is provided for the correlation value and the slope.

[0021]

(A) Average correlation value> α (B) (A value obtained by dividing the average correlation value by the minimum correlation value)> β (C) Slope> γ (where Average correlation value ≧ ε) (D) Slope> δ (where, average correlation value <ε) where α, β, γ, δ, and ε are fixed thresholds and γ ≧ δ. For example, α = 36, β = 7, γ = 8, δ = 4 ,
ε = 128 is set.

The microcomputer 26 determines whether or not the screen contrast is low based on whether or not the average correlation value is larger than a predetermined value α (condition (A)), and divides the average correlation value by the minimum correlation value. Is greater than or equal to the threshold value β [condition (B)], it is determined whether there is any moving object in the detection area. Further, when the average correlation value is equal to or larger than the threshold ε, it is determined whether the slope is larger than the threshold γ [condition (C)]. When the average correlation value is smaller than the threshold ε, whether the slope is larger than the threshold δ [condition (D )], It is determined whether there is a subject having a repetitive pattern. These processes are performed for each detection area A
-D, and determines whether or not the partial motion vector from each of the detection areas AD is reliable without being erroneously detected by a moving object other than camera shake, that is, whether or not each of the detection areas AD is an effective area. . If all of the conditions (A) to (D) of Expression 6 are satisfied, the detection area is determined to be an effective area, and if any of the conditions is not satisfied, the detection area is determined to be an invalid area.

Specifically, whether or not an area is an effective area is determined as follows. First, when the contrast of the screen is low, since the luminance difference is small, the correlation value is small. For example, when the entire screen is white, the correlation value becomes small. In such a case, since the reliability is lost, the average correlation value>
It is determined to be valid when α. The threshold α is determined by experiment. In this way, it is determined from the average correlation value whether or not the contrast is low.

When there is a moving object in the detection area, the correlation value differs between the portion occupied by the moving object and the portion not occupied, and the portion occupied by the moving object takes various correlation values. (The correlation degree is low). Therefore, when there is a moving object in the detection area, there is a high possibility that the minimum correlation value increases, and there is a possibility that a partial motion vector in the detection area is erroneously detected. When the partial motion vector is erroneously detected, the entire motion vector is erroneously detected. However, when the average correlation value is large, it is reliable even if the minimum correlation value is somewhat large. On the other hand, when the average correlation value is small, the minimum correlation value cannot be relied on unless it is smaller. Therefore, specifically, for example, when (average correlation value) / (minimum correlation value)> 7, it is determined to be valid,
By avoiding the use of the partial motion vector of the detection area that does not satisfy this condition, the above-described adverse detection is prevented. Thus, (average correlation value) / (minimum correlation value)
To determine the presence or absence of a moving object.

Further, in the microcomputer 26,
In order to detect a subject having a repetitive pattern, a correlation value of one point having the minimum correlation value and four surrounding points is used. That is, as shown in FIG. 5, assuming that the minimum correlation value is M and the correlation values at the four points on the left, right, upper and lower sides are L, R, U, and D, respectively, the difference between the respective correlation values, that is, LM, R− M, UM
And DM are calculated, and the minimum value among them is defined as the slope. If the average correlation value is equal to or larger than the threshold value ε, the slope is compared with a threshold value γ obtained by a field test.
If the inclination is larger than the threshold γ, the detection area is determined to be an effective area, and if the inclination is equal to or smaller than the threshold γ, the detection area is determined to be an invalid area. On the other hand, if the average correlation value is smaller than the threshold ε, the slope is compared with a threshold δ obtained by a field test. If the inclination is larger than the threshold γ, the detection area is determined to be an effective area, and if the inclination is equal to or smaller than the threshold γ, the detection area is determined to be an invalid area. Paraphrase
For example, in this embodiment, the inclination (FIG. 9) is determined.
Whether to use the threshold γ or the threshold δ to determine the average
Determined by whether it is greater than. That is, the inclination
The threshold value for judging is changed according to the average value.
doing. When the average correlation value is large, compare with the slope.
Power threshold is set large, and when the average correlation value is small,
Sets the threshold value small. Therefore, the screen
You can compare the slope and the threshold that is adapted to the change in trust,
Judge whether there is a subject with a repetitive pattern in the image.
More accurate.

As described above, whether the detection area is an effective area is determined based on the conditions (A) to (D). And
The amount of motion between fields, that is, the entire motion vector is determined using the partial motion vector of the effective area. The overall motion vector indicates the amount of motion between fields and its direction. As the whole motion vector, when there is an effective area, for example, the average of partial motion vectors of the effective area is used. When there is no effective area, for example, a value of (the whole motion vector of the previous field × 0.97) is used.

The overall motion vector obtained in this manner is supplied to the memory control circuit 22. Then, the memory control circuit 22 determines the read start address of the field memory 24 based on the entire motion vector, and reads the digital video signal stored in the field memory 24 from that address. That is, the memory control circuit 22 controls the extraction area 46 formed by the digital video signal of the field memory 24 according to the entire motion vector calculated by the microcomputer 26.
Move (FIG. 6).

However, if the digital video signal read from the field memory 24 is not used, the extraction area 4
6 cannot be moved, the electronic zoom circuit 48 (FIG. 1) is used. Referring to FIG. 6, electronic zoom circuit 48 sets an extraction area 46 in which an image is enlarged in accordance with the zoom magnification for the size of image field 44. The position of the extraction area 46 can be freely moved within the range of the image field 44 by changing the read start address of the field memory 24. And
In order to obtain a video signal of the entire image field 44 based on the extracted digital video signal, the image is enlarged using interpolation based on the digital video signal read from the field memory 24.

Thus, the image field 44
By electronically zooming the image of any of the extraction areas 46 in the electronic zoom circuit 48 with the electronic zoom circuit 48, a correctable range 50 corresponding to the difference between the image field 44 and the extraction area 46 can be formed. When the camera shakes as shown in FIG. 7 in response to the vibration of the hand of the person who operates the video camera 10, the image from the video camera 10 shakes, and as a result, the image field 44
If the target person exists in the lower left of (),
For example, a case where the target person exists at the upper right of the image field 44 (the lower part in FIG. 7) occurs. Therefore, the extraction area 46 is set for each field by the microcomputer 26.
By moving the object in accordance with the overall motion vector calculated as described above, the target person just fits in the extraction area 46 as shown in the right side of FIG.

The digital video signal output from the electronic zoom circuit 48 is converted into an analog signal by the D / A converter 52 and output from the output terminal 54. The main operation of the microcomputer 26 of the video camera 10 will be described with reference to FIG. In the flowchart of FIG. 8, the detection areas 1 to 4 correspond to the detection areas A to D shown in FIG.

In step S1 shown in FIG. 8, the detection area n is initialized to 1. In step S3, if the average correlation value of the detection area n is larger than the threshold α, the process proceeds to step S5. In step S5, if the value obtained by dividing the average correlation value of the detection region n by the minimum correlation value is larger than the threshold value β, the process proceeds to step S7. If it is determined in step S7 that the average correlation value is equal to or larger than the threshold ε, the process proceeds to step S9. In step S9, if the inclination of the detection area n is larger than the threshold γ, the process proceeds to step S13, and the detection area n is determined to be an effective area. On the other hand, if the average correlation value is smaller than the threshold value ε in step S7, the process proceeds to step S11. Step S
In step 11, if the inclination of the detection area n is larger than the threshold value δ, the process proceeds to step S13, and the detection area n is determined to be an effective area.

If the average correlation value of the detection area n is equal to or smaller than the threshold value α in step S3, and if the value obtained by dividing the average correlation value of the detection area n by the minimum correlation value is equal to or smaller than the threshold value β in step S5, the detection area is determined in step S9. When the slope of n is equal to or smaller than the threshold γ, and when the slope of the detection area n is equal to or smaller than the threshold δ in step S11, the process proceeds to step S15, and the detection area n is determined to be an invalid area.

After the processes in steps S13 and S15, the process proceeds to step S17, where the detection area n is incremented by one, and the process proceeds to step S19. Step S1
If the detection area n is equal to or less than 4 in 9, the process returns to step S3. That is, in step S19, the detection area n
Are repeated until the value exceeds 4. If the detection area n exceeds 4 in step S19, the process proceeds to step S21. In step S21, if the number of effective areas is larger than 0, that is, at least one, the process proceeds to step S2.
Proceed to 3. In step S23, the process ends with the average of the partial motion vectors of the effective area as the entire motion vector.
On the other hand, if there is no valid area in step S21,
Proceed to step S25. In step S25, the value of (the whole motion vector of the previous field × 0.97) is set as the whole motion vector of the current field.

According to this embodiment, it is possible to more accurately determine whether or not a subject having a repetitive pattern exists in the detection area, and it is determined that the subject is an invalid area even if the screen contrast is slightly deteriorated. Not even. Therefore, the validity / invalidity of the detection area can be determined more accurately, and the detection accuracy of the entire motion vector is improved. In the above embodiment,
Although the luminance difference is obtained between the luminance data of the adjacent fields, the present invention is not limited to this, and the luminance difference may be obtained between the luminance data of the adjacent frames. When a luminance difference is obtained between luminance data of adjacent frames, the same processing as in the above-described embodiment is performed.

In the above-described embodiment, the correlation value data is obtained by accumulatively adding the luminance difference. However, the present invention is not limited to this. Correlation value data may be obtained by using the difference between adjacent fields (frames) of the electric signal from the solid-state imaging device 12 instead of the luminance difference.

[Brief description of the drawings]

FIG. 1 is a block diagram showing one embodiment of the present invention.

FIG. 2 is a block diagram illustrating a motion detection circuit according to the embodiment in FIG. 1;

FIG. 3 is an illustrative view showing a principle of electronic zoom and showing a detection area in an image field;

FIG. 4 is an illustrative view showing a principle of electronic zoom and showing a representative point and a sampling point in a detection area.

FIG. 5 is an illustrative view for explaining a method of detecting a subject having a repetitive pattern using a pixel having a minimum correlation value and four surrounding pixels;

FIG. 6 is an illustrative view showing a principle of camera shake correction;

FIG. 7 is an illustrative view showing each block in an image field to which the representative point matching method is applied;

FIG. 8 is a flowchart showing main operations of this embodiment.

FIG. 9 is a graph showing a change in a correlation value with respect to a coordinate position when a representative point matching method is used for each image state.

[Explanation of symbols]

 DESCRIPTION OF SYMBOLS 10 ... Video camera 12 ... Solid-state image sensor 16 ... Camera circuit 18 ... A / D converter 20 ... Motion detection circuit 22 ... Memory control circuit 24 ... Field memory 26 ... Microcomputer 42 ... Detection area 44 ... Image field

Claims (2)

(57) [Claims] An image stabilizing device for a video camera,
A motion detection means by a representative point matching method, the dynamic
The correlation value, average correlation value and
Detecting a minimum correlation value, tilt further the indicated to assess whether the motion vector is correctly detected based on a change with respect to the coordinate of the correlation values, the sharpness of the previous SL minimum correlation value
Having a slope decision means but to determine whether or not greater than the threshold value
A determination as to whether the average correlation value is greater than a predetermined value.
Disconnection means, the inclination determination means determines whether the inclination is greater than a first threshold value when the average correlation value is large ,
When the correlation value is small, the slope is smaller than the first threshold.
A camera shake correction device for a video camera, wherein it is determined whether the value is greater than a second threshold value . 2. A video camera comprising the video camera shake correction device according to claim 1.