KR101429509B1 - Apparatus for correcting hand-shake - Google Patents

Abstract

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital image processing apparatus, and more particularly, to an image stabilization apparatus that corrects an unintentional hand movement by obtaining an amount of image shift using two images captured at different times. An apparatus for correcting an unintentional hand movement by obtaining a shift amount from a correlation value between a current image captured by a video camera and a previously captured image, the apparatus comprising: Reliability determining means for calculating a vector and determining reliability of the shake amount by using the inherent value, and shake correcting means for correcting the shake by moving the previous image by the shake correction amount using the reliability.

Gradient covariance matrix, eigenvalues, eigenvectors, reliability determination,

Description

Apparatus for correcting hand-shake [0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a digital image processing apparatus, and more particularly, to an image stabilization apparatus that corrects an unintentional hand movement by obtaining an amount of image shift using two images captured at different times.

Generally, the reliability of the shaking motion depends on the contrast magnitude of the image. In general shaking motion correction techniques, however, a direction in which the contrast change is small and a direction in which the contrast change is large can not be obtained and accurate camera shake correction can not be performed.

SUMMARY OF THE INVENTION It is a technical object of the present invention to provide an image stabilization apparatus for correcting image stabilization after determining the reliability of the image stabilization by obtaining an eigenvalue using a gradient covariance matrix of a correlation value for the shaking amount.

SUMMARY OF THE INVENTION The present invention has been made in view of the above problems, and it is an object of the present invention to provide an apparatus for correcting a shaking motion by obtaining a shaking motion amount as a correlation value between a current image shot by a video camera and a previously shot image, A reliability determining means for calculating an eigenvalue and an eigenvector using a gradient covariance matrix and determining reliability of the shaking motion with the eigenvalue; And camera shake correction means for correcting the shaking motion by moving the previous image by the shaking motion correction amount using the reliability.

In the present invention, the reliability determining means may include: a first calculating unit that calculates a gradient covariance matrix from coordinates at which the correlation value becomes minimum; A second calculating section for calculating the eigenvalues from the gradient covariance matrix e _0, e _1, and there is the eigenvector corresponding to the v _0, v _1; And a determination unit for determining the reliability of the eigenvalues using the eigenvalues e ₀ and e ₁ and the set threshold values.

In the present invention, the judging unit may have a unique value e if ₀ is greater than the first threshold value is set, is greater than the first threshold value is a unique value e ₁ is set higher is the contrast change cursor reliability determination, the unique value e ₀ is set first When the eigenvalue e ₁ is smaller than the first threshold value and the eigenvalue e ₁ is smaller than the first threshold value and the reliability is low due to the small contrast change. If the eigenvalue e ₀ is smaller than the first threshold value and the eigenvalue e ₁ is greater than the first threshold value It can be judged that the contrast change in one direction is large and the reliability is partially high.

In the present invention, the reliability judging means may further comprise an extracting unit for extracting a shaking motion component in a direction in which a contrast change is large, when it is determined that the contrast change in one direction is large and thus the reliability is partially high.

As described above, according to the present invention, when there is a direction in which the contrast change is high even though it is determined that the reliability of the shaking motion is low, the shaking motion correction amount for the direction in which the contrast change is high is partially calculated to correct the shaking motion, Can be improved.

Hereinafter, the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram showing a configuration of an image stabilization apparatus according to the present invention.

1 includes a video camera 100, a filtering unit 110, a downsampling unit 120, a first field memory 130, a correlation value calculation unit 140, a correlation value table 150, A reliability determining unit 170, a correction amount calculating unit 180, and a second field memory 190. The search unit 160 includes a reliability determining unit 170, a correction amount calculating unit 180,

The color signal output from the video camera 100 is stored in the second field memory 190. Also, the DC component and the high frequency component of the color signal output from the video camera 100 are removed through the bandlimiting filtering unit 110 with respect to the luminance signal.

The signal from which the direct current component and the high frequency component are removed is down-sampled by the down-sampling unit 120, for example, down-sampled at intervals of 8 pixels in the vertical / horizontal direction and stored in the first field memory 130.

The correlation value computing unit 140 obtains a correlation value between the one-field image stored in the first field memory 130 and the current image down-sampled in the horizontal / vertical direction, and records the correlation value in the correlation value table 150. Equation (1) is an equation for obtaining a correlation value.

In equation 1 is the image area A to obtain the correlation, s _n (x, y) is the time coordinate (x, y) the luminance value of the pixel, i, j in the image of the image of the visual field n- n- field Horizontal, and vertical directions.

The minimum value search unit 160 obtains the amount of misalignment (i ₀ , j ₀ ) in which the correlation value is the smallest among the correlation values stored in the correlation value table 150, and determines the shift amount of the current field with respect to the previous field.

However, there is an error between the obtained shaking amount and the actual shaking amount. Therefore, it is necessary to judge the reliability of the shake amount. If the camera shake correction is performed when the reliability is low, the shake of the image is increased and the image quality is deteriorated. Therefore, when the reliability of the shaking motion is low, the camera shake correction is not performed. The reliability of the shake amount depends on the contrast magnitude of the image.

FIG. 2A shows the relationship between the amount i shifted when j = j ₀ and the correlation value, and FIG. 2B shows the relationship between the amount j shifted and the correlation value when i = i ₀ . The slope in the horizontal direction and the vertical direction in the vicinity of the minimum value is shown in Equation (2).

The reliability c of the shaking motion amount can be expressed as a function of? Rx,? Ri, r (i ₀ , j ₀ ) as shown in Equation (3).

As shown in FIG. 4A, the relationship between the amount of misalignment and the correlation value is shown for an image having a large change in contrast in two or more different directions as shown in FIG. 3A. It is determined that the reliability of the motion vector is high with such an image. Further, as the minimum correlation value r (i ₀ , j ₀ ) is smaller, the correlation between images becomes higher, so that the reliability is also increased. Therefore, it can be determined that it is reliable when the expression (4) is satisfied. In Equation (4), th _rx , th _ry , and th _r are predetermined threshold values.

On the other hand, as shown in FIG. 3B, even when the image is shifted in a direction in which the contrast change is small in one direction and the contrast change is large in only one direction, the correlation value hardly changes, Therefore, it can be determined that the reliability of the shake amount is low. For the image of Fig. 3B, the relationship between the amount of shift and the correlation value is shown in the contour diagram of Fig. 4B. (5) for the image of FIG. 3B and (6) for the image of FIG. 3C.

Therefore, if Equation (4) is satisfied, it is determined that the reliability of the obtained shaking motion is high, and the shaking motion is corrected.

However, in the case of FIG. 3D, because it satisfies the expression (4), it is actually determined that the reliability is high despite the low reliability. In the case of Figs. 3B, 3C, and 4D, the component in the direction with a large change in contrast often shows a correct value in the obtained shaking motion. Therefore, if the shaking motion correction is performed using this value, the image quality is reduced Since the direction in which the contrast change is small and the direction in which the contrast change is large can not be obtained, it is difficult to correct the partial shaking motion. Therefore, the reliability determination unit 170 obtains the direction in which the contrast change is small and the direction in which the contrast change is large, So that partial camera-shake correction becomes possible.

The reliability determining unit 170 includes a gradient covariance matrix calculating unit 171, an eigenvalue / eigenvector calculating unit 172, an eigenvalue reliability determining unit 173, and an eigenvalue calculating unit 174 .

The gradient covariance matrix calculation unit 171 obtains a gradient covariance matrix of the correlation value (i, j) in the vicinity of the coordinate where the correlation value searched by the minimum value search unit 160 becomes minimum.

Assuming that the coordinate at which the correlation value becomes minimum is (i ₀ , j ₀ ), the correlation value target range is shown in Equation (7). In Equation (7), k is an integer of 1 or more, for example, k = 3.

The gradient covariance matrix of the correlation value (i, j) in the vicinity of the coordinate at which the correlation value becomes minimum is shown in Equation (8).

Eigenvalue / eigenvector calculation section 172 calculates an eigenvector corresponding to the eigenvalue of the calculated gradient covariance matrix A e _0, e _1, and there is v _0, v _1, is shown in Equation (9).

It is assumed in Equation (9) by a unique value e ₀ <e _1.

The eigenvalue reliability determining unit 173 determines reliability by comparing the eigenvalue with the threshold values th _e0 and th _e1 . When the eigenvalues e ₀ and e ₁ are large, it is determined that the distribution of the eigenvectors is large and the contrast change is large. When the values e ₀ and e ₁ are small, it is determined that the distribution of the eigenvectors is small and the contrast change is small.

The eigenvalue confidence determining unit 173 determines that there is reliability when the contrast change is large in the other two directions as shown in Equation 10, and the correction amount calculating unit 180 uses the shaking amount (i ₀ , j ₀ ) The camera shake is corrected by calculating the correction amount.

If the eigenvalue reliability judgment unit 173 judges that the change of the contrast is small as shown in Equation (11), it judges that there is no reliability, and the correction amount calculation unit 180 does not calculate the correction amount of the correction.

The eigenvalue confidence determining unit 173 can be considered to have a small change in contrast in any direction shown in Figs. 3B, 3C, and 4D as in Equation (12), but a large change in contrast in other directions. Therefore, the correction amount calculating unit 180 calculates the correction amount of the shaking motion using only the shaking motion component in the direction in which the contrast change is large, and performs the partial shaking correction.

The eigen value calculation unit 174 of the correction object extracts a shaking component in a direction in which the contrast change is large, and the correction amount calculation unit 180 calculates the shaking motion correction amount to correct the shaking motion, as shown in Figs. 4C and 5 The value of the shake amount (i ₀ , j ₀ ) is known and correction is performed by obtaining the e ₁ component of the shake amount (i ₀ , j ₀ ), that is, m ₁ (i ₀ ", j" ₀ ).

The correction amount calculating unit 180 calculates the amount of correction in the time n-field by using the equation (13) when the time 0 field is used as a reference. In the present invention, the correction amount calculating unit 180 calculates the shaking motion correction amount when it is determined that the reliability is high, determines that the reliability is low, does not calculate the shaking motion correction amount when the contrast change is low, , The camera-shake correction amount for the direction in which the contrast change is high is calculated and the camera-shake correction is performed.

In Equation (13), x _n and y _n are horizontal and vertical shifts in the time n field, and p _x and p _y are proper integers (0 <p _x <1,0 <p _y <1).

Then, in the second field memory 190, the image before the one field is shifted by (x _n , y _n ), and the image with the camera-shake correction is output.

The present invention has been described with reference to the preferred embodiments. It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. Therefore, the disclosed embodiments should be considered in an illustrative rather than a restrictive sense. The scope of the present invention is defined by the appended claims rather than by the foregoing description, and all differences within the scope of equivalents thereof should be construed as being included in the present invention.

1 is a block diagram showing a configuration of an image stabilization apparatus according to the present invention.

2 is a graph showing the relationship between the amount (i, j) shifted in the horizontal / vertical direction and the correlation value.

3 is a diagram showing an example of an image showing a contrast change per direction.

Fig. 4 is a diagram showing the relationship between the amount of shift of the contrast-changed image disclosed in Fig. 3 and the correlation value, in contour lines. Fig.

Fig. 5 is a diagram showing extraction of the eigenvector component of the shaking motion correction object.

Claims (4)

An apparatus for correcting an unintentional hand movement by obtaining a shift amount from a correlation value between a current image captured by a video camera and a previously captured image, Reliability determining means for calculating an eigenvalue and an eigenvector from the coordinates at which the correlation value becomes minimum from the gradient covariance matrix and determining the reliability of the shaking motion with the eigenvalue; And Shake correction means for correcting the shaking motion by moving the previous image by the shaking motion correction amount using the reliability, The reliability judging means judges, A first calculation unit for calculating a gradient covariance matrix from coordinates at which the correlation value becomes minimum; A second calculating section for calculating the eigenvalues from the gradient covariance matrix e _0, e _1, and there is the eigenvector corresponding to the v _0, v _1; And When the eigenvalue e ₀ is greater than a first threshold value and the eigenvalue e ₁ is greater than a first threshold value set, it is determined that the contrast change is large and thus the reliability is high. When the eigenvalue e ₀ is smaller than a first threshold value When the eigen value e ₁ is smaller than the set first threshold value and the reliability is low due to the small contrast change. If the eigen value e ₀ is smaller than the set first threshold value and the eigen value e ₁ is larger than the set first threshold value And a determination unit that determines that the reliability of the image is partially high because the contrast change in the direction is large. delete delete The apparatus according to claim 1, wherein the reliability judging means Further comprising an extracting section for extracting a shaking motion component in a direction in which the contrast change is large when the contrast change in one direction is large and the reliability is partially determined to be high.

Images