JP2008160294A - Blur correction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a blur correction device appropriately correcting blur with less overcorrection. <P>SOLUTION: The blur correction device comprises: an image input means for inputting images; a blur detection means 104 for detecting the blur of the image; a pixel value detection means 103 for detecting the change direction and change amount of the pixel value of the image; and a correction means 107 for correcting the blur of the image on the basis of a correction amount adjusted according to the change direction and change amount of the pixel value. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a shake correction apparatus that corrects an image deteriorated due to camera shake or the like.

As one of the main methods of conventional electronic blur correction, the captured image is subjected to two-dimensional filter processing (blur inverse filter) that corrects the blur according to the blur to reduce the image degradation. There is something to improve. Specifically, a method of obtaining a blur point spread function (PSF) at the time of photographing and performing blur correction by using an inverse filter or a Wiener filter by convolution based on the point spread function is often used (for example, Patent Document 1). In addition, a method of creating a filter that corrects blur based on the direction and degree of blur is used (for example, see Patent Document 2).
Japanese Patent Laid-Open No. 11-24122 JP 2006-25395 A

  However, in the conventional blur correction apparatus, basically, only one type of filter is created according to the blur, and the one type of filter is applied to the image to perform the blur correction. In such a blur correction using only one type of filter, it is ideal that the size of the filter is infinite. However, in practice, both the size of the image and the filter are finite, and the size of the filter is increased. Then, since the processing amount increases, it is necessary to correct blur using a filter of a limited size. As described above, if the filter size is limited, an error occurs in the blur correction process depending on the imaged subject and the shape of the blur. In particular, in a portion where the change (contrast) of the pixel value is large, partial overcorrection occurs. There was a problem that it was easy and proper correction was not made.

  The present invention has been made to solve the conventional problems, and an object thereof is to provide a shake correction apparatus that can perform more appropriate shake correction with less overcorrection.

  A blur correction apparatus according to the present invention includes an image input unit for inputting an image, a blur detection unit for detecting a blur of the image, a pixel value detection unit for detecting a change direction and a change amount of a pixel value of the image, And a correction unit that performs blur correction processing on the image based on the correction amount adjusted according to the change direction and the change amount of the pixel value.

  According to this configuration, it is possible to reduce overcorrection at a place where overcorrection is likely to occur by adjusting the correction amount of blur correction according to the change direction and change amount of the pixel value. As a result, it is possible to provide a shake correction apparatus that performs more appropriate shake correction with less overcorrection.

  The shake correction apparatus according to the present invention further includes a correction adjustment unit that adjusts a correction amount based on a filter that corrects shake according to the change direction and the change amount of the pixel value.

  According to this configuration, it is possible to reduce overcorrection caused by the filter that corrects the shake by adjusting the correction amount of the shake correction by the filter. As a result, it is possible to provide a camera shake correction apparatus that performs more appropriate camera shake correction with less overcorrection in camera shake correction.

  In the shake correction apparatus according to the present invention, the correction adjustment unit adjusts the correction amount according to the change amount of the pixel value in the change direction of the pixel value substantially matching the blur direction.

  According to this configuration, the correction amount is adjusted in accordance with the change amount of the pixel value in the change direction of the pixel value corresponding to the blur direction, thereby causing an overcorrection that occurs when the change amount of the pixel value in the blur direction is large. Can be reduced. As a result, it is possible to provide a shake correction apparatus that can reduce over-correction at a place where a correction place is likely to be generated and performs more appropriate shake correction with less over-correction.

  In the shake correction apparatus according to the present invention, the correction adjustment means adjusts the correction amount according to the weight distribution of the filter coefficients.

  According to this configuration, by adjusting the correction amount according to the weight distribution of the filter coefficient, it is possible to reduce overcorrection caused by the bias of the filter coefficient. As a result, it is possible to provide a shake correction apparatus that can reduce over-correction at a place where a correction place is likely to be generated and performs more appropriate shake correction with less over-correction.

  The blur correction method according to the present invention inputs an image, detects the blur of the image, detects the change direction and the change amount of the pixel value of the image, and the correction adjusted according to the change direction and the change amount of the pixel value Based on the amount, blur correction processing is performed on the image.

  According to this configuration, it is possible to reduce overcorrection at a place where overcorrection is likely to occur by adjusting the correction amount of blur correction according to the change direction and change amount of the pixel value. As a result, it is possible to provide a shake correction method that performs more appropriate shake correction with less overcorrection.

  In the shake correction method according to the present invention, the correction amount based on the filter for correcting the shake is adjusted according to the change direction and the change amount of the pixel value.

  According to this configuration, it is possible to reduce overcorrection caused by the filter that corrects the shake by adjusting the correction amount of the shake correction by the filter. As a result, it is possible to provide a camera shake correction apparatus that performs more appropriate camera shake correction with less overcorrection in camera shake correction.

  In the shake correction method according to the present invention, the correction amount is adjusted according to the change amount of the pixel value in the change direction of the pixel value substantially matching the blur direction.

According to this configuration, the correction amount is adjusted in accordance with the change amount of the pixel value in the change direction of the pixel value corresponding to the blur direction, so that the overcorrection that occurs when the change amount of the pixel value in the blur direction is large Can be reduced. As a result, it is possible to provide a shake correction method that can reduce overcorrection at a place where a correction place is likely to occur, and perform more appropriate shake correction with less overcorrection.
In the shake correction method according to the present invention, the correction amount is adjusted according to the weight distribution of the filter coefficients.

  According to this configuration, by adjusting the correction amount according to the weight distribution of the filter coefficient, it is possible to reduce overcorrection caused by the bias of the filter coefficient. As a result, it is possible to provide a shake correction method that can reduce overcorrection at a place where a correction place is likely to occur, and perform more appropriate shake correction with less overcorrection.

  According to the present invention, when overcorrection of the blur correction process is likely to occur, the overcorrection is corrected by adjusting the correction amount for correcting the image according to the change direction and the change amount of the pixel value of the input image. It is possible to provide a shake correction apparatus that can obtain a smaller and more appropriate image with high correction accuracy.

(First embodiment)
Hereinafter, a shake correction apparatus and a shake correction method according to a first embodiment of the present invention will be described with reference to the drawings.

  FIG. 1 shows an example of a shake correction apparatus according to the present embodiment. In FIG. 1, a shake correction apparatus according to the present embodiment includes an imaging unit 101, a signal processing unit 102, a pixel value detection unit 103, a shake detection unit 104, a filter creation unit 105, a correction adjustment unit 106, and a correction unit 107. Prepare.

  The imaging unit 101 images a subject and inputs a video signal of the subject image to the signal processing unit 102. The signal processing unit 102 performs general video signal processing such as white balance on the video signal input from the imaging unit 101. Here, the imaging means 101 and the signal processing means 102 are used as image input means. The shake detection unit 104 detects camera shake. The filter creation unit 105 creates a two-dimensional filter that corrects the shake based on the direction and shape of the shake obtained by the shake detection unit 104.

  The pixel value detection unit 103 detects the change direction and the change amount of the pixel value (for example, the luminance or saturation of the image) at a location on the image that is being processed. That is, the pixel value detection unit 103 detects the contrast direction and amount of the image.

  The correction adjustment unit 106 outputs a value for adjusting the correction amount in the blur correction process according to the change direction and the change amount of the pixel value. In the present embodiment, the correction adjustment unit 106 performs blur correction processing according to the change direction and amount of the pixel value detected by the pixel value detection unit 103 and the blur direction detected by the shake detection unit 104. Adjust the correction amount. The blur direction may be determined based on a filter created by the filter creation unit 105. In this case, the correction adjustment unit 106 adjusts the correction amount in the blur correction process by comparing the change direction and the change amount of the pixel value with the filter.

The correction unit 107 performs correction processing on the image input from the image input unit based on a filter that corrects the shake detected by the shake detection unit 104. In the present embodiment, the correction unit 107 performs shake correction processing (filter processing) on the input image based on the filter created by the filter creation unit 105. At this time, the correction amount by the filter is adjusted by the correction adjustment unit 106, and the correction unit 107 performs correction processing based on the adjusted correction amount to create a corrected image.
The operation of the shake correction apparatus configured as described above will be described with reference to the flowchart of FIG.

  First, in step S100, an image is input. In the present embodiment, a subject is imaged by the imaging means 101, and the obtained image is input as a video signal. The signal processing unit 102 performs general processing on the image such as white balance and brightness adjustment on the video signal.

  In step S101, image blur is detected. In the present embodiment, the shake detection unit 104 detects the direction and shape of the shake when the subject is imaged by the imaging unit 101 and how the camera is shaken. As specific blur detection methods, there are a method using a gyro sensor, a method for obtaining blur from a photographed image, and the like. However, the shake detection method is not limited to these. Then, as shown in FIG. 3, the shake detection unit 104 acquires a point spread function (PSF) based on the detected shake. The point spread function is a function indicating how the pixel value of an arbitrary point is diffused due to the blurring at the time of photographing.

  In step S102, a filter for correcting blur is created. In the present embodiment, the filter creation means 105 creates a filter. The filter creation means 105 creates a two-dimensional filter based on the point spread function. For example, the filter is configured by a convolution filter, a Wiener filter, or the like. However, the filter creation method is not limited to these.

  Conventionally, since the filter is applied to the image without adjusting the correction amount, partial overcorrection has occurred. One cause of overcorrection is the problem of filter size. Usually, in the blur correction using a filter, it is ideal that the size of the image and the filter is infinite. However, since the size of the filter affects the processing time and the scale of the circuit, the size of the filter is small. Often limited. For this reason, there has been a problem that an error in the correction amount occurs due to the limitation of the size of the filter. For example, because the size of the filter is limited, in the part where the calculation of the blur correction process is terminated, it is assumed that there is no blur, but the blur is not generated. Correction processing is performed. As a result, if the pixel value varies greatly depending on the presence or absence of blur, an error occurs in the blur correction amount.

  Specifically, the occurrence of overcorrection will be described. As shown in FIG. 4, in the blurred image, pixel values of surrounding pixels are mixed due to the blur. Here, as shown in FIG. 4A, when the pixel values of the target pixel and the peripheral pixels are approximate, the pixel values of the image are not so different regardless of the presence or absence of blurring. That is, even if the pixel values of the surrounding pixels are mixed, the error in the amount of blur correction is small. On the other hand, as shown in FIG. 4B, when the amount of change in the pixel value of the pixel of interest with respect to the pixel values of the surrounding pixels is large, the pixel value varies greatly depending on the presence or absence of blurring. In such a case, if the calculation for the blur correction process is terminated, the blur correction process is performed ignoring the difference in pixel value due to the presence / absence of the blur, and the difference affects the correction amount. There is an error in the quantity. For example, at a location where the amount of change in luminance is large on the image, an error in the correction amount is likely to occur if blurring occurs in the direction in which the amount of change in luminance is large. That is, overcorrection is caused by the amount of change in the pixel value in the change direction of the pixel value that matches the blur direction.

  In this embodiment, in order to adjust the error as described above, the change direction and change amount of the pixel value of the image are detected, and the correction amount is adjusted according to the change direction and change amount of the pixel value.

  Specifically, in step S103, the pixel value detection unit 103 detects the change direction and the change amount of the pixel value. For example, the pixel value detection unit 103 detects the change direction and the change amount of the luminance for the peripheral pixels at the location to be corrected. As a detection method, for example, as shown in FIG. 5, a method of obtaining the change direction and the change amount of the pixel value based on the horizontal pixel value and the vertical pixel value in the blur correction processing target region. is there. In addition, a method of obtaining the change direction and the change amount of the pixel value based on the pixel position where the pixel value in the processing target region becomes the maximum and the pixel position where the pixel value becomes the minimum may be adopted.

  In step S104, the correction adjustment unit 106 adjusts the correction amount for shake correction. The correction adjustment unit 106 receives information about the direction and change amount of the pixel value in the processing target area from the pixel value detection unit 103 and receives information about the shake direction from the shake detection unit 104. Information on the blur direction may be generated based on a filter created by the filter creation unit 105. In this case, the correction adjustment unit 106 receives information on the blur direction from the filter creation unit 105.

  The correction adjustment unit 106 determines whether or not overcorrection occurs in the blur correction processing target region based on the information on the change direction and amount of the pixel value and the information on the blur direction, and based on the result, the pixel value The correction amount of the filter created by the filter creation unit 105 is adjusted in accordance with the change direction and the change amount, and the adjusted correction amount is output to the correction unit 107. Then, the correction unit 107 creates an image in which blurring is corrected by applying a filter to the image based on the adjusted correction amount (that is, performing convolution).

Next, an example of a method for calculating a value for adjusting the correction amount will be described. In the shake correction processing in the correction unit 107, the pixel value of the image after correction is “P ′”, the pixel value of the image before correction is “P”, and the correction amount by the filter is “D”. Suppose that 1) holds.
P ′ = P + D (1)

In this case, the intensity of the correction amount D is adjusted by multiplying the correction amount D by setting the correction gain for adjusting the correction amount D to “G” as shown in the following equation (2).
P ′ = P + D × G (2)
The correction gain G is calculated by the correction adjustment unit 106 according to the change direction and the change amount of the pixel value.

  Next, an example of a method for determining the correction gain G will be described. As shown in FIG. 6, in order to suppress overcorrection in the processing target area to be corrected, the correction gain is determined by the magnitude of the change amount of the pixel value in the change direction of the pixel value substantially matching the blur direction. G is determined. For example, as illustrated in FIG. 6, when the change amount of the pixel value is in a section between “P1” and “P2”, the correction gain G is decreased as the change amount of the pixel value is increased. The correction gain obtained in this way is set as “G1”. That is, the correction amount is changed by the correction gain G1 according to the change amount of the pixel value in the change direction of the pixel value substantially matching the blur direction. Specifically, the correction adjustment unit 106 outputs a correction gain G1 according to the amount of change in the pixel value in the change direction of the pixel value that substantially matches the blur direction, and, as shown in Expression (2), The correction amount is adjusted by multiplying the correction amount by the correction gain G1.

  According to the shake correction apparatus and the shake correction method according to the present embodiment, by adjusting the correction amount according to the change amount of the pixel value in the change direction of the pixel value corresponding to the shake direction, the pixels in the shake direction are adjusted. The overcorrection that occurs when the amount of change in value is large can be reduced. As a result, it is possible to provide a shake correction apparatus that can reduce over-correction at a place where a correction place is likely to be generated and performs more appropriate shake correction with less over-correction.

(Second Embodiment)
Hereinafter, a shake correction apparatus and a shake correction method according to the second embodiment of the present invention will be described. However, the shake correction apparatus according to the second embodiment has the same configuration as the shake correction apparatus according to the first embodiment as shown in FIG. To do. Further, the shake correction method according to the second embodiment has the same steps as the shake correction method according to the first embodiment as shown in FIG. To do.

  In this embodiment, the amount of blur correction is adjusted according to the weight distribution of the filter coefficients. In other words, the cause of overcorrection in blur correction is the bias of the filter coefficient. In this embodiment, the correction gain considering the bias of the filter coefficient is set to “G2”, and correction is performed by the correction gain G2. Adjust the amount.

  Specifically, the occurrence of overcorrection will be described. Usually, when applying a filter, in order to suppress an increase in noise, the filter is often applied around a portion having the maximum coefficient of the filter, but the portion having the maximum coefficient does not always coincide with the center of gravity of the filter. Here, the center of gravity of the filter is a place where the weights of the coefficients of the filter are balanced.

  If the location of the maximum coefficient matches the center of gravity of the filter, the overcorrection is small. On the other hand, when the location of the maximum coefficient does not coincide with the center of gravity of the filter, the deviation between the location of the maximum coefficient and the center of gravity of the filter affects the occurrence of overcorrection.

  Next, with reference to FIG. 7, the overcorrection caused by the difference between the location of the maximum coefficient of the filter and the center of gravity of the filter will be described. As shown in FIG. 7A, the maximum coefficient of the filter is in the vicinity of the center of the filter, and the weights of the coefficients on both sides of the maximum coefficient are substantially balanced (the position of the maximum coefficient matches the center of gravity of the filter). If so, overcorrection is small. On the other hand, as shown in FIG. 7B, the maximum coefficient of the filter is near the end of the filter, and the weight of the coefficient on both sides of the maximum coefficient is biased to one side (the position of the maximum coefficient matches the center of gravity of the filter) If not, overcorrection will increase.

  In order to adjust the influence of overcorrection caused by the difference between the location of the maximum coefficient of the filter coefficient and the center of gravity of the filter, the weight distribution of the filter coefficient is detected in the blur direction, and the filter The correction gain G2 is determined according to the magnitude of the coefficient bias. That is, the correction gain G2 is calculated according to the weight distribution of the filter coefficients, and the correction amount in the change direction of the pixel value substantially matching the blur direction is changed. Specifically, the correction adjustment unit 106 outputs a correction gain G2 for adjusting the correction amount according to the weight distribution of the filter coefficients, and adjusts the correction amount according to the correction gain G2.

An example of a method for determining the correction gain G2 will be described. The correction adjustment unit 106 calculates the correction gain G2 depending on how much the position of the maximum coefficient of the filter is biased in the blur direction. For example, as shown in FIG. 8, in the case where there is a shake in the vertical direction, the deviation of the location of the maximum coefficient in the vertical direction is detected. Specifically, the upper total “m” and the lower total “n” of the coefficients of the filter are obtained on the basis of the location of the maximum coefficient, and the total of the coefficients “m” is obtained as shown in the following formula (3). By calculating the ratio “R” of “” and “n”, the difference between the location of the maximum coefficient and the center of gravity is detected.
R = m / n (m <n)
R = n / m (m ≧ n) (3)

  According to Equation (3), it is determined that the degree of divergence is smaller as the ratio R is closer to “1”, and the degree of divergence is larger as the ratio R is closer to “0”. Thus, the deviation between the location of the maximum coefficient of the filter coefficient and the center of gravity is determined based on the ratio R, and when the degree of deviation is large, the correction gain G2 is reduced.

  In the above example, the correction gain G2 is calculated based on the ratio of the total coefficients of the filter with the location of the maximum coefficient as a reference. However, as another example, the location of the center of gravity of the filter is obtained, and this is the location of the maximum coefficient. May be detected, and the correction gain G2 may be calculated based on the degree of the deviation.

  In the above example, the correction gain G2 is calculated based on the difference between the location of the maximum coefficient of the filter and the center of gravity of the filter. However, the difference between the location of the filter coefficient corresponding to the maximum coefficient of the PSF and the center of gravity of the filter is calculated. The correction gain G2 may be calculated based on the above.

According to the shake correction apparatus and the shake correction method according to the present embodiment, it is possible to reduce overcorrection caused by the bias of the filter coefficient by adjusting the correction amount according to the weight distribution of the filter coefficient. . As a result, it is possible to provide a shake correction apparatus that can reduce overcorrection at a place where a correction place is likely to be generated, and perform more appropriate shake correction with less overcorrection.
(Other embodiments)

As described above, in the first and second embodiments, the correction gain G1 and the correction gain G2 are calculated. As another embodiment, as shown in the following equation (4), the correction gain G1 is calculated. The correction gain G may be obtained by multiplying G1 and the correction gain G2.
G = G1 × G2 (4)

  In this case, the correction unit 107 performs shake correction on the image with the correction amount adjusted by the correction gain G shown in Expression (4). As a result, the overcorrection caused by both the change amount of the pixel value and the weight distribution of the coefficient of the filter can be reduced, and a more appropriate correction result can be obtained.

Further, as shown in the following equation (5), the correction amount may be adjusted by providing a clip value C that limits the range of the correction amount D.
−C ≦ D ≦ C (5)

According to the equation (5), the correction amount D is clipped by the clip value C, so that the adjustment of the correction amount is limited so as not to be excessive. As a result, a more appropriate correction result can be obtained.
Further, the adjustment of the correction amount may be performed by a method of partially increasing or decreasing the filter coefficient itself.

  In the first embodiment, as shown in FIG. 1, a means for performing blur correction processing such as the correction means 107 is disposed after the signal processing means 102 for performing general image signal processing. Accordingly, a means for performing the shake correction process may be arranged in the preceding stage of the signal processing means 102. Further, the means for performing the blur correction process may be implemented as a part of the process of the signal processing means 102.

  Further, in the first embodiment, the input image is obtained by the image pickup means 101 and the signal processing means 102 which are image input means. However, instead of these image input means, a photograph was taken. It is also possible to adopt a configuration in which blur correction processing is performed on an image read from a recording medium that stores the image.

  In the first embodiment, the filter creation unit 105 creates a two-dimensional filter that corrects the shake based on the direction and shape of the shake obtained by the shake detection unit 104. May selectively read out a desired filter from a recording medium storing the filter.

  As described above, the shake correction apparatus according to the present invention has an effect of obtaining a more appropriate correction result with less overcorrection, and is an image pickup apparatus that performs shake correction on an image deteriorated due to camera shake or the like. Useful.

1 is a block diagram showing an example of a shake correction apparatus according to the present invention. The flowchart which shows an example of the blurring correction method based on this invention Diagram showing an example of a point spread function caused by blurring The figure which shows an example of the difference in the contrast of the image by the presence or absence of blurring The figure which shows an example of the change direction and change amount of the contrast of an image The figure which shows an example of the relationship between pixel value variation | change_quantity and a correction gain. The figure which shows an example of the overcorrection by the bias of the coefficient of a filter The figure which shows an example of the bias | inclination of the coefficient of the filter in a blurring direction

Explanation of symbols

DESCRIPTION OF SYMBOLS 101 Image pickup means 102 Signal processing means 103 Pixel value detection means 104 Blur detection means 105 Correction filter creation means 106 Correction adjustment means 107 Correction means

Claims (8)

An image input means for inputting an image;
Blur detection means for detecting blur of the image;
Pixel value detection means for detecting a change direction and a change amount of a pixel value of the image;
Correction means for performing the blur correction processing on the image based on a correction amount adjusted according to a change direction and a change amount of the pixel value;
A shake correction apparatus comprising:   The shake correction apparatus according to claim 1, further comprising a correction adjustment unit that adjusts the correction amount based on a filter that corrects the shake according to a change direction and a change amount of the pixel value.   The correction adjustment unit adjusts the correction amount according to a change amount of the pixel value in a change direction of the pixel value substantially matching the blur direction. Blur correction device.   The shake correction apparatus according to claim 2, wherein the correction adjustment unit adjusts the correction amount according to a weight distribution of the coefficient of the filter. Enter an image,
Detecting blur of the image,
Detecting a change direction and a change amount of a pixel value of the image;
A blur correction method, wherein the blur correction process is performed on the image based on a correction amount adjusted according to a change direction and a change amount of the pixel value.   The blur correction method according to claim 5, wherein the correction amount based on a filter that corrects the blur is adjusted according to a change direction and a change amount of the pixel value.   The blur correction method according to claim 6, wherein the correction amount is adjusted according to a change amount of the pixel value in a change direction of the pixel value substantially coincident with the blur direction.   The blur correction method according to claim 6, wherein the correction amount is adjusted in accordance with a weight distribution of the coefficient of the filter.

Images