JP2010079815A - Image correction device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image correction device, capable of performing correction processing of an input image without using a camera shake sensor. <P>SOLUTION: The image correction device includes a maximum local difference value determination means 2 which determines, from the input image, a window area maximum local difference value for each coordinate of a pixel unit showing a relative position of a near pixel to a target pixel; and a gradient calculation means 3 which determines a difference between window area maximum local difference values as a maximum local difference gradient value. The device further includes a filter range determination means 4 which determines, among the maximum local difference gradient values, a range of coordinates corresponding to maximum local difference gradient values of a predetermined value or more as a filter factor existing range; a filter factor determination means 5 which calculates a filter factor in the filter factor existing range; and a correction means 102 which performs reverse processing of filter processing to the input image by use of the filter factor. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to an image correction apparatus applied to an image photographing apparatus and an image display apparatus such as a digital camera, a printer, a personal computer, and a television. Is.

  Conventionally, various functions have been proposed for correcting or repairing image degradation due to aberrations such as aperture value, focal length, and focus, and image degradation due to camera shake in a digital camera or camera-equipped mobile terminal device. For example, as described in Patent Document 1, a recent digital camera having an optical system such as a lens and an image pickup device such as a CCD or C-MOS sensor has a vibration of the optical system in order to correct camera shake. It has a mechanical mechanism to reduce In addition, as described in Patent Document 2, a technique for correcting an image using an arithmetic circuit that converts image data of an image captured by an image sensor has been proposed.

  In the above prior art, when the camera vibrates due to camera shake, the vibration is detected by a sensor, and a correction amount is calculated based on the detected signal, that is, the moving speed of the camera due to camera shake. Then, based on the calculated correction amount, the optical lens and / or the image sensor is moved, or the value of each pixel of the image sensor is corrected by image processing calculation, thereby correcting image degradation caused by camera shake or To prevent. As a result, an image in which image degradation due to camera shake is corrected or an image in which image degradation is prevented is recorded in a storage medium such as a flash memory.

JP 2001-188272 A (page 7, FIG. 1) JP 2000-224461 A (pages 8 to 9, FIGS. 9 to 10)

  However, the above-described conventional technique has a problem that the configuration of the apparatus is complicated because a camera shake sensor for detecting the moving speed of the camera at the time of shooting is required. In addition, it is necessary to correct image degradation based on the output of the camera shake sensor. However, if a camera shake image is input to an image display device such as a printer or a personal computer that is not linked to the imaging device, the image cannot be corrected. There was a problem.

  The present invention has been made to solve the above-described problems, and an object thereof is to provide an image correction apparatus capable of performing an input image correction process without a camera shake sensor.

  The image correction apparatus according to the present invention includes, for each target pixel in a difference value determination region including all or part of an image region of an input image, image data of the target pixel and a window set around the target pixel. A local difference value consisting of a difference from the image data of neighboring pixels in the region is calculated for each pixel coordinate indicating a relative position of the neighboring pixel with respect to the target pixel, and all the attentions in the difference value determination region are calculated. A maximum local difference value determining unit that determines a maximum value of the local difference value for each coordinate relating to a pixel as a window region maximum local difference value for each coordinate is provided. A gradient calculating unit that determines, for each coordinate, a difference between the window region maximum local difference values adjacent to each other in at least one of a horizontal direction or a vertical direction of the coordinates; and a gradient calculating unit And a filter range determining means for obtaining, as a filter coefficient existence range, a range of the coordinates corresponding to the maximum local difference gradient value not less than a predetermined value among the maximum local difference gradient values determined in (1). Then, based on the maximum local difference gradient value in the filter coefficient existence range determined by the filter range decision means, filter coefficient determination means for calculating a filter coefficient in the filter coefficient existence range, and the filter coefficient calculation Correction means for performing reverse processing of filter processing on the input image using the filter coefficient calculated by the means.

  According to the image correction apparatus of the present invention, the filter coefficient representing the degradation of the input image is calculated only from the input image, and the input image is corrected using the filter coefficient. Therefore, the input image correction process can be performed without a camera shake sensor.

<Embodiment 1>
FIG. 1 is a block diagram showing the configuration of the image correction apparatus according to the present embodiment. As shown in the figure, the image correction apparatus according to the present embodiment includes a storage unit 100, a maximum local difference value determination unit 2, a gradient calculation unit 3, a filter range determination unit 4, and a filter coefficient determination unit 5. , A correction unit 102 and a storage unit 103. The maximum local difference value determining means 2, the gradient calculating means 3, the filter range determining means 4, and the filter coefficient determining means 5 constitute a deterioration estimating means 101.

  An image is stored in the storage unit 100. The image output from the storage unit 100 is input to the maximum local difference value determination unit 2 as an input image. The maximum local difference value determining unit 2 obtains the window region maximum local difference value of the input image output from the storage unit 100. The gradient calculating unit 3 obtains the maximum local difference gradient value from the window region maximum local difference value output from the maximum local difference value determining unit 2. The filter range determination unit 4 obtains the filter coefficient existence range from the maximum local difference gradient value output from the gradient calculation unit 3.

  The filter coefficient determination unit 5 obtains a filter coefficient from the maximum local difference gradient value output from the gradient calculation unit 3 and the filter coefficient existence range output from the filter range determination unit 4. The correction unit 102 generates a corrected image by correcting the input image output from the storage unit 100 using the filter coefficient output from the filter coefficient determination unit 5. The storage unit 103 stores the corrected image output from the correction unit 102.

  Next, the basic operation will be described. In general, it is considered that deterioration such as camera shake and defocus can be expressed linearly. That is, image data of a degraded input image is y (i, j) (i, j; coordinates in pixel units, 0 ≦ i <I, 0 ≦ j <J). When the filter coefficient is a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M), and image data without deterioration is x (i, j), the relationship of the following equation (1) is established.

  The degradation estimation unit 101 of the image correction apparatus according to the present embodiment is based on image data y (i, j) of the input image from the storage unit 100 (hereinafter sometimes referred to as input image y (i, j)). The filter coefficient a (n, m) representing the degradation of the input image is estimated. Thereafter, the correction unit 102 of the image correction apparatus according to the present embodiment uses the filter coefficient a (n, m) within a predetermined range (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M). The reverse process of the filter process is performed on the input image y (i, j). In this way, the image correction apparatus according to the present embodiment corrects image data (hereinafter referred to as corrected image w) corresponding to image data (hereinafter also referred to as image x (i, j)) without deterioration. (I, j) may be obtained).

Hereinafter, the detailed operation of the image correction apparatus according to the present embodiment will be described with reference to FIG. First, the storage unit 100 stores an input image y (i, j) (0 ≦ i <I, 0 ≦ j <J). The input image y (i, j) read from the storage unit 100 is input to the maximum local difference value determination unit 2. As shown in FIG. 2, the maximum local difference value determining means 2 according to the present embodiment is a part of an image area 201 of an input image of I pixels × J pixels, for example, a central area (Ic Pixel × Jc pixel) is defined as the difference value determination area 202. The maximum local difference value determination unit 2 calculates a local difference value d _{(i, j)} (n, m) represented by the following equation (2) for each target pixel 203 in the difference value determination area 202.

  In this equation (2), y (i, j) (Is ≦ i ≦ Is + Ic, Js ≦ j ≦ Js + Jc) is image data for the target pixel 203. As shown in FIG. 3, the window area 204 is set in advance around each pixel of interest 203. Y (i + n, j + m) according to Expression (2) is image data of the neighboring pixel 205 in the window region 204. The coordinates (n, m) are pixel unit coordinates indicating the relative position of the neighboring pixel 205 with respect to the target pixel 203.

  The window area 204 shown in FIG. 3 is set so as to spread horizontally (−N to N) in the horizontal direction with the target pixel 203 as a reference (0, 0), and upward (0 to M) in the vertical direction. It is set to spread only. The window area 204 according to the present embodiment includes first and second window areas adjacent to each other in the target pixel 203. In the present embodiment, the first window region is a window region 204a (−N ≦ n ≦ 0, 0 ≦ m ≦ M) on the left side from the center line of the window region 204, and the second window region is a window It is assumed that the window region 204b on the right side of the center line of the region 204 (0 ≦ n ≦ N, 0 ≦ m ≦ M). As will be described later, the size of the window area 204 affects the existence range of the filter coefficient. Therefore, the system to which the image correction apparatus is connected, the range where deterioration such as camera shake can occur, or the deterioration to be corrected. Set according to the range.

As described above, the maximum local difference value determining unit 2 determines the image data y (i, j of the target pixel 203 for each target pixel 203 in the difference value determination area 202 that is a part of the image area 201 of the input image. ) And the local difference value d _{(i, j)} (n, m), which is the difference between the image data y (i + n, j + m) of the neighboring pixel 205 in the window region 204 set around the pixel of interest 203. calculate. The maximum local difference value determination unit 2 calculates the local difference value d _{(i, j)} (n, m) for each pixel coordinate (n, m) indicating the relative position of the neighboring pixel 205 with respect to the target pixel 203. To do.

Next, the maximum local difference value determining unit 2 determines the local difference value d _{(i, j} ) for each coordinate (n, m) related to all the target pixels 203 in the difference value determining area 202 as shown in the following equation (3). determined as n ≦ n ≦ n, 0 ≦ m ≦ M) -) (n, the maximum value of m), coordinates (n, m) for each of the window regions maximum local difference value D (n, m) (.

  In the present embodiment, the window region 204 is set in the horizontal direction as −N ≦ n ≦ N in the horizontal direction, but is set only in the upward direction as 0 ≦ m ≦ M in the vertical direction. The reason for this setting is basically D (n, m) = D (−n, −m), and therefore the upper half, the lower half, or the left half of the region centered on the target pixel 203. Alternatively, it is sufficient to obtain one of the right half areas.

  The window region maximum local difference value D (n, m) thus determined is, for example, D (n) using a predetermined threshold α (> 0) when the input image is a natural image without deterioration such as camera shake. , M)> α (where (n, m) ≠ (0,0)). For example, when the input image is not deteriorated outside the range of the pixel (N ′, M ′), that is, in the expression (1), a (n, m) = 0 (n> N ′ or m> M ′ ) Is characterized by D (n, m)> α (n <−N ′ or n> N ′ or m> M ′).

  Further, for example, when the input image is deteriorated due to camera shake only in the horizontal direction, that is, in equation (1), a (n, m) = 0 (0 ≦ n ≦ N, m> 0), a (n , 0)> 0 (0 ≦ n ≦ N), the window region maximum local difference value D (n, m) is D (n, m)> α (0 ≦ n ≦ N, m> 0). . The inventor considered the window region maximum local difference value D (n, m) having the above characteristics. As a result, the gradient value h (n) (= D (n + 1,0) −D (n, 0)) consisting of the difference between the window region maximum local difference values adjacent in the horizontal direction of the coordinate (n, 0) is It has been found that there is a tendency to be approximately proportional to the filter coefficient a (n, 0).

  Actually, based on an image x (i, j), that is, a natural image without camera shake, N = 4, M = 0, a (n, 0) = 0.2 (0 ≦ n ≦ N), An input image y (i, j) having degradation in the horizontal direction is created by the equation (1), and the window region maximum local difference value D (n, m) is obtained. The results of D at that time are as follows: D (0,0) = 0, D (1,0) = 43, D (2,0) = 82, D (3,0) = 116, D (4,0) = 149, D (5,0) = 178, D (6,0) = 179, D (n, 0) ≧ 179 (n> 6).

  When the gradient value h (n) (= D (n + 1,0) −D (n, 0)) comprising these differences is obtained, h (0) = 43, h (1) = 39, h (2) = 34, h (3) = 33, h (4) = 29, h (5) = 1, h (n) <2 (n> 6). Thus, when the filter coefficient a (n, 0) in the expression (1) is a constant value (0.2 in the above example), the gradient values h (0) to h (4) are also substantially constant. It became the value of.

  Next, based on the natural image x (i, j) with no camera shake, N = 4 and M = 0, and an input image y (i, j) having a degradation in the horizontal direction is created by Expression (1). The window region maximum local difference value D (n, m) was obtained. However, as described above, the filter coefficient a (n, 0) in the equation (1) is not set to a constant value, but a (0, 0) = 3/11, a (1,0) = 2/11, a (2, 0) = 1/11, a (3, 0) = 2/11, a (4, 0) = 3/11. That is, specific gravity of large, medium, small, medium, and large was given in order of the value of n. The results of D at that time are as follows: D (0,0) = 0, D (1,0) = 55, D (2,0) = 88, D (3,0) = 104, D (4,0) = 137, D (5,0) = 176, D (6,0) = 179, D (n, 0) ≧ 179 (n> 6).

  When the gradient value h (n) consisting of the difference is obtained, h (0) = 55, h (1) = 33, h (2) = 16, h (3) = 33, h (4) = 39, h (5) = 3, h (n) <2 (n> 6), and the specific gravity was large, medium, small, medium, and large in the order of the value of n. As described above, the gradient values h (0) to h (4) also change in accordance with the filter coefficient a (n, 0) of the equation (1).

  From the above, although there is a slight error, if there is a horizontal hand shake, the filter coefficient of the hand shake is calculated from the gradient value h (n) (= D (n + 1,0) −D (n, 0)). a (n, m) can be estimated.

  Similarly, for example, when the input image is deteriorated due to camera shake only in the vertical direction, that is, in equation (1), a (n, m) = 0 (n> 0), a (0, m)> 0 In the case of (0 ≦ m ≦ M), the window region maximum local difference value D (n, m) is D (n, m)> α (n> 0). The inventor makes a gradient value h (m) (= D (0, m + 1) −D (0, m)) consisting of a difference between window region maximum local difference values adjacent in the vertical direction of the coordinates (0, m). I also investigated. As a result, it was found that the gradient value h (m) tends to be substantially proportional to the filter coefficient a (0, m), as described above. Due to such a tendency, when there is camera shake in the vertical direction, the filter coefficient a (n, m) of camera shake is calculated from the gradient value h (m) (= D (0, m + 1) −D (0, m)). Can be estimated.

  Note that, as described above, the gradient coefficient h, which is the difference between the window region maximum local difference values D (n, m) adjacent in the horizontal direction or the vertical direction of the coordinates, represents the filter coefficient representing the horizontal or vertical camera shake. Proportional to a (n, m) can be explained by modeling an image x (i, j) having no deterioration as an image having discrete transformation points.

  As described above, the filter coefficient a (n, m) can be estimated. Therefore, the gradient calculating means 3 according to the present embodiment calculates the difference between the window region maximum local difference values D (n, m) adjacent in at least one direction of the coordinate (n, m) in the horizontal direction or the vertical direction. The maximum local differential gradient value h (n, m) is determined and output for each coordinate (n, m).

  Next, the configuration of the gradient calculating means 3 that performs such an operation will be described. FIG. 4 is a block diagram illustrating a configuration example of the gradient calculating unit 3. As shown in the figure, the gradient calculation unit 3 includes an input image evaluation unit 301, a region selection unit 302, a gradient calculation range determination unit 303, a subtraction unit 304, and a maximum value selection unit 305. Next, the operation of each unit included in the gradient calculation unit 3 will be described with reference to FIG.

  First, the window region maximum local difference value D (n, m) (−N ≦ n ≦ N, 0 ≦ m ≦ M) output from the maximum local difference value determination unit 2 is input to the input image evaluation unit 301. . Based on the window region maximum local difference value D (n, m) (−N ≦ n ≦ N, 0 ≦ m ≦ M) determined by the maximum local difference value determining unit 2, the input image evaluation unit 301 A predetermined threshold value α for estimating the range is determined.

  The predetermined threshold α is, for example, a maximum value Dmax of D (n, m) (−N ≦ n ≦ N, 0 ≦ m ≦ M) and a predetermined constant γ (0 <γ ≦ 1). , Α = Dmax · γ may be determined. Alternatively, the average value and standard deviation of D (n, m) (−N ≦ n ≦ N, 0 ≦ m ≦ M) may be Dave and Dσ, respectively, and α = Dave−Dσ.

  The gradient calculation means 3 according to the present embodiment includes a window region maximum local difference value D (n, m) in the left window region 204a and a window region maximum local difference value D (n, m) in the right window region 204b. m), either the left window area 204a or the right window area 204b is selected. In the present embodiment, the region selection unit 302 included in the gradient calculation unit 3 is based on the window region maximum local difference value in the left window region 204a and the window region maximum local difference value in the right window region 204b. Then, a window region having a strong deterioration characteristic is selected from the left window region 204a or the right window region 204b.

  Specifically, for example, the region selection unit 302 has the maximum absolute value of the horizontal coordinate n of the window region maximum local difference value D (n, m) that is smaller than the predetermined threshold value α output from the input image evaluation unit 301. The values are compared for the left window area 204a and the right window area 204b. That is, n_min = Max {n; D (−n, m) <α, 0 ≦ n ≦ N, 0 ≦ m ≦ M} and n_max = Max {n; D (n, m) <α, 0 ≦ n Compare ≦ N, 0 ≦ m ≦ M}. If n_max> n_min, the right window region 204b (0 ≦ n ≦ N, 0 ≦ m ≦ M) is selected as shown in FIG. On the other hand, if not, as shown in FIG. 5B, the left window region 204a (−N ≦ n ≦ 0, 0 ≦ m ≦ M) is selected.

  Next, the gradient calculation range determination unit 303 determines the maximum local difference gradient value h (() based on the window region maximum local difference value D (n, m) and the predetermined threshold value α determined by the input image evaluation unit 301. The range of coordinates (n, m) from which n, m) is to be calculated is determined. In the present embodiment, the ranges of the window areas 204 a and b selected by the area selection unit 302 are output to the gradient calculation range determination unit 303. Then, the gradient calculation range determination unit 303 according to the present embodiment performs the following operation on the window region maximum local difference value D (n, m) of the region selected by the region selection unit 302. In the following, the range of coordinates (n, m) determined by the gradient calculation range determination unit 303 may be referred to as a gradient calculation range.

  The gradient calculation range determination unit 303 obtains the maximum value of the vertical coordinates m of the window region maximum local difference value D (n, m) that is smaller than the predetermined threshold value α output from the input image evaluation unit 301. That is, m_max = Max {m; D (n, m) <α, −N ≦ n ≦ N, 0 ≦ m ≦ M} is obtained. The gradient calculation range determination unit 303 then gradients the range of coordinates (n, m) (0 ≦ n ≦ n_max, 0 ≦ m ≦ m_max) when the region selection unit 302 selects the right window region 204b. The calculation range. On the other hand, when the region selection unit 302 selects the left window region 204a, the gradient calculation range determination unit 303 calculates the range (−n_min ≦ n ≦ 0, 0 ≦ m ≦ m_max) of the coordinates (n, m). The gradient calculation range.

  The subtracting means 304 is a window area adjacent in the horizontal direction and the vertical direction of coordinates (n, m) within the gradient calculation range determined by the gradient calculation range determination means 303 among the window areas selected by the area selection means 302. A gradient value consisting of a difference between the maximum local difference values D (n, m) is calculated. That is, when the gradient calculation range is 0 ≦ n ≦ n_max and 0 ≦ m ≦ m_max, the horizontal gradient value dh (n, m) (= D (n + 1, m) −D (n, m) ) And the vertical gradient value dv (n, m) (= D (n, m + 1) −D (n, m)). When the gradient calculation range is −n_min ≦ n ≦ 0 and 0 ≦ m ≦ m_max, the horizontal gradient value dh (n, m) (= D (n−1, m) −D (n, m )) And the vertical gradient value dv (n, m) (= D (n, m + 1) -D (n, m)).

  The maximum value selection unit 305 selects and obtains the larger one of the two gradient values dh (n, m) and dv (n, m) output from the subtraction unit 304. That is, h ′ (n, m) = Max {dh (n, m), dv (n, m)} is obtained. For convenience of display, when the gradient calculation range is 0 ≦ n ≦ n_max and 0 ≦ m ≦ m_max, it is replaced with h represented by the following equation (4), and the gradient calculation range is −n_min ≦ n. When ≦ 0 and 0 ≦ m ≦ m_max, the following description will be made by replacing with h represented by the following formula (5). In the following equation (4), n_lim = n_max, and in the following equation (5), n_lim = n_min.

  The maximum value selection unit 305 outputs h (n, m) (0 ≦ n ≦ n_lim, 0 ≦ m ≦ m_max) obtained as described above as the maximum local difference gradient value. Summarizing the above, the gradient calculating means 3 according to the present embodiment calculates the difference between the window region maximum local difference values D (n, m) adjacent in at least one direction of the coordinate (n, m) in the horizontal direction or the vertical direction. The difference is determined and output for each predetermined coordinate (n, m) as the maximum local difference gradient value h (n, m). In addition, the gradient calculation unit 3 according to the present embodiment calculates the maximum local difference gradient value h (n, m) only for the coordinates in the window regions 204a, b selected by the region selection unit 302. Further, the gradient calculation means 3 according to the present embodiment calculates the maximum local difference gradient value h (n, m) only for the coordinates within the range determined by the gradient calculation range determination means 303.

  Thus, the maximum local difference gradient value h (n, m) output from the gradient calculating means 3 is the window region maximum local difference value D adjacent in the horizontal or vertical direction of the coordinates (n, m) as described above. The difference value is (n, m). Therefore, the value of the maximum local difference gradient value h (n, m) is proportional to the value of the filter coefficient a (n, m) representing the deterioration of the input image y (i, j), as exemplified above. Conceivable. However, in the previous example, as h (5), h (n) (n> 6) have a minute value, the input image y (i, j) with actual deterioration has a filter. Even if the coefficient a (n, m) is outside the existing range, the maximum local difference gradient value h (n, m) may not be completely zero.

  Therefore, the filter range determination unit 4 according to FIG. 1 has a maximum local difference equal to or greater than a predetermined value ε (≧ 0) among the maximum local difference gradient values h (n, m) determined by the gradient calculation unit 3. A range of coordinates (n, m) corresponding to the gradient value is obtained as a filter coefficient existence range. That is, the range S obtained by the following equation (6) is output as the filter coefficient existence range.

  The filter coefficient determination means 5 according to FIG. 1 is based on the maximum local difference gradient value h (n, m) ((n, m) εS) in the filter coefficient existence range S determined by the filter range determination means 4. Then, the filter coefficient a (n, m) within the filter coefficient existence range S is calculated. In the present embodiment, the filter coefficient determining means 5 includes the maximum local difference gradient value h (n, m) output from the gradient calculating means 3 and the filter coefficient existence range S output from the filter range determining means 4. Based on this, the constant A is obtained from the following equation (7).

  Then, the filter coefficient determination means 5 calculates the filter coefficient existence based on the maximum local difference gradient value h (n, m) in the filter coefficient existence range S and the constant A from the upper expression of the following expression (8). A filter coefficient a (n, m) ((n, m) εS) within the range S is calculated. Then, the filter coefficient determination means 5 replaces the filter coefficients in the remaining range as shown in the lower expression of the following expression (8), thereby obtaining all the filter coefficients a (n, m in the predetermined range. ) (0 ≦ n ≦ N, 0 ≦ m ≦ M) is determined.

  In the above description, the filter range determination unit 4 calculates the filter coefficient existence range S at the coordinates (n, m) where h (n, m) ≧ ε, but the present invention is not limited to this. For example, the filter range determination means 4 replaces the value with h (n, m) = 0 for the maximum local difference gradient value h (n, m) where h (n, m) <ε, and the filter coefficient exists. The range S may be output as S = {(n, m; 0 ≦ n ≦ n_lim, 0 ≦ m ≦ m_max)}. Also in this case, the filter coefficient a (n, m) output from the filter coefficient determining means 5 has the same result as the above-described configuration.

  The degradation estimation means 101 including the maximum local difference value determination means 2, the gradient calculation means 3, the filter range determination means 4, and the filter coefficient determination means 5 described above is the input image y (i, j). From (0 ≦ i <I, 0 ≦ j <J), the filter coefficient a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) of the linear filter representing the degradation of the input image is output.

  The filter coefficient a (n, m) output from the filter coefficient determination unit 5 provided in the deterioration estimation unit 101 is input to the correction unit 102. The correction unit 102 according to FIG. 1 performs an inverse process of the filter process on the input image y (i, j) using the filter coefficient a (n, m), and outputs it as a corrected image w (i, j). The inverse processing of the filter processing is generally called deconvolution, and for example, a method such as a Richardson-Lucy algorithm or a Wiener filter can be used.

  The corrected image w (i, j) output from the correction unit 102 is written in the storage unit 103. Note that the storage unit 103 may share a storage device such as a hard disk or memory that is physically the same as the storage unit 100.

  As described above, the image correction apparatus according to the present embodiment estimates the filter coefficient a (n, m) representing the degradation of the input image from only the input image y (i, j), and uses this filter coefficient. In this way, the input image can be corrected. For this reason, there is an effect that a camera shake sensor or the like for identifying deterioration at the time of shooting is not necessary. Further, even when applied to an image display device that is not interlocked with the imaging device, there is an effect that the image degradation is estimated and a corrected image w (i, j) can be obtained.

  In the present embodiment, the input image y (i, j) is assumed to be an image stored in the storage unit 100 and input to the deterioration estimation unit 101 and the correction unit 102. However, the present invention is not limited to this, and the input image y (i, j) is directly input to the deterioration estimation unit 101 and the correction unit 102 as an image of an imaging device such as a CCD or CMOS. Also good.

  The maximum local difference value determining means 2, the gradient calculating means 3, the filter range determining means 4, the filter coefficient determining means 5, and the correcting means 102 according to the present embodiment may be configured as an electronic circuit. Alternatively, it may be configured as software that operates on a processor such as a microcomputer.

  In addition, the maximum local difference value determination unit 2 according to the present embodiment sets a region about a quarter of the center of the image region 201 of the input image as the difference value determination region 202. However, the difference value determination area 202 is not limited to this, and may be set to an arbitrary area composed of all or part of the image area 201 of the input image. For example, when a photograph is taken while focusing on a person at the center of the image area 201, the focus may be in the center of the screen, and a distant view may be formed in the periphery of the screen. In such a case, as described in the present embodiment, the center area of the image area 201 of the input image is set as the difference value determination area 202, so that the input by hand movement is matched to the person at the center. It is possible to effectively detect the filter coefficient a (n, m) representing the image degradation. Conversely, when it is desired to correct the blur of a distant view that is out of focus, the entire image area 201 or the peripheral area may be used as the difference value determination area 202.

  In addition, when a means for designating a position to be focused on the image area 201 using a technique such as face detection is further provided, the area around the focused position is set as the difference value determination area 202. Thus, the above-described filter coefficient a (n, m) can be detected with a small amount of calculation.

  Further, in the present embodiment, the maximum local difference value determining means 2 uses the window region 204 as a window region 204 from the target pixel 203 in a range of −N ≦ n ≦ N in the horizontal direction and 0 ≦ m ≦ M in the vertical direction. The region maximum local difference value D (n, m) was obtained. However, the same effect as described above can be obtained even when the range of the window region 204 is set to 0 ≦ n ≦ N in the horizontal direction and −M ≦ m ≦ M in the vertical direction from the target pixel 203. Basically, since D (n, m) = D (−n, −m) holds, half of the region of coordinates (n, m) of −N ≦ n ≦ N and −M ≦ m ≦ M By setting the area as the window area 204, the amount of calculation can be reduced.

  In addition, the gradient calculation unit 3 according to the present embodiment includes a region selection unit 302, and the region selection unit 302 causes the left window region 204a (−N ≦ n ≦ 0) and the right window region 204b ( Of 0 ≦ n ≦ N), the one in which the degradation characteristic of the input image appears strongly is selected. However, the present invention is not limited to this, and the gradient calculating unit 3 may obtain the maximum local difference gradient value h from the entire window region 204 without selecting the left and right regions. The configuration of the gradient calculating means 3 in this case will be described below with reference to FIG.

  The gradient calculation range determination unit 303 included in the gradient calculation unit 3 according to FIG. 6 calculates the above-described n_min, n_max, and m_max based on the predetermined threshold value α determined by the input image evaluation unit 301. The gradient calculation range determination unit 303 sets the range of coordinates (n, m) (−n_min ≦ n ≦ n_max, 0 ≦ m ≦ m_max) as the gradient calculation range.

  The subtracting unit 304 calculates the gradient calculation range determined by the gradient calculation range determination unit 303 from the difference between the window region maximum local difference values D (n, m) adjacent in the horizontal direction and the vertical direction of the coordinates (n, m). The gradient value is calculated. For example, in the range of 0 ≦ n ≦ n_max and 0 ≦ m ≦ m_max, the horizontal gradient value dh1 (n, m) (= D (n + 1, m) −D (n, m)) and the vertical direction The gradient value dv1 (n, m) (= D (n, m + 1) −D (n, m)) is calculated. For the ranges of −n_min ≦ n ≦ 0 and 0 ≦ m ≦ m_max, the horizontal gradient value dh2 (n, m) (= D (n−1, m) −D (n, m)) and the vertical direction The gradient value dv2 (n, m) (= D (n, m + 1) −D (n, m)) is calculated.

  The maximum value selection unit 305 obtains h (n, m) from the four gradient values output from the subtraction unit 304 by the following equation (9) (where n_lim = Max {n_max, n_min}). Then, the maximum value selection unit 305 outputs the obtained h (n, m) (0 ≦ n ≦ n_lim, 0 ≦ m ≦ m_max) as the maximum local difference gradient value.

  As described above, the gradient calculation unit 3 does not include the region selection unit 302 and does not select the left window region 204a and the right window region 204b that have a strong deterioration characteristic of the input image. The maximum local difference gradient value h (n, m) can be output. However, in general, when the degradation of the input image is caused by camera shake, for example, the camera shake in the upper right direction and the camera shake in the upper left direction that forms 90 degrees with this direction are rarely mixed at the same intensity. Therefore, as in the present embodiment, the gradient calculation means 3 selects either the left window area 204a or the right window area 204b by the area selection means 302, so that the calculation of the subtraction means 304 is performed. The amount can be reduced.

  In order to describe a configuration that can obtain this effect, in the present embodiment, the first and second window regions adjacent to each other in the target pixel 203 are the left window region 204a and the right window region 204b. It was supposed to be. Then, the gradient calculating unit 3 selects, from the window regions 204a and 204b, the window region in which the degradation characteristics of the input image are strongly expressed by the region selecting unit 302. However, the selected window area is not limited to the left window area 204a and the right window area 204b.

  For example, if the window region 204 set in the maximum local difference value determining unit 2 is 0 ≦ n ≦ N (or −N ≦ n ≦ 0) and −M ≦ m ≦ M, the horizontal direction is the pixel 203 of interest. A case is assumed in which the setting is set so as to extend in the right direction (or left direction) and the vertical direction in the vertical direction. In this case, it is assumed that one of the first and second window regions is an upper window region (0 ≦ m ≦ M) from the center of the window region 204, and the other is lower than the center of the window region 204. It is good also as a window area | region (-M <= m <= 0). Then, the gradient calculating unit 3 may be configured to select, by the region selecting unit 302, a window region in which the degradation characteristic of the input image appears strongly among the upper and lower window regions. Even if comprised in this way, the completely same effect as the above-mentioned can be acquired.

  Further, in the present embodiment, the area selecting unit 302 has the absolute value of the horizontal coordinate n of the window area maximum local difference value D (n, m) that is smaller than the predetermined threshold value α determined by the input image evaluating unit 301. Were compared for the left window region 204a and the right window region 204b, and one of the window regions was selected based on the comparison. However, the region selection method is not limited to this. As the value of the window region maximum local difference value D (n, m) is smaller, image deterioration characteristics such as camera shake appear. Therefore, an area having a smaller value of D (n, m) is selected from the left and right window regions 204a and 204b. What is necessary is just to comprise so that it may select.

  For example, the window region may be selected by comparing the sum of D (n, m) in the left window region 204a with the sum of D (n, m) in the right window region 204b. That is, the sum D1 of D (n, m) in the left window region 204a and the sum Dr of D (n, m) in the right window region 204b are obtained by the following equation (10). Then, the region selection unit 302 selects the right window region 204b if Dl> Dr, and selects the left window region 204a if Dl> Dr is not satisfied.

  In addition, the gradient calculation unit 3 according to the present embodiment is configured to include an input image evaluation unit 301 and a gradient calculation range determination unit 303. Then, the gradient calculation range determination unit 303 calculates the maximum local difference gradient value from the predetermined threshold value α output from the input image evaluation unit 301 and the window region maximum local difference value D (n, m). The calculation range was determined. However, the present invention is not limited to this, and the gradient calculation unit 3 does not include the input image evaluation unit 301 and the gradient calculation range determination unit 303, and any one of the entire window region 204, the left window region 204a, and the right window region 204b. Alternatively, it may be configured to determine this as the gradient calculation range.

  FIG. 7 is a diagram illustrating a configuration of the gradient calculation unit 3 when the input image evaluation unit 301 and the gradient calculation range determination unit 303 are not provided. In the gradient calculating unit 3 according to FIG. 7, first, the window region maximum local difference value D (n, m) is input to the region selecting unit 302. For example, the area selection unit 302 obtains Dl and Dr from the above equation (10), and if Dl> Dr, selects the right window area 204b, and if Dl> Dr does not hold, selects the left window area 204a. Select.

  Next, the subtracting means 304 consists of the difference between the window area maximum local difference values D (n, m) adjacent in the horizontal direction and the vertical direction of the coordinates (n, m) for the area selected by the area selecting means 302. Calculate the slope value. For example, when the right window region 204b is selected, the horizontal gradient value dh (n, m) (for the range of coordinates (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) ( = D (n + 1, m) -D (n, m)) and a vertical gradient value dv (n, m) (= D (n, m + 1) -D (n, m)) are calculated. When the left window region 204b is selected, the horizontal gradient value dh (n, m) (= for the range of coordinates (n, m) (−N ≦ n ≦ 0, 0 ≦ m ≦ M). D (n−1, m) −D (n, m)) and a vertical gradient value dv (n, m) (= D (n, m + 1) −D (n, m)) are calculated.

  The maximum value selection unit 305 selects and obtains the larger one of the two gradient values dh (n, m) and dv (n, m) output from the subtraction unit 304. That is, h ′ (n, m) = Max {dh (n, m), dv (n, m)} is obtained. When the left window region 204b is selected, h (n, m) = h ′ (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) is set, and the right window region 204a is selected. In this case, h (n, m) = h ′ (n−N, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M). Then, the maximum value selecting unit 305 outputs h (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) obtained as described above as the maximum local difference gradient value.

  As described above, the gradient calculating unit 3 according to FIG. 7 has a feature that the circuit configuration is simplified because the calculation range is always fixed. However, when the range of camera shake that can occur is wide and (N, M) that determines the range of the window area 204 needs to be set large, the amount of calculation processing increases. On the other hand, since the gradient calculation unit 3 according to the present embodiment includes the input image evaluation unit 301 and the gradient calculation range determination unit 303, the gradient calculation range is limited by a predetermined threshold α corresponding to the input image. be able to. Therefore, it is possible to reduce the amount of processing (or circuit scale or power consumption) in the calculation after the subtracting means 304.

  Further, the gradient calculating means 3 according to the present embodiment obtains a difference between the window region maximum local difference values D (n, m) adjacent in the horizontal direction and the vertical direction of the coordinates (n, m), and the maximum value thereof. Is output as the maximum local difference gradient value h (n, m). However, the present invention is not limited to this, and the gradient calculation means 3 maximizes the difference between the adjacent window region maximum local difference values D (n, m) only in either the horizontal direction or the vertical direction of the coordinates (n, m). You may make it output as local difference gradient value h (n, m).

  For example, in the case where the blur in the horizontal direction hardly occurs like the blur of the in-vehicle image pickup apparatus, but the blur in the vertical direction occurs strongly, the gradient calculation means 3 is only in the vertical direction of the coordinates (n, m). The difference between the adjacent window region maximum local difference values D (n, m) may be output as the maximum local difference gradient value h (n, m). In general, even when a digital camera is photographed by hand, a camera shake in the vertical direction is likely to occur due to the operation of pressing the shutter. By configuring the gradient calculating means 3 in this way, the calculation of the gradient value in the horizontal direction is omitted, so that the amount of calculation can be reduced. In addition, depending on the configuration of the imaging device and the installation method of the imaging device, the direction in which the shutter is pressed may be the horizontal direction of the stored image. In such a case, the gradient calculating unit 3 is configured to output the difference between the window region maximum local difference values D (n, m) adjacent only in the horizontal direction as the maximum local difference gradient value h (n, m). If configured, the amount of calculation can be reduced as described above.

<Embodiment 2>
FIG. 8 is a block diagram showing the configuration of the image correction apparatus according to the present embodiment. In addition, about the structure which is not newly demonstrated among the structures which concern on the following Embodiment 2, the same code | symbol shall be attached | subjected about the same structure as Embodiment 1. FIG. As shown in the figure, the image correction apparatus according to the present embodiment includes storage means 100, deterioration estimation means 101, inverse filter setting means 6, cyclic filter means 7, and storage means 103. The inverse filter setting unit 6 and the recursive filter unit 7 constitute a correction unit 102.

  An image is stored in the storage unit 100. The image output from the storage unit 100 is input to the deterioration estimation unit 101 as an input image. The degradation estimation means 101 obtains a linear filter representing the degradation of the input image from the input image within a predetermined range, and outputs the filter coefficient as the first filter coefficient. The correction unit 102 according to the present embodiment performs an inverse process of the filter process on the input image using the first filter coefficient indicating the two-dimensional linear filter in a predetermined range output from the deterioration estimation unit 101.

  The inverse filter setting unit 6 included in the correcting unit 102 applies the second filter coefficient for performing the cyclic filter processing and the pixels of the input image based on the first filter coefficient from the deterioration estimating unit 101. The order in which the cyclic filter processing is to be performed is determined. The recursive filter means 7 included in the correcting means 102 performs recursive filter processing on the input image from the storage means 100 based on the second filter coefficient determined by the inverse filter setting means 6 and the order. In this way, the cyclic filter unit 7 generates a corrected image by performing a cyclic filter process corresponding to the inverse process of the filter process on the input image. The storage unit 103 stores the corrected image output from the cyclic filter unit 7.

  Next, the basic operation will be described. As described in Embodiment 1, it is generally considered that deterioration such as camera shake and defocus can be expressed linearly. In other words, the image data with deterioration is y (i, j) (i, j; coordinates in pixel units, 0 ≦ i <I, 0 ≦ j <J), and the filter coefficient of the linear filter representing the deterioration of the input image is set. When a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) is assumed and image data without deterioration is x (i, j), the relationship of Expression (1) is established. Similar to the first embodiment, the degradation estimation unit 101 of the image correction apparatus according to the present embodiment estimates the filter coefficient a (n, m) representing the degradation of the input image from the input image y (i, j). To do. The filter coefficient a (n, m) here is given for each coordinate (n, m) within a predetermined range, and the shape of the predetermined range according to the present embodiment is 0 ≦ n ≦ N, It is a rectangle represented by 0 ≦ m ≦ M.

  The correction unit 102 of the image correction apparatus according to the present embodiment uses the first filter coefficient a (n, m) indicating the two-dimensional linear filter in a predetermined range to perform the inverse process of the filter process on the input image y ( i, j). Thus, the image correction apparatus according to the present embodiment is configured to obtain a corrected image w (i, j) corresponding to an image x (i, j) having no deterioration.

  The detailed operation of the image correction apparatus according to this embodiment will be described below with reference to FIG. First, the storage unit 100 stores an input image y (i, j) (0 ≦ i <I, 0 ≦ j <J). The input image y (i, j) read from the storage unit 100 is input to the deterioration estimation unit 101. The degradation estimation means 101 obtains and outputs a filter coefficient a (n, m) of a linear filter representing the degradation of the input image from the input image y (i, j). Deterioration estimation means 101 according to the present embodiment is configured from maximum local difference value determination means 2, gradient calculation means 3, filter range determination means 4, and filter coefficient determination means 5 according to the first embodiment. Shall be.

  The filter coefficient a (n, m) output from the degradation estimation unit 101 and the input image y (i, j) from the storage unit 100 are input to the correction unit 102. The correction unit 102 included in the image correction apparatus according to the present embodiment includes a first filter coefficient a (n, m) that is a filter coefficient of a linear filter in a predetermined range (0 ≦ n ≦ N, 0 ≦ m ≦ M). ) Is applied to the input image y (i, j).

  In order to perform this operation, the inverse filter setting unit 6 and the cyclic filter unit 7 included in the correction unit 102 according to the present embodiment perform the following operations. First, the inverse filter setting means 6 is based on the filter coefficient a (n, m), the filter coefficient b (n, m), which is the second filter coefficient for performing the cyclic filter processing, and the input image. The order in which the cyclic filter processing is to be performed on the pixels is determined. Hereinafter, the order in which the cyclic filter processing should be performed on the pixel (i, j) of the input image y (i, j) may be referred to as the cyclic filter processing order. Next, the recursive filter means 7 applies the recursive filter to the input image from the storage means 100 based on the filter coefficient b (n, m) determined by the inverse filter setting means 6 and the order of the recursive filter processing. Apply processing. By this processing, the correction unit 102 generates a corrected image w (i, j) and outputs the corrected image w (i, j). Hereinafter, operations of the inverse filter setting unit 6 and the cyclic filter unit 7 will be described in detail.

  In general, when equation (1) holds, the following equation (11) holds on condition that a (n, m) ≠ 0.

  The conversion from the input image y (i, j) having a deterioration to the corrected image w (i, j) is realized by a two-dimensional cyclic filter process represented by the following expression (12) obtained by rewriting the expression (11). it can. Note that b (n, m) shown in Expression (12) is a filter coefficient for performing the cyclic filter processing, and b (0,0) = 1 / a (0,0), b (n, m) = a (n, m) / a (0,0) ((n, m) ≠ (0,0), 0 ≦ n ≦ N, 0 ≦ m ≦ M).

  From Expression (12), an input image y (i, j) of a pixel (i, j) and a corrected image w (in, jm) (0 ≦ n ≦ N, 0 ≦ m) obtained in the past. ≦ M, (n, m) ≠ (0,0)), a new corrected image w (i, j) can be calculated. In the two-dimensional cyclic filter processing expressed by Expression (12), the calculation of a new corrected image is performed in an “ascending order” in the horizontal and vertical directions of the pixel (i, j) of the input image. Means processing. By this two-dimensional cyclic filter processing, it becomes possible to calculate a corrected image w (i, j) of all pixels (i, j) (0 ≦ i <I, 0 ≦ j <J) of the input image. .

  However, in order to perform the two-dimensional recursive filter processing, the past corrected image that is the basis of the new corrected image w (i, j), that is, the corrected image w (i, j) on the right side of Expression (12). The initial value of is required. Next, the initial value of the corrected image w (i, j) and the range of the pixel (i, j) that should be given the initial value will be described. The relationship represented by the following equation (13) is established between the corrected image w (i, j) calculated by equation (12) and the image x (i, j) without deterioration.

  In this equation (13), w (i, j) = x (i, j) is established in all pixels (i, j) within a predetermined range (0 ≦ i <N, 0 ≦ j <M). In that case, it is recursively determined that w (i, j) = x (i, j) holds for all pixels (i, j) (0 ≦ i <I, 0 ≦ j <J). means. That is, if w (i, j) = x (i, j) (0 ≦ i <N, 0 ≦ j <M) is given as an initial value, all pixels (i, j) ( In 0 ≦ i <I, 0 ≦ j <J), it means that a corrected image w (x, j) corresponding to an image x (i, j) having no deterioration is obtained.

  However, in practice, since the value of the image x (i, j) without deterioration is unknown, in the present embodiment, as the initial value of the corrected image w (i, j), w (i, j) = y (i, j) (0 ≦ i <N, 0 ≦ j <M) is set. That is, it is assumed that the cyclic filter processing is not performed on the pixel (i, j) of the input image within the range (0 ≦ i <N, 0 ≦ j <M).

  In this case, there is a problem that the error of the initial value of the corrected image w (i, j) is propagated to the entire image by the cyclic filter processing. Therefore, the inverse filter setting means 6 according to the present embodiment sets the absolute value of the filter coefficient b (n, m) ((n, m) ≠ (0,0)) to be less than 1. For example, the inverse filter setting means 6 determines the filter coefficient b (n, m) whose absolute value is set to less than 1 by the following equation (14).

  However, B = Max {| a (n, m) |; (n, m) ≠ (0,0), 0 ≦ n ≦ N, 0 ≦ m ≦ M}, and C (0 <C <1) Is a constant. After setting the filter coefficient b (n, m) ((n, m) ≠ (0,0)) by the equation (14), b (0,0) is expressed by the following equation so that the gain of the filter becomes 1. Set by (15).

  As described above, by making the absolute value of the filter coefficient b (n, m) ((n, m) ≠ (0,0)) smaller than 1, the error of the initial value of the corrected image w (i, j) However, the influence which propagates to the whole image can be made small. According to experiments, when a value of about 0.5 was set as the constant C, the error pattern could be reduced and a sufficient correction effect was obtained.

  In the above description, it is assumed that a (0,0) ≠ 0, and the corrected image w (i, j) with no initial value is set in the horizontal direction of the pixel (i, j) of the input image and Calculations were performed in ascending order for each vertical direction. However, when a (0, 0) = 0, the same processing as described above can be performed by changing the order of the cyclic filter processing to an order different from the above. For example, it is assumed that the degradation of the input image is represented by a one-dimensional filtering process in any one of the horizontal, vertical, and diagonal directions. Assuming this, filter coefficients a (0,0), a (N, 0), a (0,0) at the coordinates of the four corners within a predetermined range (0 ≦ n ≦ N, 0 ≦ m ≦ M). Most of the values of M) and a (N, M) are non-zero.

  Even if those values do not become non-zero, the deterioration estimation means 101 is in a region where the filter coefficients a (N, 0), a (0, M), and a (N, M) are non-zero. In addition, if the coefficients N and M that define a predetermined range are set, any one of them is always non-zero. Thus, hereinafter, a case where any one of the filter coefficients a (N, 0), a (0, M), and a (N, M) is non-zero will be described.

  For example, when a (N, 0) ≠ 0, the filter coefficient b (n, m) is set by the following equation (16).

  However, B = Max {| a (n, m) |; (n, m) ≠ (N, 0), 0 ≦ n ≦ N, 0 ≦ m ≦ M}, and C (0 <C <1) Is a constant. After setting the filter coefficient b (n, m) ((n, m) ≠ (0,0)) by the equation (16), b (0,0) is set by the equation (15). Further, w (i, j) = y (i, j) (IN ≦ i <I, 0 ≦ j <M) is set as an initial value of the corrected image w (i, j). Then, a two-dimensional cyclic filter process represented by the following equation (17) is performed.

  From Expression (17), an input image y (i, j) of a certain pixel (i, j) and a corrected image w (i + n, j−m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) obtained in the past. , (N, m) ≠ (0,0)), a new corrected image w (i, j) can be calculated. The two-dimensional cyclic filter processing expressed by Expression (17) is to calculate a new corrected image by “descending order” for the horizontal direction of the pixel (i, j) of the input image and “ascending order” for the vertical direction. This means processing performed in a round.

  The case where a (N, 0) ≠ 0 has been described above. However, when a (0, M) ≠ 0, the filter coefficient b (n, m) is set by the following equation (18).

  However, B = Max {| a (n, m) |; (n, m) ≠ (0, M), 0 ≦ n ≦ N, 0 ≦ m ≦ M}, and C (0 <C <1) Is a constant. After setting the filter coefficient b (n, m) ((n, m) ≠ (0,0)) according to the equation (18), b (0,0) is set according to the equation (15). Further, w (i, j) = y (i, j) (0 ≦ i <N, J−M ≦ j <J) is set as an initial value of the corrected image w (i, j). Then, a two-dimensional cyclic filter process represented by the following equation (19) is performed.

  From Expression (19), an input image y (i, j) of a certain pixel (i, j) and a corrected image w (i−n, j + m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) obtained in the past. , (N, m) ≠ (0,0)), a new corrected image w (i, j) can be calculated. The two-dimensional cyclic filter processing represented by the equation (19) is to calculate a new corrected image by “ascending order” for the horizontal direction of the pixel (i, j) of the input image and “descending order” for the vertical direction. This means processing performed in a round.

  The case where a (0, M) ≠ 0 has been described above. However, when a (N, M) ≠ 0, the filter coefficient b (n, m) is set by the following equation (20).

  However, B = Max {| a (n, m) |; (n, m) ≠ (N, M), 0 ≦ n ≦ N, 0 ≦ m ≦ M}, and C (0 <C <1) Is a constant. After setting the filter coefficient b (n, m) ((n, m) ≠ (0,0)) by the equation (20), b (0,0) is set by the equation (15). Further, w (i, j) = y (i, j) (IN ≦ i <I, J−M ≦ j <J) is set as an initial value of the corrected image w (i, j). Then, a two-dimensional cyclic filter process represented by the following equation (21) is performed.

  From Expression (21), an input image y (i, j) of a pixel (i, j) and a corrected image w (i + n, j + m) (0 ≦ n ≦ N, 0 ≦ m ≦ M, ( n, m) ≠ (0,0)), a new corrected image w (i, j) can be calculated. The two-dimensional cyclic filter processing represented by Expression (21) is to perform calculation of a new corrected image by cycling in “descending order” in the horizontal direction and the vertical direction of the pixel (i, j) of the input image. Means processing.

  In order to perform the processing as described above, the inverse filter setting means 6 is configured to operate according to the flowchart shown in FIG. First, the inverse filter setting means 6 according to the present embodiment has coordinates (n, m) (n = 0 or N, m) of four corners within a predetermined range (0 ≦ n ≦ N, 0 ≦ m ≦ M). = 0 or M), the coordinates corresponding to the filter coefficient a (n, m) (n = 0 or N, m = 0 or M) having a predetermined value are obtained. The predetermined value here is a value other than zero. Then, the inverse filter setting means 6 according to the present embodiment performs a cyclic filter process on a predetermined order corresponding to the obtained coordinates (n, m) (n = 0 or N, m = 0 or M). Is determined as the order (step S1).

  In the present embodiment, the predetermined order corresponding to the coordinates (0, 0) is ascending order for each of the horizontal direction and the vertical direction of the pixels of the input image. Similarly, the predetermined order corresponding to the coordinates (N, 0) is a descending order in the horizontal direction and an ascending order in the vertical direction of the pixels of the input image. The predetermined order corresponding to the coordinates (0, M) is ascending order in the horizontal direction of pixels of the input image and descending order in the vertical direction. The predetermined order corresponding to the coordinates (N, M) is a descending order for each of the horizontal direction and the vertical direction of the pixels of the input image.

  Thus, for example, first, the value of a (0,0) is checked, and if a (0,0) ≠ 0, “the cyclic filter processing represented by Expression (12), that is, the cyclic filter processing is performed. Is determined as “ascending order in the horizontal direction and the vertical direction”, and the process proceeds to step S2. Otherwise, the value of a (N, 0) is examined, and if a (N, 0) ≠ 0, “the cyclic filter processing represented by the equation (17), that is, the cyclic filter processing order. Is determined as “descending order for the horizontal direction and ascending order for the vertical direction”, and the process proceeds to step S3. Otherwise, the value of a (0, M) is examined, and if a (0, M) ≠ 0, “the cyclic filter processing represented by the equation (19), that is, the cyclic filter processing order. Are determined as “ascending order for the horizontal direction and descending order for the vertical direction”, and the process proceeds to step S4. Otherwise, it is determined that “the cyclic filter processing represented by Expression (21), that is, the order of the cyclic filter processing is descending in the horizontal direction and the vertical direction”, and the process proceeds to step S5.

  In step S2, b (n, m) ((n, m) ≠ (0,0)) is obtained from equation (14), and the process proceeds to step S6. In step S3, b (n, m) ((n, m) ≠ (0,0)) is obtained from equation (16), and the process proceeds to step S6. In step S4, b (n, m) ((n, m) ≠ (0,0)) is obtained from equation (18), and the process proceeds to step S6. In step S5, b (n, m) ((n, m) ≠ (0,0)) is obtained from equation (20), and the process proceeds to step S6. In step S6 after these steps S2 to S5, b (0, 0) is obtained by equation (15).

  As described above, the inverse filter setting unit 6 according to the present embodiment, based on the filter coefficient a (n, m), the filter coefficient b (n, m) for performing the cyclic filter processing, and The operation of determining the order in which the cyclic filter processing is to be performed on the pixel (i, j) of the input image y (i, j) is performed. In the present embodiment, in Expression (14), Expression (16), Expression (18), and Expression (20), when B = 0, that is, nonzero among the filter coefficients a (n, m). If only one value exists, b (n, m) = 0 ((n, m) ≠ 0, 0 ≦ n ≦ N, 0 ≦ m ≦ M) is set.

  Then, the recursive filter unit 7 of the present embodiment includes a filter coefficient b (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) determined by the inverse filter setting unit 6 and a recursive filter process. On the basis of the order, the input image y (i, j) is subjected to cyclic filter processing corresponding to the reverse processing of the filter processing. Then, the recursive filter means 7 outputs the corrected image w (i, j) generated by this processing.

  The corrected image w (i, j) output from the cyclic filter unit 7 is written in the storage unit 103. Note that the storage unit 103 may share a storage device such as a hard disk or memory that is physically the same as the storage unit 100. Each means of the inverse filter setting means 6 and the recursive filter means 7 may be configured as an electronic circuit or as software operating on a processor such as a microcomputer.

  According to the image correction apparatus according to the present embodiment that performs the operation as described above, the correction process that performs the reverse process of the filter process on the input image using the filter coefficient a (n, m) that represents the degradation of the input image. Can be performed by a simple filter calculation process. This eliminates the need for iterative processing such as the conventional Richardson-Lucy algorithm or repeated FFT and inverse FFT, thereby reducing the processing amount (or circuit scale or power consumption) of the correction means 102. be able to. Moreover, since the degradation estimation means 101 which concerns on this Embodiment is the same as the degradation estimation means which concerns on Embodiment 1, the effect of Embodiment 1 can also be acquired.

  In the present embodiment, when there is only one non-zero value among the filter coefficients a (n, m), b (n, m) = 0 ((n, m) ≠ 0, 0 ≦ n ≦ N, 0 ≦ m ≦ M). However, the present invention is not limited to this. If there is only one non-zero value among the filter coefficients a (n, m), the input image y (i, j) is not passed through the recursive filter means 7. The input image y (i, j) may be output as it is as the corrected image w (i, j).

  In this embodiment, the inverse filter setting means 6 determines that the values of the filter coefficients a (n, m) (n = 0 or N, m = 0 or M) at the four corner coordinates are non-zero in step S1. It was judged in order whether it was zero. Then, based on the determination, the inverse filter setting means 6 has a predetermined order corresponding to the coordinates (n, m) (n = 0 or N, m = 0 or M) of the filter coefficient taking a non-zero value. Was determined as the order of the cyclic filter processing. However, the present invention is not limited to this. In step S1, the inverse filter setting means 6 uses the filter coefficient a (n, m) (n = n) that takes the maximum absolute value among the coordinates (n, m) at the four corners. The coordinates corresponding to 0 or N, m = 0 or M) are obtained. Then, the inverse filter setting means 6 determines a predetermined order corresponding to the obtained coordinates (n, m) (n = 0 or N, m = 0 or M) as the order of the cyclic filter processing. Also good.

  When configured in this way, the inverse filter setting means 6 indicates that “the order of the cyclic filter processing is in each of the horizontal direction and the vertical direction when the filter coefficient having the maximum absolute value is a (0, 0). Ascending order "is determined, and the process proceeds to step S2. When the filter coefficient that takes the maximum absolute value is a (N, 0), it is determined that “the order of cyclic filter processing is descending in the horizontal direction and ascending in the vertical direction”, and the process proceeds to step S3. . If the filter coefficient that takes the maximum absolute value is a (0, M), “the order of the cyclic filter processing is ascending order in the horizontal direction and descending order in the vertical direction”, and the process proceeds to step S4. . If the filter coefficient that takes the maximum absolute value is a (N, M), it is determined that “the order of cyclic filter processing is descending in the horizontal direction and the vertical direction”, and the process proceeds to step S5. The subsequent operation is the same as the operation after step S6 described above.

  As described above, the coordinates corresponding to the filter coefficient a (n, m) (n = 0 or N, m = 0 or M) taking the maximum absolute value among the coordinates of the four corners are obtained, and the corresponding coordinates are obtained. If the predetermined order is determined as the above-described order, the accuracy of the reverse processing is increased, and the image quality of the corrected image w (i, j) can be improved.

  Note that the image correction apparatus according to the present embodiment described above includes the same deterioration estimation unit 101 as in the first embodiment, and the filter coefficient a (n, m) from the deterioration estimation unit 101 is an inverse filter setting unit. 6 was input. However, the present invention is not limited to this. For example, camera shake is detected by a camera shake sensor attached to the imaging apparatus, and the detection result is a filter coefficient a (n, m) (0 ≦ n ≦ N, The configuration may be such that 0 ≦ m ≦ M) is input to the inverse filter setting means 6. Also in this case, similarly to the above, the correction process for applying the reverse process of the filter process to the input image using the filter coefficient a (n, m) representing the degradation of the input image can be performed by a simple filter calculation process. it can. Note that there is a case where a target imaging device is determined in advance, and blur correction due to a defocus of the imaging device is desired. In that case, a plurality of blur characteristics depending on the depth of focus assumed in advance are held as the filter coefficient a (n, m), and the filter coefficient a (n, m) switched from the value of the depth of focus is used as the inverse filter setting means 6. It may be configured to input to

<Embodiment 3>
The image correction apparatus according to the present embodiment differs from the image correction apparatus according to the second embodiment in the operation of the inverse filter setting means 6. In the second embodiment, for example, it is assumed that the degradation of the input image is represented by one-dimensional filter processing in one of the horizontal, vertical, and diagonal directions. Then, the inverse filter setting means 6 according to the second embodiment uses the filter coefficient b ((n, m) (n = 0 or N, m = 0 or M) as a reference based on any of the four corner coordinates. n, m) and the order of the cyclic filter processing. However, when there is degradation such as blur due to defocusing, the distribution of the values of the filter coefficients a (n, m) spreads circularly, and the filter coefficients a (n, m) at the four corners are all zero. In some cases. Even in the case of linear camera shake, depending on the characteristics of camera shake, the values of the filter coefficients a (n, m) at the four corners may be small, and the accuracy of the inverse processing may be low.

Therefore, the inverse filter setting means 6 according to the present embodiment has a filter coefficient that takes the maximum absolute value among coordinates (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) within a predetermined range. One coordinate (N0, M0) corresponding to a (N0, M0) is obtained. Then, a predetermined range (0 ≦ n ≦ N, 0 ≦ m ≦ M) is divided into a plurality of divided ranges S _{1 to} S ₄ on the basis of the obtained one coordinate (N0, M0). Then, based on the values of one segment range S _{k (k} = 1~4 of any one of the values) filter coefficients in a (n, m) (( n, m) ∈S k), the corresponding partition A filter coefficient b (n, m) ((n, m) εS _k ) within the range is determined.

  The operation of the inverse filter setting unit 6 according to this embodiment will be described with reference to the flowchart of FIG. The operation before step S11 is the same as the operation before step S1 of the second embodiment. First, the inverse filter setting means 6 is a filter coefficient filter coefficient a (N0, N) that takes the maximum absolute value among coordinates (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) within a predetermined range. One coordinate (N0, M0) corresponding to (M0) is obtained (step S11).

In the next step, a predetermined range (0 ≦ n ≦ N, 0 ≦ m ≦ M) is divided into a plurality of divided ranges S _{1 to} S ₄ . In the present embodiment, the segment range S ₁ = {(n, m); 0 ≦ n ≦ N0, 0 ≦ m ≦ M0}, the segment range S ₂ = {(n, m); N0 ≦ n ≦ N, 0 ≦ m ≦ M0}, section range S ₃ = {(n, m); 0 ≦ n ≦ N0, M0 ≦ m ≦ M}, section range S ₄ = {(n, m); N0 ≦ n ≦ N, M0 ≦ m ≦ M}.

Then, one division range S _k (one value of k = 1 to 4) is selected from the division ranges S _{1 to} S ₄ . As a method of selecting this one segment range S _k , for example, the segment range S _k having the widest range is selected. That is, in the horizontal direction, if N0> N−N0, the segment ranges S ₁ and S ₃ are selected, and if not, S ₂ and S ₄ are selected. In the vertical direction, if M0> M−M0, the segment ranges S ₁ and S ₂ are selected, and if not, S ₃ and S ₄ are selected. With these two determinations, one segment range S _k can be selected from the _four segment ranges S _{1 to} S ₄ . Here, division range S ₁ has been described a method of selecting on the basis of the breadth of the range of to S _4, the present invention is not limited to this, the filter coefficients a (n of each segment range S ₁ to S _4, obtains the sum of the values of m), the largest division range of the summation, may be selected as one of the division range S _k.

Then, replace one segment range S _k selected, the predetermined range used in the description so far (0 ≦ n ≦ N, 0 ≦ m ≦ M) ( step S12). For example, when the segment range S ₁ is selected, a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) = a (n, m) (0 ≦ n ≦ N0, 0 ≦ m ≦ M0), and the process proceeds to step S13. Similarly, when the segment range S ₂ is selected, a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) = a (n + N0, m) (0 ≦ n ≦ N−N0) , 0 ≦ m ≦ M0), and the process proceeds to step S13. When the segment range S ₃ is selected, a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) = a (n, m + M0) (0 ≦ n ≦ N0, 0 ≦ m ≦ M−M0), the process proceeds to step S13. Further, when the division range S ₄ is selected, a (n, m) ( 0 ≦ n ≦ N, 0 ≦ m ≦ M) = a (n + N0, m + M0) (0 ≦ n ≦ N-N0,0 ≦ m ≦ M−M0), and the process proceeds to step S13.

After step S12, the inverse filter setting unit 6 determines the predetermined order in which to correspond to the divided ranges S _k selected in the step S12, as the order of the recursive filter process performed by the recursive filter unit 7 (step S13). For example, when the segment range S ₁ is selected in step S12, the filter coefficient a (N, M) of the filter coefficients a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) is selected. The value is the largest. Therefore, in that case, the inverse filter setting means 6 determines the order of the cyclic filter processing as descending order for each of the horizontal direction and the vertical direction of the pixels of the input image, and proceeds to step S17. Further, when the division range S ₂ is selected in step S12, the filter coefficients a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) of the filter coefficient a (0, M) The value is the largest. Therefore, in that case, the inverse filter setting means 6 determines the order of the cyclic filter processing as ascending order for the horizontal direction of pixels of the input image and descending order for the vertical direction, and proceeds to step S16.

Further, when the division range S ₃ is selected in step S12, out of the filter coefficients a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M), the filter coefficient a (N, 0) The value is the largest. Therefore, in that case, the inverse filter setting means 6 determines the order of the cyclic filter processing as descending order for the horizontal direction and ascending order for the vertical direction of the pixels of the input image, and proceeds to step S15. Further, in the case where division range S ₄ is selected step S12, among the filter coefficients a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M), the filter coefficient a (0,0) The value is the largest. Therefore, in this case, the inverse filter setting unit 6 determines the order of the cyclic filter processing as ascending order in each of the horizontal direction and the vertical direction of the pixels of the input image, and proceeds to step S14.

The operations in steps S14 to S17 are the same as the operations in steps S2 to S5 in the second embodiment. In this way, the inverse filter setting means 6 determines the filter coefficient b (n, m) (based on the value of the filter coefficient a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) replaced in step S12. 0 ≦ n ≦, N, 0 ≦ m ≦ M). In other words, with the filter coefficient a (n, m) ((n, m) εS _k ) before replacement in step S12, the inverse filter setting means 6 has one section range S _k (k = 1 to 1). Based on the value of the filter coefficient a (n, m) ((n, m) εS _k ) in any one of 4), the filter coefficient b (n, m) ((n , M) εS _k ). The operation after step S18 performed after step S14 to step S17 is the same as the operation after step S6 of the second embodiment.

  The image correction apparatus according to the present embodiment as described above corresponds to the inverse process of the filter process based on the coordinates (N0, M0) of the filter coefficient a (N0, M0) having the maximum absolute value. Perform recursive filtering. Therefore, even if the value of the filter coefficient a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) representing the degradation of the input image is small at any of the four corner coordinates within the predetermined range, the accuracy is high. Therefore, it is possible to obtain an effect that the image quality of the corrected image can be improved.

In step S13 according to this embodiment, the predetermined order in which to correspond to the divided ranges S _k, was determined as the order of the recursive filter. However, the present invention is not limited to this, and may be the same as step S1 in the second embodiment. For example, the inverse filter setting unit 6, the newly set range in step S12 (0 ≦ n ≦ N, 0 ≦ m ≦ M), that is, among the four corner coordinates in one segment range S _k, given The order of the cyclic filter processing may be determined based on the coordinates corresponding to the filter coefficient a (n, m) that takes a value.

<Embodiment 4>
In the image correction apparatus according to the third embodiment, one of the filter coefficients a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) within a predetermined range output from the degradation estimation unit 101. Based on the filter coefficient a (n, m) ((n, m) ∈S _k ) within the segment range S _k (k = 1 to 4), the filter coefficient b (n, m) is Were determined. However, there may be a filter coefficient a (n, m) ((n, m) εS ₁ ) having a large value in the remaining segment range S ₁ (l ≠ k). Therefore, the image correction apparatus according to this embodiment, by repeating the operation of the inverse filter setting unit 6 according to the third embodiment, the filter coefficients in the plurality of sections ranging _{S k a (n, m)} ((n , M) εS _k ), the filter coefficient b (n, m) ((n, m) εS _k ) in the corresponding segment range is determined.

  The operation of the image correction apparatus including the inverse filter setting unit 6 will be described with reference to the flowchart of FIG. First, the image correction apparatus obtains a filter coefficient a (n, m) representing deterioration of the input image within a predetermined range (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) (step S21). ). As a method for obtaining the filter coefficient a (n, m), for example, the degradation estimation means 101 shown in the first embodiment is used.

Next, the inverse filter setting means 6 has a filter coefficient a (N0, M0) that takes the maximum absolute value among the coordinates (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M) within a predetermined range. ) (1) (N0, M0) is obtained (step S22). In addition, since the operation | movement from step S23 to step S29 is the same as the operation | movement from step S11 of Embodiment 3 to step 18, description is abbreviate | omitted. Thus, the inverse filter setting unit 6 in step S24, determines the order of the recursive filter processing, in step S25 to S29, the filter coefficients in one segment range _{S k b (n, m)} ((n, m) Determine ∈ S _k ).

The recursive filter means 7 includes the order of recursive filter processing determined by the inverse filter setting means 6 and the filter coefficients b (n, m) ((n, m) εS _k ) within one division range S _k . Based on the above, a cyclic filter process is performed on the input image to obtain a corrected image (step S30).

After step S30, based on the value of the filter coefficient a (n, m) ((n, m) εS _l ) in the remaining segment range S ₁ (l ≠ k), it is determined whether there is any remaining degradation. Judgment is made (step S31). In step S31, when it is determined that there is deterioration remains the inverse filter setting unit 6, from the rest of the division range S _l, selects one division range, the flow returns to step S24. If it is determined in step S31 that there is no remaining deterioration, the image correction process is terminated. The operation of each step after the second time by the loop is substantially the same as the first time, but in step S30, the cyclic filter process is performed on the corrected image obtained in the previous loop.

According to the image correction apparatus according to the present embodiment as described above, only correction is performed using only the filter coefficient a (n, m) ((n, m) εS _k ) within one division range S _k . If there is a filter coefficient a (n, m) ((n, m) ∈ S _l ) having a large value in the remaining segment range S _l (l ≠ k), the filter coefficient is used. Correction can be performed.

In the present embodiment described above, in step S31, based on the value of the remaining sections range S _{l (l} ≠ k) filter coefficients in a (n, m) (( n, m) ∈S l) Thus, it is determined whether or not there is any remaining deterioration. If there is deterioration, the process returns to step S24. However, the looping method is not limited to this. For example, in the process of step S31, the process of step S21 is performed using the corrected image obtained in step S30 instead of the input image, and the filter coefficient a (n, m) (0 ≦ n) for the entire predetermined range is again obtained. ≦ N, 0 ≦ m ≦ M). Then, based on the value of the obtained filter coefficient a (n, m) (0 ≦ n ≦ N, 0 ≦ m ≦ M), it is determined whether there is any remaining deterioration. You may return to step S22.

1 is a block diagram illustrating a configuration of an image correction apparatus according to Embodiment 1. FIG. 3 is a conceptual diagram illustrating a difference value determination area according to Embodiment 1. FIG. 3 is a conceptual diagram showing a window area according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating a configuration of a gradient calculating unit according to the first embodiment. 6 is a diagram illustrating an operation of a gradient calculating unit according to Embodiment 1. FIG. FIG. 3 is a block diagram illustrating a configuration of a gradient calculating unit according to the first embodiment. FIG. 3 is a block diagram illustrating a configuration of a gradient calculating unit according to the first embodiment. 6 is a block diagram illustrating a configuration of an image correction apparatus according to Embodiment 2. FIG. 10 is a flowchart illustrating an operation of the image correction apparatus according to the second embodiment. 10 is a flowchart illustrating an operation of the image correction apparatus according to the third embodiment. 10 is a flowchart illustrating an operation of the image correction apparatus according to the fourth embodiment.

Explanation of symbols

  2 maximum local difference value determining means, 3 gradient calculating means, 4 filter range determining means, 5 filter coefficient determining means, 6 inverse filter setting means, 7 cyclic filter means, 100, 103 storage means, 101 deterioration estimation means, 102 correction Means, 201 image area, 202 difference value determination area, 203 pixel of interest, 204, 204a, 204b window area, 205 neighboring pixels, 301 input image evaluation means, 302 area selection means, 303 gradient calculation range determination means, 304 subtraction means, 305 Maximum value selection means.

Claims (8)

For each pixel of interest in the difference value determination area consisting of all or part of the image area of the input image, image data of the pixel of interest, and image data of neighboring pixels in the window area set around the pixel of interest The local difference value consisting of the difference is calculated for each pixel coordinate indicating the relative position of the neighboring pixel with respect to the target pixel, and the local difference for each coordinate regarding all the target pixels in the difference value determination region Maximum local difference value determining means for determining a maximum value as a window region maximum local difference value for each coordinate;
A gradient calculating means for determining a difference between the window region maximum local difference values adjacent in at least one of the horizontal direction or the vertical direction of the coordinates for each coordinate as a maximum local difference gradient value;
A filter range determining means for obtaining, as a filter coefficient existence range, a range of the coordinates corresponding to the maximum local difference gradient value not less than a predetermined value among the maximum local difference gradient values determined by the gradient calculating means;
Filter coefficient determining means for calculating a filter coefficient within the filter coefficient existence range based on the maximum local difference gradient value within the filter coefficient existence range determined by the filter range determination means;
Correction means for performing reverse processing of filter processing on the input image using the filter coefficient calculated by the filter coefficient calculation means;
Image correction device. The gradient calculating means includes
An input image evaluation unit that determines a predetermined threshold based on the window region maximum local difference value determined by the maximum local difference value determination unit;
Gradient calculation range determining means for determining a range of the coordinates where the maximum local differential gradient value should be calculated based on the window region maximum local difference value and the predetermined threshold value determined by the input image evaluation means. With
Calculating the maximum local difference gradient value only for the coordinates within the range determined by the gradient calculation range determination means;
The image correction apparatus according to claim 1. The window region includes first and second window regions adjacent to each other in the target pixel,
The gradient calculating means includes
Based on the window region maximum local difference value in the first window region and the window region maximum local difference value in the second window region, one of the first and second window regions is determined. Selecting and calculating the maximum local difference gradient value only for the coordinates in the selected window region;
The image correction apparatus according to claim 1. Using a first filter coefficient indicating a two-dimensional linear filter in a predetermined range, and comprising a correcting means for performing an inverse process of the filter process on the input image,
The correction means includes
A second filter coefficient for performing cyclic filter processing based on the first filter coefficient, and an inverse filter setting means for determining an order in which cyclic filter processing should be performed on pixels of the input image; ,
Cyclic filter means for applying cyclic filter processing to the input image based on the second filter coefficient determined by the inverse filter setting means and the order,
Image correction device. The predetermined range is represented by coordinates indicating a rectangle,
The inverse filter setting means includes
Based on the value of the first filter coefficient corresponding to the coordinates of the four corners within the predetermined range, one coordinate is selected from the coordinates of the four corners, and a predetermined order corresponding to the selected coordinates is Decide as order,
The image correction apparatus according to claim 4. The inverse filter setting means includes
Setting the absolute value of the second filter coefficient to less than 1;
The image correction apparatus according to claim 4 or 5. The inverse filter setting means includes
Among the coordinates within the predetermined range, one coordinate corresponding to the first filter coefficient having the maximum absolute value is obtained, and the predetermined range is divided based on the obtained one coordinate. Determining the second filter coefficient in the corresponding segment range based on the value of the first filter coefficient in at least one of the segment ranges among a plurality of segment ranges;
The image correction apparatus according to claim 4. For each pixel of interest in the difference value determination area consisting of all or part of the image area of the input image, image data of the pixel of interest, and image data of neighboring pixels in the window area set around the pixel of interest The local difference value consisting of the difference is calculated for each pixel coordinate indicating the relative position of the neighboring pixel with respect to the target pixel, and the local difference for each coordinate regarding all the target pixels in the difference value determination region Maximum local difference value determining means for determining a maximum value as a window region maximum local difference value for each coordinate;
A gradient calculating means for determining a difference between the window region maximum local difference values adjacent in at least one of the horizontal direction or the vertical direction of the coordinates for each coordinate as a maximum local difference gradient value;
A filter range determining means for obtaining, as a filter coefficient existence range, a range of the coordinates corresponding to the maximum local difference gradient value not less than a predetermined value among the maximum local difference gradient values determined by the gradient calculating means;
Filter coefficient determining means for calculating a first filter coefficient in the filter coefficient existence range based on the maximum local difference gradient value in the filter coefficient existence range determined by the filter range determination means;
A second filter coefficient for performing cyclic filter processing based on the first filter coefficient, and an inverse filter setting means for determining an order in which cyclic filter processing should be performed on pixels of the input image; ,
Cyclic filter means for applying cyclic filter processing to the input image based on the second filter coefficient determined by the inverse filter setting means and the order,
Image correction device.

Images