JP5040519B2 - Image processing apparatus, image processing method, and program - Google Patents

Description

  The present invention relates to an image processing apparatus, and more particularly to an image processing apparatus that performs demosaic processing, a processing method thereof, and a program that causes a computer to execute the method.

  In an image sensor of a digital still camera, a color filter is arranged on the light receiving surface of each pixel arranged in two dimensions, and various types of color filter coding specifications have been proposed. As a typical example, high-resolution RGB signals can be obtained by arranging primary color filters such as red (R), green (G), and blue (B) in the same spatial phase of three image sensors. There is a so-called 3CCD camera that can realize the above. However, this 3CCD camera uses three imaging elements and needs to use a prism that separates incident light for each color of RGB, so it is not suitable for miniaturization and cost reduction.

  On the other hand, there is a camera that realizes a small and low-cost digital still camera by arranging RGB color filters on one image pickup device. A plurality of color filters are arranged in one image sensor to realize performance close to 3CCD. For example, in the Bayer array, R pixels and G pixels are alternately arranged in odd rows, and G pixels and B pixels are alternately arranged in even rows. In this arrangement, since many G pixels are arranged with respect to the R pixel and the B pixel, the resolution of the G pixel is higher than the resolution of the R pixel and the G pixel. The G pixel is a color component that is a main component when generating a luminance signal, and it can be said that a human being is an array that makes good use of the fact that the resolution for luminance is high and the resolution for color is low.

  Since simply generating the luminance signal Y and the color difference signals Cr and Cb from RGB signals having spatially different phases and sampling rates in the Bayer array causes a false signal, an RGB signal having the same spatial phase is generated. Therefore, it is necessary to generate the luminance signal Y and the color difference signals Cr and Cb. In this way, a process of interpolating RGB signals having spatially different phases and sampling rates as if the same spatial phase RGB signals exist as in 3CCD (hereinafter referred to as 3 plates) is called demosaic processing. When performing this demosaic process, it is necessary to pay attention to resolution and color falsification.

In the conventional demosaic process, the correlation direction is determined from the surrounding pixels of the interpolation target pixel, and the interpolation process is performed based on the correlation direction. For example, a technique has been proposed in which correlation directions are determined in a horizontal direction and a vertical direction in a Bayer array, and interpolation processing is performed based on the correlation directions (see, for example, Patent Document 1). Further, in the Bayer array or the diagonal pixel array, the correlation direction is determined in the horizontal direction, the vertical direction, the direction rotated right by 45 degrees with respect to the horizontal direction, and the direction rotated left by 45 degrees with respect to the horizontal direction. A technique for performing an interpolation process based on a direction has been proposed (for example, see Patent Document 2).
Japanese Patent Laid-Open No. 11-220742 (FIG. 2) Japanese Patent Laying-Open No. 2007-037104 (FIG. 16)

  Here, when the resolutions of the Bayer array and the three plates are compared, not only the resolution in the oblique direction but also the resolutions near the Nyquist frequency in the horizontal direction and the vertical direction are different. The resolution in the oblique direction differs in principle because the theoretical limit value differs from the difference in the number of G pixels. On the other hand, with respect to the vicinity of the Nyquist frequency in the horizontal direction and the vertical direction, if interpolation is performed using correct pixels, the resolution should not be impaired, but it may not be possible to determine which pixel is the correct pixel. Here, the Nyquist frequency means a limit frequency that can be expressed. Specifically, the Nyquist frequency is half that of the pixel sampling frequency fs ((1/2) · fs).

  The above-described problem is the same in color filter arrays other than the Bayer array. For example, in a so-called diagonal pixel array, a signal having a frequency of “(1/4) · fs” in the horizontal direction and the vertical direction is “ The same phenomenon as in the case of (1/2) · fs ”may occur.

  The present invention has been made in view of such a situation, and an object of the present invention is to generate an appropriate interpolation value even when the interpolation direction is unclear due to the color filter array during the demosaic process. And

  The present invention has been made to solve the above problems, and a first aspect of the present invention is to detect a change amount of a pixel value in a plurality of directions in a peripheral area of a specific pixel in image data including a plurality of pixels. A surrounding area change amount detecting means, an outer periphery area change amount detecting means for detecting a change amount of a pixel value in a plurality of directions in the outer peripheral area of the peripheral area, a change amount in the peripheral area, and a change amount in the outer peripheral area. Change amount summing means for summing up each of the plurality of directions, and pixel values in the extension region combining the peripheral region and the outer peripheral region based on the amount of change summed up in the plurality of directions with respect to the plurality of directions. Extended correlation degree determination means for determining a correlation degree, and a pixel value of the specific pixel from a pixel value adjacent to the specific pixel according to the determined correlation degree An image processing apparatus and processing method, and the method characterized by comprising an interpolation means for interpolating a is a program causing a computer to execute the. This brings about the effect | action that the pixel value of an interpolation object pixel is interpolated from the pixel value adjacent to an interpolation object pixel according to the correlation with respect to the several direction of the pixel value in an extended area | region.

  Further, in this first aspect, the peripheral correlation degree determining means for determining the correlation degree of the pixel values in the peripheral area with respect to the plurality of directions based on the amount of change in the pixel values in the peripheral area with respect to the plurality of directions; When the amount of change in at least two directions among the amounts of change in the plurality of directions in the peripheral region is less than a predetermined threshold, the degree of correlation in the extended region is selected. In other cases, the degree of correlation in the peripheral region is selected. Correlation level selection means for selecting the pixel value, and the interpolation means may interpolate the pixel value of the specific pixel from pixel values adjacent to the specific pixel according to the selected correlation level. Thereby, only when the amount of change in the surrounding area is less than the predetermined threshold, the pixel value of the interpolation target pixel is interpolated from the pixel value adjacent to the interpolation target pixel according to the degree of correlation in the expansion area.

  Further, in this first aspect, the interpolating means may perform the interpolation for the plurality of directions when the amount of change in at least two directions out of the amount of change totaled for the plurality of directions does not satisfy a predetermined threshold. The pixel value of the specific pixel may be interpolated from pixel values adjacent to the specific pixel equally. As a result, when the amount of change in the extended region is less than the predetermined threshold, the pixel value of the interpolation target pixel is evenly interpolated in each direction.

  According to a second aspect of the present invention, there is provided a peripheral region change amount detecting means for detecting a change amount of a pixel value in a plurality of directions in a peripheral region of a specific pixel in image data including a plurality of pixels, Further, a peripheral area change amount detecting means for detecting a change amount of a pixel value in a plurality of directions in the peripheral area, and a change amount adding means for adding the change amount in the peripheral area and the change amount in the peripheral area for each of the plurality of directions. And, if the amount of change in at least two directions among the amounts of change in the plurality of directions in the peripheral region is less than a predetermined threshold, select the amount of change added for each of the plurality of directions, otherwise Includes a change amount selecting means for selecting a change amount in the peripheral region, and a pixel value adjacent to the specific pixel according to the reciprocal of the selected change amount. An image processing apparatus characterized by comprising an interpolation means for interpolating a pixel value of a particular pixel. Accordingly, there is an effect that the pixel value of the interpolation target pixel is interpolated from the pixel value adjacent to the interpolation target pixel in accordance with the reciprocal of the amount of change in the plurality of directions of the pixel value in the peripheral region or the extension region.

  Further, in this second aspect, the interpolating unit may perform the correction with respect to the plurality of directions when the amount of change in at least two directions out of the amount of change added for each of the plurality of directions does not satisfy a predetermined threshold. The pixel value of the specific pixel may be interpolated from pixel values adjacent to the specific pixel equally. As a result, when the amount of change in the extended region is less than the predetermined threshold, the pixel value of the interpolation target pixel is evenly interpolated in each direction.

  According to the present invention, it is possible to achieve an excellent effect that an appropriate interpolation value can be generated even when the interpolation direction is unclear due to the color filter array during the demosaic process.

  Next, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a diagram illustrating a configuration example of an imaging apparatus which is an example of an image processing apparatus according to an embodiment of the present invention. This imaging apparatus includes a lens 110, an imaging element 120, an A / D (analog / digital) converter 130, and a camera signal processing unit 140.

  The lens 110 focuses light from the subject on the image sensor 120. The image sensor 120 is a photoelectric conversion circuit that converts light received from the lens 110 into an electrical signal. The image sensor 120 is provided with a red (R), green (G), and blue (B) color filter for each light receiving element of each pixel, and only light of each color component passes through the color filter to the light receiving element. Incident. Light incident on the light receiving element is photoelectrically converted by a photodiode and read out as an analog signal. The image pickup device 120 is, for example, a charge transfer type solid-state image pickup device represented by a CCD (Charge Coupled Device) or an XY address type solid state represented by a MOS (Metal Oxide Semiconductor). This is realized by an image sensor or the like. The A / D converter 130 converts the electrical signal photoelectrically converted by the image sensor 120 from an analog signal to a digital signal. This digital signal is input to the camera signal processing unit 140.

  The camera signal processing unit 140 includes an optical system correction circuit 141, a WB (white balance) circuit 142, an interpolation processing circuit 200, a gamma correction circuit 144, a Y signal processing circuit 145, a C signal processing circuit 146, and an LPF. (Low-pass filter) 147 and a thinning processing circuit 148 are provided.

  The optical system correction circuit 141 performs a digital clamp that matches a black level with a digital signal (image signal) input to the camera signal processing unit 140, a defect correction that corrects a defect of the image sensor 120, and a peripheral light amount drop of the lens 110. The image sensor 120 and the optical system are corrected such as shading correction to be corrected.

  The white balance circuit 142 performs white balance processing on an image signal that has passed through the optical system correction circuit 141 so that each color of RGB has the same level with respect to a white subject.

  The interpolation processing circuit 200 performs an interpolation process for generating three planes (RGB signals at the same spatial position) from RGB signals that are spatially out of phase. This interpolation processing is also called demosaic processing. The interpolation processing in the interpolation processing circuit 200 is a characteristic part of the present invention, and details thereof will be described later.

  The gamma correction circuit 144 performs gamma correction on RGB signals at the same spatial position. In this gamma correction, the RGB output from the white balance circuit 142 is set so that the photoelectric conversion characteristics of the entire image pickup device 120 and the subsequent image reproduction system are set to 1 in order to correctly represent the color gradation of the subject. Is a process of multiplying each color signal by a predetermined gain. The gamma-corrected image signal is supplied to the Y signal processing circuit 145 and the C signal processing circuit 146.

The Y signal processing circuit 145 generates a luminance signal (Y) from the RGB signals. The C signal processing circuit 146 generates color difference signals (Cr and Cb) from the RGB signals. For example, in the case of SD (Standard Definition), the Y signal processing circuit 145 and the C signal processing circuit 146 generate a luminance signal (Y) color difference signal (Cr and Cb) by the following equation. However, the coefficient of this formula shows a rough numerical value, and is not exact.
Y = 0.3R + 0.6G + 0.1B
Cr = R−Y = 0.7R−0.6G−0.1B
Cb = B−Y = −0.3R−0.6G + 0.9B

  The low-pass filter 147 is a filter that limits the passband of the color difference signal generated by the C signal processing circuit 146. For example, the low-pass filter 147 reduces the passband of the color difference signals (Cr and Cb) to ¼ of the sampling frequency fs, and sets Y: Cr: Cb = 4: 2: 2. The thinning processing circuit 148 thins out the sampling of the color difference signals (Cr and Cb).

  Here, demosaic processing in the interpolation processing circuit 200, which is a characteristic part of the present invention, will be described.

  FIG. 2 is a diagram illustrating an example of the arrangement of each color based on the Bayer arrangement. In the Bayer array, rows in which R pixels and G pixels are alternately arranged and rows in which G pixels and B pixels are alternately arranged are alternately repeated. Focusing on the G pixel, the G pixel is arranged in a checkered pattern, and the R pixel or the B pixel is arranged at a spatial position where the G pixel does not exist. If the sampling frequency of these pixels is fs, the pixel pitch is “1 / fs”.

  In the embodiment of the present invention, it is assumed that a G pixel component at a position where no G pixel exists (hereinafter, also referred to as “interpolation target pixel”) is interpolated from other G pixel components as demosaic processing. This is because the luminance component is important for realizing a high resolution, and the G pixel is a color component that becomes a main component when generating a luminance signal. However, the same method can be applied to R pixels or B pixels. In addition, Cy (cyan), Ye (yellow), or the like can be used as other color components that are the main components when generating a luminance signal.

  In the demosaic process, a signal up to a limit resolution that can be expressed by each of the RGB pixels of the image sensor is reproduced. As shown in FIG. 3, in the case of three plates, the theoretical limit value of the resolution is “(1/2) · fs” for each pixel of RGB. On the other hand, in the case of the Bayer array, the theoretical limit value of the resolution of the G pixel is “(1/2) · fs” in the horizontal direction and the vertical direction, but the resolution decreases in the oblique direction. Further, in the case of the Bayer arrangement, the pixel pitch is doubled for the R pixel and the B pixel, so the theoretical limit value of the resolution is “(1/4) · fs”. Therefore, since the sampling rates of the G pixel, the R pixel, and the B pixel are different as described above, a false signal is generated when the interpolation processing is inappropriate, resulting in a decrease in resolution and color falseness. .

  For example, pixel interpolation processing is considered by taking an input image (CPZ: Circular Zone Plate) composed of a resolution chart as shown in FIG. The resolution chart is a chart in which the central portion is a low-frequency signal and becomes a high-frequency signal as the distance from the center increases. The resolution chart has various directions even for signals of the same frequency. By inputting the resolution chart signal to the signal processing circuit, it is possible to analyze what processing is suitable for the various signals. . In the same figure, when the subject is a horizontal line such as point A on the vertical axis, the image signal is wavy in the vertical direction, but the correlation between the horizontal image signals is high. Interpolate using. In addition, when the subject is a vertical line such as point C on the horizontal axis, undulation is observed in the horizontal direction, while the correlation between the vertical image signals is high, so interpolation is performed using vertical pixels. To do. If interpolation processing can be performed using surrounding images in a direction with high correlation in this way, higher resolution can be realized.

  However, as described below, there are cases in which a direction with a high correlation cannot be detected well. For example, when black lines and white lines are arranged in the horizontal direction or the vertical direction at the Nyquist frequency ((1/2) · fs), it is impossible to determine which direction is actually arranged.

  For example, when the pixel value of the black line is “10” and the pixel value of the white line is “80”, when vertical lines are arranged as shown in FIG. 5A, and as shown in FIG. In the case where the horizontal lines are arranged, the signals are exactly the same when viewed with only G pixels. In this case, assuming that the correlating direction is unknown, if interpolation is performed using the average value of the surrounding four pixels as shown in FIG. 5C, the original signal cannot be reproduced. That is, although pixels for creating a correct interpolation value are present in the vicinity, a correlated direction cannot be detected, resulting in an interpolation error.

  In this regard, for a complete Nyquist signal, since the signal level of the light input to the image sensor due to the degradation of the lens MTF and the optical LPF approaches DC (direct current component), the correlation error is not so noticeable. The actual problem is that a correlation error may occur near a frequency slightly lower than the Nyquist frequency (see the dotted line portion in FIG. 3). The fact that the correlation cannot be detected at the Nyquist frequency in the horizontal direction or the vertical direction means that the correlation direction becomes unknown as it is closer to the Nyquist frequency and interpolation errors are likely to occur. Since there is no noise when the input image is artificial data, it is not a big problem because the correlation detection can be performed sensitively until just before the correlation detection becomes unknown, but in reality there is noise, so if the correlation detection is made sensitive, The interpolation direction becomes unstable due to noise, resulting in an unnatural image. Therefore, in order to stabilize the interpolation, it is necessary to make the correlation detection somewhat insensitive, but there is a problem that the resolution of the signal near the Nyquist frequency is lost as the correlation detection is made insensitive.

  On the other hand, even when the R pixel and the B pixel are viewed when detecting the correlation direction with respect to a signal near the Nyquist frequency, an interpolation error may occur. For example, as shown in FIG. 6A, green vertical lines in which the pixel values of the R pixel and the B pixel are “10” and the pixel values of the G pixel are “10” to “80” are arranged. In this case, when an interpolation target pixel of the G pixel is generated using the R pixel and the B pixel, the interpolation direction of the G pixel becomes the horizontal direction. As a result, the original vertical lines become a lattice pattern as shown in FIG. In this way, when interpolating G pixels having a large contribution ratio in luminance signal generation, if pixels of different colors are used, there is a possibility that a large side effect is caused on a chromatic color subject, and there is a high risk.

  As described above, in the Bayer array, there is a problem in principle of the color filter array that the accuracy of correlation detection decreases near the Nyquist frequency and the interpolation direction is not known, but here, first, there is a sense of incongruity when an interpolation error occurs. Consider what the subject is.

  For example, if the entire image is composed of 25 pixels of 5 × 5, and only the G pixel at that time is observed, in the case of the left side of FIG. I don't know what the image is. In such a case, any subject out of a grid, a horizontal line, and a vertical line may be used, and various interpolations can be assumed as shown on the left side of FIG.

  On the other hand, as shown on the left side of FIG. 7B, even when the entire image is composed of 25 pixels of 5 × 5, interpolation when the surrounding G is all aligned is a grid, horizontal line, vertical Lines cannot be assumed in any case, and it is assumed that the probability of a horizontal line is very high as shown on the right side of FIG. 7B. At this time, it is not possible to adopt G interpolated from the vertical direction. A natural image.

  As described above, the case where the sense of incongruity is caused by the interpolation is determined by comparison with the surrounding image such that the surrounding line is a horizontal line but only the target pixel is a vertical line. I understand.

  Therefore, in the embodiment of the present invention, when the accuracy of correlation detection deteriorates as in the vicinity of the Nyquist frequency and the interpolation direction is not clearly known, the interpolation is performed so as to match the direction of the surrounding interpolation direction. Natural interpolation with relatively high resolution.

  FIG. 8 is a diagram illustrating a first configuration example of the interpolation processing circuit 200 according to the embodiment of the present invention. The interpolation processing circuit 200 includes a peripheral region vertical direction change amount detection unit 211, a peripheral region horizontal direction change amount detection unit 212, an outer periphery region vertical direction change amount detection unit 221, and an outer periphery region horizontal direction change amount detection unit 222. , Adders 231 and 232, a peripheral region correlation degree determination unit 240, an extended region correlation degree determination unit 250, an extension determination unit 260, a correlation degree selection unit 270, and an interpolation generation unit 290.

  The peripheral region vertical direction change amount detection unit 211 and the peripheral region horizontal direction change amount detection unit 212 detect the amount of change in each direction of the pixel value in the peripheral region of the interpolation target pixel. For example, in FIG. 9, the center of 49 pixels of 7 × 7 is the interpolation target pixel X, and the peripheral region is a region composed of G pixels of 12 pixels G1 to G12. In this peripheral region, the peripheral region vertical direction change amount detection unit 211 detects the vertical direction change amount, and the peripheral region horizontal direction change amount detection unit 212 detects the horizontal direction change amount.

Here, the peripheral region vertical direction change amount BPF_V_X and the peripheral region horizontal direction change amount BPF_H_X are calculated by the following equations.
BPF_V_X = |-(G1 + G2) +2 (G6 + G7)-(G11 + G12) |
BPF_H_X = |-(G3 + G8) +2 (G4 + G9)-(G5 + G10) |
When this amount of change is large, there is a signal with a large amplitude in that direction, and it can be seen that the correlation is low. On the other hand, when the amount of change is small, it can be seen that the signal fluctuation is small in that direction and the correlation is high. This amount of change is equivalent to applying a band-pass filter having frequency characteristics (-1, 0, 2, 0, -1) as shown in FIG. The characteristics of this bandpass filter have a peak at “(1/4) · fs”, and output a value for a signal up to the limit resolution of “(1/2) · fs”. Yes.

  The outer peripheral region vertical direction change amount detection unit 221 and the outer peripheral region horizontal direction change amount detection unit 222 detect the amount of change in each direction of the pixel value in the outer peripheral region of the peripheral region of the interpolation target pixel. For example, in FIG. 9, the outer peripheral region is a region made up of G pixels for 12 pixels G20 to 31. In this outer peripheral region, the outer peripheral region vertical direction change amount detecting unit 221 detects the vertical direction change amount, and the outer peripheral region horizontal direction change amount detecting unit 222 detects the horizontal direction change amount.

Here, the outer peripheral region vertical direction change amount BPF_V_Y and the outer peripheral region horizontal direction change amount BPF_H_Y are calculated by the following equations as the sum of the change amounts centered at points a, b, c, and d in FIG.
BPF_V_Y = |-(G20 + G21) +2 (G3 + G4)-(G8 + G9) |
+ |-(G21 + G22) +2 (G4 + G5)-(G9 + G10) |
+ |-(G3 + G4) +2 (G8 + G9)-(G29 + G30) |
+ |-(G4 + G5) +2 (G9 + G10)-(G30 + G31) |
BPF_H_Y = |-(G23 + G25) +2 (G1 + G6)-(G2 + G7) |
+ |-(G1 + G6) +2 (G2 + G7)-(G24 + G26) |
+ |-(G25 + G27) +2 (G6 + G11)-(G7 + G12) |
+ |-(G6 + G11) +2 (G7 + G12)-(G26 + G28) |

The adders 231 and 232 output the amount of change in the extended region that combines the peripheral region and the outer peripheral region. That is, the adder 231 adds the peripheral region vertical direction change amount BPF_V_X and the outer peripheral region vertical direction change amount BPF_V_Y, and outputs the extended region vertical direction change amount BPF_V_Z. The adder 232 adds the peripheral region horizontal direction change amount BPF_H_X and the outer peripheral region horizontal direction change amount BPF_H_Y, and outputs an extended region horizontal direction change amount BPF_H_Z.
BPF_V_Z = BPF_V_X + BPF_V_Y
BPF_H_Z = BPF_H_X + BPF_H_Y

The surrounding area correlation degree determination unit 240 determines the degree of correlation of the pixel values in the surrounding area of the interpolation target pixel with respect to the horizontal direction and the vertical direction. This peripheral area correlation degree determination unit 240 calculates the vertical correlation degree S_V_X and the horizontal correlation degree S_H_X in the peripheral area by the following equations. When this degree of correlation is large, the correlation is high, and when the degree of correlation is small, the correlation is low.
S_H_X = BPF_V_X / (BPF_H_X + BPF_V_X)
S_V_X = BPF_H_X / (BPF_H_X + BPF_V_X) = 1.0−S_H_X

  For example, at point A in FIG. 4, S_H_X = 1 and S_V_X = 0, and the degree of correlation is high in the horizontal direction and there is no correlation in the vertical direction. At point B, S_H_X = S_V_X = 0.5, which indicates that the correlation in the horizontal direction and the vertical direction is the same, that is, the same image changes in both the horizontal direction and the vertical direction. At point C, S_H_X = 0 and S_V_X = 1, indicating that the degree of correlation in the vertical direction is high.

The extended region correlation degree determination unit 250 determines the degree of correlation of pixel values in the extended region with respect to the horizontal direction and the vertical direction. The extended area correlation degree determination unit 250 calculates the correlation degree S_V_Z in the vertical direction and the correlation degree S_H_Z in the horizontal direction in the extended area by the following equations.
S_H_Z = BPF_V_Z / (BPF_H_Z + BPF_V_Z)
S_V_Z = 1.0−S_H_Z

  The extension determination unit 260 determines whether or not the degree of correlation should be determined in the extension region. In order to determine the degree of correlation in the extended area, the pixel values of the pixels away from the interpolation target pixel are used. Therefore, when the degree of correlation can be determined in the surrounding area, the degree of correlation in the surrounding area is used as it is. Is considered good. Therefore, the extension determination unit 260 detects that the value of each amount of change in the horizontal direction and the vertical direction in the peripheral area is less than a predetermined threshold, and if not, the pixel value of the extension area The degree of correlation is adopted, and when the threshold value is exceeded, the degree of correlation of pixel values in the peripheral region is adopted.

  A case where the threshold value is not satisfied is a subject in the vicinity of the Nyquist frequency in the horizontal direction or the vertical direction or a DC component. In the vicinity of the Nyquist frequency, the interpolation accuracy is improved by using the extended region, and in the case of a subject having a DC component, there is no influence regardless of the correlation direction. Therefore, it is possible to operate without a side effect by a method of detecting the degree of correlation using the extended region when the value of each change amount in the horizontal direction and the vertical direction is less than a predetermined threshold value.

  Correlation degree selection section 270 selects the output of peripheral area correlation degree determination section 240 or extended area correlation degree determination section 250 according to the determination result by extension determination section 260. That is, the correlation degree selection unit 270 selects the correlation degree determined by the extended area correlation degree determination unit 250 when the change values in the horizontal direction and the vertical direction in the peripheral area are less than a predetermined threshold value. If the threshold value is exceeded, the correlation degree determined by the peripheral region correlation degree determination unit 240 is selected.

The interpolation generation unit 290 interpolates pixel values adjacent to the interpolation target pixel according to the correlation level selected by the correlation level selection unit 270. For example, when the correlation degree of the extended region is selected by the correlation degree selection unit 270, the interpolation value GX in the interpolation target pixel X in FIG. 9 is calculated as follows.
GX = ((G6 + G7) × S_H_Z + (G4 + G9) × S_V_Z) / 2

  That is, the interpolation value GX is interpolated by applying a large weight to the direction in which the degree of correlation is high. For example, at point A in FIG. 4, GX = (G6 + G7) / 2 and interpolation is performed in the horizontal direction. At point B, GX = (G6 + G7 + G4 + G9) / 4, and horizontal and vertical pixels are interpolated with equal weights. At point C, GX = (G4 + G9) / 2, and interpolation is performed in the vertical direction.

  FIG. 11 is a diagram illustrating a processing procedure of an image processing method according to the first configuration example of the interpolation processing circuit 200 according to the embodiment of the present invention.

  First, the peripheral region vertical direction change amount detection unit 211 and the peripheral region horizontal direction change amount detection unit 212 calculate the amount of change in the vertical and horizontal pixel values in the peripheral region (step S911). As a result of the determination in the extension determination unit 260, when the value of each change amount in the horizontal direction and the vertical direction in the peripheral region exceeds a predetermined threshold value, interpolation is performed based on the change amount of the pixel value in the peripheral region ( Step S913). That is, if the horizontal direction change amount in the peripheral region is smaller than the vertical direction change amount in the peripheral region (step S912), interpolation is performed in the horizontal direction (step S922). On the other hand, if the horizontal direction change amount in the peripheral region is not smaller than the vertical direction change amount in the peripheral region (step S912), interpolation is performed in the vertical direction (step S921).

  As a result of the determination by the extension determination unit 260, when the value of each change amount in the horizontal direction and the vertical direction in the peripheral region is less than a predetermined threshold value (step S913), based on the change amount of the pixel value in the extension region. Is interpolated. For this reason, the outer peripheral region vertical direction change amount detecting unit 221 and the outer peripheral region horizontal direction change amount detecting unit 222 calculate the horizontal direction and vertical direction change amounts in the outer peripheral region, and the adders 231 and 232 change the peripheral region change amount. Is added to calculate the amount of change in the pixel value in the extended region (step S914). If the horizontal direction change amount in the extension region is smaller than the vertical direction change amount in the extension region (step S916), interpolation is performed in the horizontal direction (step S924). On the other hand, if the horizontal direction change amount in the extension region is not smaller than the vertical direction change amount in the extension region (step S916), interpolation is performed in the vertical direction (step S923).

  In the processing procedure shown in the figure, an example in which the interpolation direction is determined based on the amount of change with respect to each direction has been described. It may be generated.

  Up to this point, the application example of the demosaic processing to the Bayer array in the embodiment of the present invention has been described. However, the present invention is also applicable to other color filter arrays. Below, the example of application to what is called a diagonal arrangement | sequence is demonstrated.

  FIG. 12 is a diagram illustrating an example of another color filter array that is a target of demosaic processing according to the embodiment of this invention. This arrangement is a so-called diagonal pixel arrangement in which each pixel pitch in the horizontal and vertical directions is √2d, and each pixel is shifted by ½ of the pixel pitch √2d for each row and each column. Are arranged in units of four rows of RG lines in which G is arranged alternately, G lines in which only G is arranged, GB lines in which B and G are arranged alternately, and G lines in which only G is arranged. ing.

  In this diagonal pixel arrangement, when only G pixels are observed, the same problem occurs in the case of “(1/4) · fs” and “(1/2) · fs” in the horizontal and vertical directions. Is known to happen. In the case of “(1/2) · fs” in the horizontal and vertical directions, exactly the same thing as the Bayer array occurs, but in the case of “(1/4) · fs” in the horizontal and vertical directions, correlation detection is not possible. What is clear is unique to this sequence.

  Thus, when the color filter array is different, the frequency at which the correlation of interpolation is not known may be different. However, even in such a case, the interpolation accuracy can be improved by expanding the correlation detection area from the peripheral area to the extended area.

For example, as shown in FIG. 13, when a subject with a vertical line of “(1/4) · fs” is photographed, even if only the originally existing G pixel is observed, the vertical or horizontal line is used. It cannot be determined whether it exists. At this time, if correlation detection is performed in the peripheral region, the following is obtained.
Vertical variation = 0
Horizontal change = 0
Right oblique direction change amount = 400
Left oblique direction change amount = 400
This value is unnatural for a correlated subject.

Here, when viewed from a resolution chart having various angles as shown in FIG. 4, the amount of change at point A is as follows.
Vertical change amount = large Horizontal change amount = 0
Right oblique direction change amount = Middle Left oblique direction change amount = Medium

Similarly, at point C:
Vertical variation = 0
Horizontal change amount = Large Right oblique direction change amount = Medium Left oblique direction change amount = Medium

Similarly, at point D:
Vertical change amount = Small Horizontal change amount = Large Right oblique direction change amount = Large Left oblique direction change amount = Medium

  That is, except for “(1/4) · fs”, it can be seen that the amount of change in the horizontal and vertical directions and the amount of change in the oblique direction change continuously.

  In contrast, in the case of “(1/4) · fs” in the horizontal and vertical directions, the change in the horizontal and vertical directions is “0”, but the change in the oblique direction is a large value. It can be seen that the correlation is discontinuous. As described above, in the case of the diagonal pixel arrangement, the determination as to whether or not “(1/4) · fs” is in the horizontal direction and the vertical direction where the correlation of the G pixel is unknown is detected by looking at the amount of change in the four directions. It is considered possible.

  FIG. 14 is a diagram illustrating a second configuration example of the interpolation processing circuit 200 according to the embodiment of the present invention. In addition to the first configuration example, the second configuration example of the interpolation processing circuit 200 includes a peripheral region right oblique direction change amount detection unit 213, a peripheral region left oblique direction change amount detection unit 214, and a compensation processing unit 280. And further.

  The peripheral area vertical direction change amount detection unit 211, the peripheral area horizontal direction change amount detection unit 212, the peripheral area right oblique direction change amount detection unit 213, and the peripheral area left oblique direction change amount detection unit 214 are arranged in the peripheral region of the interpolation target pixel. The amount of change in each direction of the pixel value is detected. For example, in FIG. 15, the center is the interpolation target pixel X, and the peripheral area is an area composed of G pixels of 16 pixels G1 to G16. In this peripheral region, the peripheral region vertical direction change amount detection unit 211 detects the vertical direction change amount, the peripheral region horizontal direction change amount detection unit 212 detects the horizontal direction change amount, and the peripheral region right oblique direction change amount The detection unit 213 detects a change amount in the right oblique direction (a direction inclined 45 degrees upward to the right shoulder), and the peripheral region left oblique direction change amount detection unit 214 detects a change in the left oblique direction (a direction inclined 45 degrees upward to the left shoulder). Detect the amount.

Here, the peripheral region vertical direction change amount BPF_V_X, the peripheral region horizontal direction change amount BPF_H_X, the peripheral region right oblique direction change amount BPF_R_X, and the peripheral region left oblique direction change amount BPF_L_X are calculated by the following equations.
BPF_V_X = | −G1 + 2 × G5−G11 | + | −G5 + 2 × G11−G15 |
+ | −G2 + 2 × G6−G12 | + | −G6 + 2 × G12−G16 |
BPF_H_X = | −G4 + 2 × G5−G6 | + | −G5 + 2 × G6−G7 |
+ | −G10 + 2 × G11−G12 | + | −G11 + 2 × G12−G13 |
BPF_R_X = | −G2 + 2 × G3−G5 | + | −G5 + 2 × G8−G10 |
+ | −G7 + 2 × G9−G12 | + | −G12 + 2 × G14−G15 |
BPF_L_X = | −G1 + 2 × G3−G6 | + | −G6 + 2 × G9−G13 |
+ | −G4 + 2 × G8−G11 | + | −G11 + 2 × G14−G16 |

  The outer peripheral region vertical direction change amount detection unit 221 and the outer peripheral region horizontal direction change amount detection unit 222 detect the amount of change in each direction of the pixel value in the outer peripheral region of the peripheral region of the interpolation target pixel. For example, in FIG. 15, the outer peripheral area is an area composed of G pixels for 28 pixels of G20 to 47. In this outer peripheral region, the outer peripheral region vertical direction change amount detecting unit 221 detects the vertical direction change amount, and the outer peripheral region horizontal direction change amount detecting unit 222 detects the horizontal direction change amount.

Here, the outer peripheral region vertical direction change amount BPF_V_Y and the outer peripheral region horizontal direction change amount BPF_H_Y are calculated by the following equations as the sum of the change amounts centered at points a, b, c, and d in FIG.
BPF_V_Y = | −G20 + 2 × G23−G1 | + | −G23 + 2 × G1−G5 |
+ | −G21 + 2 × G24−G2 | + | −G24 + 2 × G2−G6 |
+ | −G26 + 2 × G31−G35 | + | −G31 + 2 × G35−G38 |
+ | −G27 + 2 × G4−G10 | + | −G4 + 2 × G10−G39 |
+ | −G28 + 2 × G7−G13 | + | −G7 + 2 × G13−G40 |
+ | −G29 + 2 × G32−G36 | + | −G32 + 2 × G36−G41 |
+ | −G11 + 2 × G15−G43 | + | −G15 + 2 × G43−G46 |
+ | −G12 + 2 × G16−G44 | + | −G16 + 2 × G44−G47 |
BPF_H_Y = | −G22 + 2 × G23−G24 | + | −G23 + 2 × G24−G25 |
+ | −G27 + 2 × G1−G2 | + | −G1 + 2 × G2−G28 |
+ | −G30 + 2 × G31−G4 | + | −G31 + 2 × G4−G5 |
+ | −G34 + 2 × G35−G10 | + | −G35 + 2 × G10−G11 |
+ | −G6 + 2 × G7−G32 | + | −G7 + 2 × G32−G33 |
+ | −G12 + 2 × G13−G36 | + | −G13 + 2 × G36−G37 |
+ | −G39 + 2 × G15−G16 | + | −G15 + 2 × G16−G40 |
+ | −G42 + 2 × G43−G44 | + | −G43 + 2 × G44−G45 |

  The adders 231 and 232 output the amount of change in the extended region including the peripheral region and the outer peripheral region, as in the first configuration example.

  The surrounding area correlation degree determination unit 240 determines the degree of correlation of the pixel values in the surrounding area of the interpolation target pixel with respect to the horizontal direction, the vertical direction, the right oblique direction, and the left oblique direction. In this case, the peripheral region correlation degree determination unit 240 can output the direction with the smallest amount of change among the horizontal direction, the vertical direction, the right oblique direction, and the left oblique direction as a direction with a high degree of correlation.

The extension determination unit 260 determines whether or not the degree of correlation should be determined in the extension region. In the second configuration example, the extension determination unit 260 employs the correlation value of the pixel values in the extension region when “(1/4) · fs” is obtained in the horizontal direction and the vertical direction. Therefore, it is determined whether or not the following two conditions are satisfied.
Condition (1): Sorting in four direction change amounts, the direction of the smallest value and the second smallest value is the horizontal direction and the vertical direction. However, the order may be any.
Condition (2): The difference between the amount of change in the horizontal direction and the vertical direction is not more than a predetermined threshold.

  When both of the conditions (1) and (2) are satisfied, the correlation value of the pixel values in the extended region is adopted, and when at least one of the conditions is not satisfied, the correlation value of the pixel values in the peripheral region is adopted. Further, when only the condition (1) is satisfied and the condition (2) is not satisfied, it is considered that the image is composed of a direct current component, and therefore, interpolation is performed equally from adjacent pixel values.

  A correlated object does not satisfy the above conditions (1) and (2). Further, when noise is included in the DC component image, the conditions (1) and (2) may be satisfied. However, since the G pixel interpolation direction is not important in such an image, the problem is particularly problematic. Don't be.

  Similar to the first configuration example, the extended region correlation degree determination unit 250 determines the correlation degree of the pixel values in the extended region with respect to the horizontal direction and the vertical direction. The surrounding area correlation degree determination unit 240 according to the second configuration example determines the degree of correlation not only in the horizontal direction and the vertical direction but also in the right oblique direction and the left oblique direction. When the condition (1) is satisfied, Since the amount of change is large in the diagonal direction, diagonal interpolation is not assumed. Therefore, in this case, the expanded region correlation degree determination unit 250 outputs a direction with a small amount of change between the horizontal direction and the vertical direction as a direction with a high degree of correlation.

  Correlation degree selection section 270 selects the output of peripheral area correlation degree determination section 240 or extended area correlation degree determination section 250 according to the determination result by extension determination section 260. That is, the correlation degree selection unit 270 selects the correlation degree determined by the extended region correlation degree determination unit 250 when both of the above conditions (1) and (2) are satisfied, and does not satisfy at least one of them. In this case, the correlation degree determined by the peripheral region correlation degree determination unit 240 is selected.

The compensation processing unit 280 performs compensation processing when the degree of correlation is detected in the extended region. Even when the above conditions (1) and (2) are satisfied, the interpolation direction is not constant and unstable with a signal in which the signals of “(1/4) · fs” are listed forever like artificial data There is a risk. Therefore, the compensation processing unit 280 determines that there is no correlation in any direction when the following expression is satisfied, and instructs the interpolation generation unit 290.
｜ Horizontal change amount of extended area−Vertical change amount of extended area | <Threshold

  The interpolation generation unit 290 interpolates pixel values adjacent to the interpolation target pixel according to the correlation level selected by the correlation level selection unit 270. For example, when the correlation degree of the surrounding area is selected by the correlation degree selecting unit 270, the interpolation value GX in the interpolation target pixel X in FIG. 15 is GX = (G8 + G9) / if the horizontal direction change amount of the surrounding area is minimum. 2. If the amount of change in the right oblique direction of the peripheral region is the minimum, GX = (G6 + G11) / 2 is calculated.

  In addition, when the extension determination unit 260 determines that the component is a direct current component, the interpolation generation unit 290 generates an interpolation value evenly from adjacent pixel values. The same applies to the case where the compensation processing unit 280 determines that there is no correlation in any direction, and the interpolation generation unit 290 generates interpolation values equally from adjacent pixel values. In this case, for example, the interpolation value GX in the interpolation target pixel X in FIG. 15 is calculated as GX = (G5 + G6 + G11 + G12) / 4.

If the compensation processing unit 280 determines that the interpolation direction is not constant even if the degree of correlation is detected in the extended region, an equal interpolation value and an interpolation value determined in the extended region are determined by a predetermined coefficient α. You may mix. In this case, the interpolation value GX is calculated as follows.
GX = α × ((G5 + G6 + G11 + G12) / 4)
+ (Α-1) × (interpolated value determined in the extended region)

  FIG. 16 is a diagram illustrating a processing procedure of an image processing method according to the second configuration example of the interpolation processing circuit 200 according to the embodiment of the present invention.

  First, the peripheral region vertical direction change amount detection unit 211, the peripheral region horizontal direction change amount detection unit 212, the peripheral region right oblique direction change amount detection unit 213, and the peripheral region left oblique direction change amount detection unit 214 perform the vertical direction in the peripheral region. The change amount of the pixel value in the horizontal direction, the right oblique direction, and the left oblique direction is calculated (step S931). As a result of the determination in the extension determination unit 260, when it is determined that the direction of the smallest value and the second smallest value among the calculated change amounts in the four directions is not the horizontal direction and the vertical direction (step S932). The interpolation generation unit 290 generates an interpolation value with the direction in which the change amount of the peripheral area is the smallest as the interpolation direction (step S941).

  As a result of the determination in the extension determination unit 260, it is determined that the direction of the smallest value and the second smallest value among the calculated amounts of change in the four directions is the horizontal direction and the vertical direction (step S932), and When it is determined that the difference between the change amounts in the horizontal direction and the vertical direction is equal to or greater than the predetermined threshold (step S933), since it is considered to be a direct current component, the interpolation generation unit 290 equalizes the adjacent pixel values. An interpolation value is generated (step S942).

  As a result of the determination in the extension determination unit 260, it is determined that the direction of the smallest value and the second smallest value among the calculated amounts of change in the four directions is the horizontal direction and the vertical direction (step S932), and If it is determined that the difference between the change amounts in the horizontal direction and the vertical direction is less than the predetermined threshold (step S933), it is considered that “(1/4) · fs” is obtained in the horizontal direction and the vertical direction. Interpolation is performed based on the amount of change in the pixel value in the extended region. For this reason, the outer peripheral region vertical direction change amount detecting unit 221 and the outer peripheral region horizontal direction change amount detecting unit 222 calculate the horizontal direction and vertical direction change amounts in the outer peripheral region, and the adders 231 and 232 change the peripheral region change amount. Is added to calculate the amount of change in the pixel value in the extended region (step S934).

  As a result, when the compensation processing unit 280 determines that the absolute value of the difference between the horizontal direction change amount of the extension region and the vertical direction change amount of the extension region is less than the predetermined threshold (step S935), Therefore, the interpolation generation unit 290 generates an interpolation value equally from adjacent pixel values (step S942).

  On the other hand, when the compensation processing unit 280 determines that the absolute value of the difference between the horizontal direction change amount of the extension region and the vertical direction change amount of the extension region is equal to or greater than a predetermined threshold, the change amount in the extension region is set. In response, an interpolated value is generated. That is, if the horizontal direction change amount in the extension region is smaller than the vertical direction change amount in the extension region (step S936), interpolation is performed in the horizontal direction (step S944). On the other hand, if the horizontal direction change amount in the extension region is not smaller than the vertical direction change amount in the extension region (step S936), interpolation is performed in the vertical direction (step S943).

  FIG. 17 is a diagram showing an example of the effect in the embodiment of the present invention. FIG. 17A shows an example of an image after the interpolation processing when the correlation is detected only in the peripheral region in the above-described oblique pixel arrangement. In this example, it can be seen that a part of the region surrounded by the central ellipse is interpolated in the wrong direction.

  On the other hand, FIG. 17B is an image example after the interpolation processing in the case where the correlation is detected in the extended region in the above-described oblique pixel array. In this example, it can be seen that the portion that was interpolated in the wrong direction in FIG.

In the embodiment of the present invention, the example in which the degree of correlation is obtained by comparing the amount of change in each direction of the pixel value and the interpolation value is generated according to the degree of correlation has been described. Interpolation may be performed. In this case, as described below, the interpolation value GX can be obtained by weighted average using the reciprocal of the amount of change in each direction of the pixel value as a weight.

Here, assuming the interpolation target pixel X in FIG. 15, each interpolation value can be calculated by the following equation.
Horizontal interpolation value = (G8 + G9) / 2
Vertical interpolation value = (G3 + G14) / 2
Interpolated value in right oblique direction = (G6 + G11) / 2
Left diagonal direction interpolation value = (G5 + G12) / 2

  FIG. 18 is a diagram showing a modification of the interpolation processing circuit 200 in the embodiment of the present invention. This modification is for performing the above-described weighted interpolation, and is replaced with the peripheral region correlation degree determination unit 240, the extended region correlation degree determination unit 250, and the correlation degree selection unit 270, as compared with the configuration example of FIG. Thus, a change amount selection unit 277 is provided. The change amount selection unit 277 selects a change amount of the peripheral region or the outer peripheral region according to the determination result by the extension determination unit 260. The interpolation generation unit 290 generates an interpolation value in the interpolation target pixel based on the selected change amount.

  FIG. 19 is a diagram illustrating a configuration example of the change amount selection unit 277 according to the embodiment of the present invention. The change amount selection unit 277 includes a vertical direction change amount selector 271, a horizontal direction change amount selector 272, a right oblique direction change amount selector 273, and a left oblique direction change amount selector 274.

  The vertical direction change amount selector 271 determines the vertical direction change amount of the peripheral region from the peripheral region vertical direction change amount detection unit 211 or the vertical direction change of the extension region from the adder 231 according to the determination result of the extension determination unit 260. One of the quantities is selected.

  The horizontal direction change amount selector 272 determines the horizontal direction change amount of the peripheral region from the peripheral region horizontal direction change amount detection unit 212 or the horizontal direction change of the extension region from the adder 232 according to the determination result of the extension determination unit 260. One of the quantities is selected.

  The right oblique direction change amount selector 273 selects either the right oblique direction change amount or the maximum value of the peripheral region from the peripheral region right oblique direction change amount detection unit 213 according to the determination result of the extension determination unit 260. Either one is selected.

  The left oblique direction change amount selector 274 determines either the left oblique direction change amount or the maximum value of the peripheral region from the peripheral region left oblique direction change amount detection unit 214 according to the determination result of the extension determination unit 260. Either one is selected.

  When the degree of correlation in the extended region is adopted by the extension determination unit 260, the right oblique direction change amount selector 273 and the left oblique direction change amount selector 274 output a maximum value as the change amount. This means that there is no correlation between the right oblique direction and the left oblique direction.

  FIG. 20 is a diagram showing a modification of the processing procedure of the image processing method according to the second configuration example of the interpolation processing circuit 200 in the embodiment of the present invention.

  In this modification, the processes in steps S961 and S963 are different from the process procedure of FIG. In step S941 in FIG. 16, the direction in which the amount of change in the peripheral region is minimized is selected, and interpolation is performed according to the direction. In this modification, weighted interpolation by weighted average is performed using the reciprocal of the amount of change in the peripheral region as a weight. Is performed (step S961). In addition, in steps S936, S943, and S944 in FIG. 16, the direction in which the amount of change in the extended region is minimized is selected, and interpolation is performed according to the direction. In this modification, the reciprocal of the amount of change in the extended region is weighted. As a result, weighted interpolation by weighted average is performed (step S963).

  As described above, according to the embodiment of the present invention, in the demosaic process, by detecting the amount of change in the pixel value in each direction in the extended region including the outer peripheral region of the peripheral region of the interpolation target pixel, the surrounding interpolation is performed. Interpolation values can be generated according to the direction, so that the interpolation accuracy can be improved and natural interpolation can be performed. Moreover, noise resistance can be improved by adding the amount of change in different regions.

  In the embodiment of the present invention, the specific example of the extended region has been described. However, the present invention is not limited to this, and may be further expanded as necessary in order to improve the interpolation accuracy. .

  Further, the embodiment of the present invention shows an example for embodying the present invention, and has a corresponding relationship with the invention specific matter in the claims as shown below, but is not limited thereto. However, various modifications can be made without departing from the scope of the present invention.

  The processing procedure described in the embodiment of the present invention may be regarded as a method having a series of these procedures, and a program for causing a computer to execute these series of procedures or a recording medium storing the program May be taken as

It is a figure which shows the structural example of the imaging device which is an example of the image processing apparatus in embodiment of this invention. It is a figure which shows the example of an arrangement | sequence of each color by a Bayer arrangement | sequence. It is a figure which shows the limiting resolution of 3 plates and a Bayer arrangement. It is a figure which shows a resolution chart (CPZ). It is a figure which shows the problem in a Bayer arrangement | sequence. It is another figure which shows the problem in a Bayer arrangement. It is a figure which shows the example of interpolation in a Bayer arrangement | sequence. It is a figure which shows the 1st structural example of the interpolation processing circuit 200 in embodiment of this invention. It is the figure which extracted G pixel in Bayer arrangement. It is a figure which shows the frequency characteristic of a band pass filter. It is a figure which shows the process sequence of the image processing method by the 1st structural example of the interpolation processing circuit 200 in embodiment of this invention. It is a figure which shows an example of the other color filter arrangement | sequence (diagonal pixel arrangement | sequence) used as the object of the demosaic process of embodiment of this invention. It is a figure which shows the problem in a diagonal pixel arrangement | sequence. It is a figure which shows the 2nd structural example of the interpolation processing circuit 200 in embodiment of this invention. It is the figure which extracted G pixel in the diagonal pixel arrangement. It is a figure which shows the process sequence of the image processing method by the 2nd structural example of the interpolation processing circuit 200 in embodiment of this invention. It is a figure which shows an example of the effect in embodiment of this invention. It is a figure which shows the modification of the interpolation processing circuit 200 in embodiment of this invention. It is a figure which shows one structural example of the variation | change_quantity selection part 277 in embodiment of this invention. It is a figure which shows the modification of the process sequence of the image processing method by the 2nd structural example of the interpolation process circuit 200 in embodiment of this invention.

Explanation of symbols

DESCRIPTION OF SYMBOLS 110 Lens 120 Image pick-up element 130 A / D converter 140 Camera signal processing part 141 Optical system correction circuit 142 White balance circuit 144 Gamma correction circuit 145 Y signal processing circuit 146 C signal processing circuit 147 Low-pass filter 148 Decimation processing circuit 200 Interpolation processing circuit 211 Peripheral area vertical direction change detection unit 212 Peripheral area horizontal direction change detection part 213 Peripheral area right oblique direction change detection part 214 Peripheral area left oblique direction change detection part 221 Outer peripheral area vertical direction change detection part 222 Outer peripheral area Horizontal direction change detection unit 231, 232 Adder 240 Peripheral region correlation degree determination unit 250 Extended region correlation degree determination unit 260 Extension determination unit 270 Correlation degree selection unit 271 Vertical direction change amount selector 272 Horizontal direction change amount selector 273 Right Diagonal change amount selector 274 Left diagonal Direction change amount selector 277 changes amount selector 280 compensation processor 290 interpolation data generation unit

Claims (4)

A peripheral region change amount detecting means for detecting a change amount in a plurality of directions of a pixel value in a peripheral region of a specific pixel in image data composed of a plurality of pixels;
A peripheral region change amount detecting means for detecting a change amount of a pixel value in a plurality of directions in a further peripheral region of the peripheral region;
Change amount adding means for adding the change amount in the peripheral region and the change amount in the outer peripheral region for each of the plurality of directions;
An expanded correlation degree determining means for determining a correlation degree of the pixel values in the extended area combining the peripheral area and the outer peripheral area based on the amount of change added for each of the plurality of directions;
Peripheral correlation degree determining means for determining a correlation degree of the pixel values in the peripheral region with respect to the plurality of directions based on a change amount of the pixel value in the peripheral region with respect to the plurality of directions;
When the amount of change in at least two directions among the amounts of change in the peripheral region in the plurality of directions is less than a predetermined threshold, the degree of correlation in the extended region is selected, and in other cases, the correlation in the peripheral region is selected. Correlation degree selecting means for selecting the degree;
The selected the particular said specific to that images processing device and a interpolation means for interpolating the pixel value of the pixel from the pixel values adjacent to the pixel in accordance with the degree of correlation. The interpolating unit is adjacent to the specific pixel evenly with respect to the plurality of directions when the amount of change in at least two directions out of the amount of change combined for each of the plurality of directions is less than a predetermined threshold. Interpolate the pixel value of the specific pixel from the pixel value
The image processing apparatus 請 Motomeko 1 wherein. A peripheral region change amount detection procedure for detecting a change amount in a plurality of directions of pixel values in a peripheral region of a specific pixel in image data including a plurality of pixels;
A peripheral region change amount detection procedure for detecting a change amount in a plurality of directions of pixel values in a further peripheral region of the peripheral region;
A change amount adding procedure for adding the change amount in the peripheral region and the change amount in the outer peripheral region for each of the plurality of directions;
An extended correlation degree determination procedure for determining a correlation degree of the pixel values in the extension area combining the peripheral area and the outer peripheral area based on the amount of change added for each of the plurality of directions;
A peripheral correlation degree determination procedure for determining a correlation degree of the pixel values in the peripheral area with respect to the plurality of directions based on a change amount of the pixel value in the peripheral area with respect to the plurality of directions;
When the amount of change in at least two directions among the amounts of change in the peripheral region in the plurality of directions is less than a predetermined threshold, the degree of correlation in the extended region is selected, and in other cases, the correlation in the peripheral region is selected. Correlation degree selection procedure for selecting the degree,
The selected interpolation procedure to that images processing method comprising the interpolating a pixel value of the particular pixel from the pixel values adjacent to the specific pixel in accordance with correlation. A peripheral region change amount detection procedure for detecting a change amount in a plurality of directions of pixel values in a peripheral region of a specific pixel in image data including a plurality of pixels;
A peripheral region change amount detection procedure for detecting a change amount in a plurality of directions of pixel values in a further peripheral region of the peripheral region;
A change amount adding procedure for adding the change amount in the peripheral region and the change amount in the outer peripheral region for each of the plurality of directions;
An extended correlation degree determination procedure for determining a correlation degree of the pixel values in the extension area combining the peripheral area and the outer peripheral area based on the amount of change added for each of the plurality of directions;
A peripheral correlation degree determination procedure for determining a correlation degree of the pixel values in the peripheral area with respect to the plurality of directions based on a change amount of the pixel value in the peripheral area with respect to the plurality of directions;
When the amount of change in at least two directions among the amounts of change in the peripheral region in the plurality of directions is less than a predetermined threshold, the degree of correlation in the extended region is selected, and in other cases, the correlation in the peripheral region is selected. Correlation degree selection procedure for selecting the degree,
Help program to execute the interpolation procedure on the computer to interpolate the pixel value of the particular pixel from the pixel values adjacent to the specific pixel in accordance with the selected correlation.