JP2008160586A - Image processor, image processing method, and imaging apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processor, an image processing method and an imaging apparatus by which noise of an image is removed without detecting the noise. <P>SOLUTION: The image processor, which processes an input image in a pixel unit, is provided with; a feature image generation part which generates a predetermined feature image from the input image; an interpolation value calculation part which calculates a plurality of interpolation values for interpolating a pixel value for every pixel based on the feature image generated by the feature image generation part; a correction value calculation part which calculates feature values for every pixel from the input image and calculates correction values for correcting each of the plurality of interpolation values calculated for every pixel; an interpolation value correction part which corrects each of the plurality of interpolation values calculated for every pixel in the interpolation value calculation part based on the correction values for every pixel calculated in the correction value calculation part; and a correction image generation part which corrects the feature image based on the interpolation values for every pixel corrected in the interpolation value correction part. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing device, an image processing method, and an imaging device.

  In recent years, digital cameras have become widespread, such as portable communication devices such as mobile phones having a digital camera function. The digital camera as described above performs imaging using an image sensor using a plurality of imaging elements such as a charge coupled device (CCD) and a complementary metal oxide semiconductor (CMOS).

  However, as the resolution of digital cameras increases, the signal-to-noise ratio (SNR) decreases, that is, noise that occurs in the captured image becomes a problem. Here, the cause of the noise includes, for example, an increase in the number of the image pickup elements with an increase in resolution, and a decrease in the amount of received light per image pickup element and a decrease in signal level.

  Under such circumstances, a technique for reducing noise by correcting a captured image has been developed. For example, as a technique for detecting an edge of a captured image, performing noise reduction processing on a pixel that is not detected as an edge, and not performing noise reduction processing on a pixel detected as an edge, for example, Patent Document 1 Is mentioned. Further, as a technique for classifying image data into a contour portion, a noise portion, and a non-contour portion, performing noise removal processing on the noise portion, and emphasizing processing on the contour portion, for example, Patent Document 2 is cited.

JP 2005-303731 A JP 2003-219180 A

  However, the edge of the captured image is detected, noise reduction processing is performed on pixels that are not detected as edges, and noise reduction processing is not performed on pixels detected as edges, Conventional image processing techniques, such as a technique that classifies noise parts and non-contour parts, performs noise removal processing on the noise parts, and performs enhancement processing on the contour parts, need to detect noise and perform specific processing. , Circuit and algorithm become complicated. In addition, since noise removal processing is not performed on pixels detected as edges, noise on the edges cannot be removed. Therefore, the image corrected by the conventional image processing technique cannot remove noise existing on the edge, and thus becomes an unnatural image. Here, the edge refers to a boundary between a portion having a high luminance and a portion having a low luminance in the image (that is, a light / dark boundary of the image).

  Therefore, the present invention has been made in view of the above problems, and an object of the present invention is a new and improved image noise that can be removed without detecting noise. An object is to provide an image processing apparatus, an image processing method, and an imaging apparatus.

  In order to achieve the above object, according to a first aspect of the present invention, there is provided an image processing apparatus that processes an input image in units of pixels, and a feature image generation unit that generates a predetermined feature image from the input image; Based on the feature image generated by the feature image generation unit, an interpolation value calculation unit that calculates a plurality of interpolation values for each pixel to interpolate pixel values, calculates a feature value for each pixel from the input image, and A correction value calculation unit that calculates a correction value for correcting each of the plurality of interpolation values calculated for each pixel based on the value, and a correction value for each pixel calculated by the correction value calculation unit based on the correction value An interpolation value correction unit that corrects each of the interpolation values calculated for each pixel in the interpolation value calculation unit, and a correction image that corrects the feature image based on the interpolation value for each pixel corrected in the interpolation value correction unit Living The image processing apparatus and a part is provided.

  The image processing apparatus includes, for example, a feature image generation unit, an interpolation value calculation unit, a correction value calculation unit, an interpolation value correction unit, and a correction image generation unit. The feature image generation unit can generate a predetermined feature image from an input image input to the image processing apparatus. Here, examples of the predetermined feature image include a color difference image obtained by removing a luminance signal from an input image, and an edge image obtained by extracting an edge based on the luminance difference. The interpolation value calculation unit can calculate a plurality of interpolation values for interpolating pixel values for each pixel. The correction value calculation unit can calculate a feature value for each pixel from the input image, and calculate a correction value for correcting each of the plurality of interpolation values for each pixel calculated by the interpolation value calculation unit based on the feature value. Here, examples of the feature value include a vertical change amount and a horizontal change amount in a pixel for which a correction value is calculated. The interpolation value correction unit can correct each of the interpolation values calculated for each pixel by the interpolation value calculation unit based on the correction value calculated by the correction value calculation unit. The corrected image generation unit can correct the feature image based on the interpolation value for each pixel corrected by the interpolation value correction unit. With this configuration, it is possible to remove image noise regardless of whether or not the noise is on the edge and without detecting noise.

  The feature image generation unit generates a color difference image from the input image, and the interpolation value calculation unit calculates a pixel value of a pixel in a vertical direction adjacent to the pixel of interest for each pixel of interest for which an interpolation value is calculated. A first interpolation value calculation unit that calculates an average value; and a second interpolation value calculation unit that calculates an average value of pixel values in pixels in the horizontal direction adjacent to the pixel of interest. A feature value calculator that calculates the amount of change in the vertical direction and the amount of change in the horizontal direction as a feature value in the pixel of interest, and a correction value corresponding to the amount of change in the vertical direction calculated by the feature value calculator for each pixel of interest And a correction value setting unit that calculates and sets a correction value corresponding to the amount of change in the horizontal direction for each pixel of interest, and the interpolation value correction unit includes the pixel of interest set by the correction value setting unit. For each vertical change A first correction unit for correcting the interpolation value for each target pixel calculated by the first interpolation value calculation unit based on a correction value corresponding to the amount; and the first correction unit for each target pixel set by the correction value setting unit. A second correction unit that corrects the interpolation value for each pixel of interest calculated by the second interpolation value calculation unit based on a correction value corresponding to the amount of change in the horizontal direction. An average value of the interpolation value for each target pixel corrected by the first correction unit and the interpolation value for each target pixel corrected by the second correction unit is calculated for each target pixel, and each pixel of the target pixel is calculated. The value may be replaced.

  With this configuration, it is possible to remove image noise without performing noise detection regardless of whether the noise is on the edge in the interpolation process.

  The input image may be a captured data image captured through a single-plate image sensor.

  With this configuration, it is possible to remove noise generated in the captured data image captured by the image sensor based on the Bayer array regardless of whether the noise is on the edge or without detecting noise. .

  The input image may be a luminance image, and the feature image generation unit may generate an edge image from the input image based on a luminance difference.

  With this configuration, in the edge enhancement process, it is possible to remove image noise regardless of whether or not the noise is on the edge and without detecting noise.

  The correction value calculation unit may calculate the correction value by fuzzy inference using a predetermined membership function.

  With this configuration, since the correction value can be calculated regardless of the input image, the noise of the image is removed regardless of whether the noise is on the edge or without detecting the noise. be able to.

  In order to achieve the above object, according to a second aspect of the present invention, there is provided an image processing method for processing an input image in units of pixels, and a step of generating a predetermined feature image from the input image; Based on the feature image generated in the generating step, calculating a plurality of interpolation values for each pixel to interpolate pixel values; calculating a feature value for each pixel from the input image; and based on the feature value The interpolation value is calculated for each pixel based on the correction value for each pixel calculated in the step of calculating a correction value for correcting each of the plurality of interpolation values calculated for each pixel and the step of calculating the correction value. In the step of calculating the interpolation value calculated for each pixel calculated in the step of calculating the interpolation value for each pixel corrected in the step of correcting the interpolation value And Zui, image processing method and a step of correcting the characteristic image is provided.

  By using this method, it is possible to calculate a correction value based on the feature value calculated from the input image, and to correct the interpolation value for interpolating the pixel value for each pixel using the correction value. Therefore, by this method, it is possible to remove the noise of the image regardless of whether the noise is on the edge and without detecting the noise.

  In order to achieve the above object, according to a third aspect of the present invention, there is provided an imaging apparatus capable of correcting a captured image obtained by imaging on a pixel-by-pixel basis, An image data generation unit that generates a captured data image from an imaging result captured by the imaging unit, and an image processing unit that receives the captured data image and corrects the captured data image. A feature image generation unit that generates a predetermined feature image from the image; an interpolation value calculation unit that calculates a plurality of interpolation values for each pixel based on the feature image generated by the feature image generation unit; A correction value calculation unit that calculates a feature value for each pixel from the input image, calculates a correction value for correcting each of a plurality of interpolation values calculated for each pixel based on the feature value, and the correction value calculation In the department Based on the correction value calculated for each pixel, an interpolation value correction unit that corrects each of the interpolation values calculated for each pixel in the interpolation value calculation unit, and for each pixel corrected in the interpolation value correction unit An imaging device is provided that includes a corrected image generation unit that corrects the feature image based on a correction value.

  The imaging device includes, for example, an imaging unit, an image data generation unit, and an image processing unit. The imaging unit can perform imaging. The image data generation unit can generate an imaging data image corresponding to the imaging result in the imaging unit. The image processing unit can correct the captured data image generated by the image data generation unit. The image processing unit can include a feature image generation unit, an interpolation value calculation unit, a correction value calculation unit, an interpolation value correction unit, and a correction image generation unit. With this configuration, the noise of the image can be removed regardless of whether or not the noise is on the edge of the captured data image obtained by imaging, and without detecting the noise.

  The feature image generation unit receives the captured data image and generates a color difference image from the input image. The interpolation value calculation unit is adjacent to the target pixel for each target pixel for which an interpolation value is calculated. A first interpolation value calculation unit that calculates an average value of pixel values in vertical pixels, and a second interpolation value calculation unit that calculates an average value of pixel values in horizontal pixels adjacent to the target pixel. The correction value calculation unit calculates a vertical change amount and a horizontal change amount of the target pixel as a feature value, and the vertical value calculated by the feature value calculation unit for each target pixel. A correction value setting unit that calculates and sets a correction value corresponding to the amount of change in direction and a correction value corresponding to the amount of change in the horizontal direction for each pixel of interest, and the interpolation value correction unit includes the correction value Set by the value setting section A first correction unit that corrects the interpolation value for each target pixel calculated by the first interpolation value calculation unit based on a correction value corresponding to the amount of change in the vertical direction for each target pixel; and the correction value setting unit A second correction unit that corrects the interpolation value for each pixel of interest calculated by the second interpolation value calculation unit based on the correction value corresponding to the amount of change in the horizontal direction for each pixel of interest set by The corrected image generation unit calculates, for each target pixel, an average value of the interpolation value for each target pixel corrected by the first correction unit and the interpolated value for each target pixel corrected by the second correction unit. The pixel value of each pixel of interest may be calculated and replaced.

  With this configuration, it is possible to remove image noise without performing noise detection regardless of whether the noise is on the edge in the interpolation process.

  In addition, a luminance image may be input to the feature image generation unit, and an edge image may be generated from the luminance image based on a luminance difference.

  With this configuration, in the edge enhancement process, it is possible to remove image noise regardless of whether or not the noise is on the edge and without detecting noise.

  The correction value calculation unit may calculate the correction value by fuzzy inference using a predetermined membership function.

  With this configuration, since the correction value can be calculated regardless of the input image, the noise of the image is removed regardless of whether the noise is on the edge or without detecting the noise. be able to.

  In order to achieve the above object, according to a fourth aspect of the present invention, there is provided an imaging apparatus capable of correcting a captured image obtained by imaging on a pixel-by-pixel basis, An image data generation unit that generates a captured data image from an imaging result captured by the imaging unit, and an image processing unit that receives the captured data image and corrects the captured data image. An interpolation processing unit that interpolates pixel values, and the interpolation processing unit includes a feature image generation unit that generates a predetermined feature image from an image, and a pixel based on the feature image generated by the feature image generation unit. An interpolation value calculation unit that calculates a plurality of interpolation values for each pixel, and a plurality of interpolation values that are calculated for each pixel based on the feature values. Correct each A correction value calculation unit for calculating a correction value, and an interpolation value for correcting each of a plurality of interpolation values calculated for each pixel in the interpolation value calculation unit based on the correction value for each pixel calculated in the correction value calculation unit An imaging apparatus is provided that includes a correction unit and a corrected image generation unit that corrects the feature image based on a correction value for each pixel corrected by the interpolation value correction unit.

  The imaging device includes, for example, an imaging unit, an image data generation unit, and an image processing unit. The imaging unit can perform imaging. The image data generation unit can generate an imaging data image corresponding to the imaging result in the imaging unit. The image processing unit can correct the captured data image generated by the image data generation unit. The image processing unit can include an interpolation processing unit that performs interpolation of at least pixel values. Here, the interpolation processing unit can include a feature image generation unit, an interpolation value calculation unit, a correction value calculation unit, an interpolation value correction unit, and a correction image generation unit. With this configuration, the noise of the image can be removed regardless of whether or not the noise is on the edge of the captured data image obtained by imaging, and without detecting the noise.

  The imaging unit may include a single plate type imaging device.

  With this configuration, the structure of the imaging device can be simplified.

  In order to achieve the above object, according to a fifth aspect of the present invention, there is provided an imaging apparatus capable of correcting a captured image obtained by imaging on a pixel-by-pixel basis, An image data generation unit that generates a captured data image from an imaging result captured by the imaging unit, and an image processing unit that receives the captured data image and corrects the captured data image. An edge enhancement processing unit that enhances the edge of the image extracted based on the luminance difference of the image, the edge enhancement processing unit including a feature image generation unit that generates a predetermined feature image from the image, and the feature image Based on the feature image generated by the generation unit, an interpolation value calculation unit that calculates a plurality of interpolation values for each pixel to interpolate pixel values, calculates a feature value for each pixel from the input image, and based on the feature value ,the above A correction value calculation unit that calculates a correction value for correcting each of a plurality of interpolation values calculated for each element, and a pixel in the interpolation value calculation unit based on the correction value for each pixel calculated in the correction value calculation unit An imaging apparatus comprising: an interpolation value correction unit that corrects each of the plurality of interpolation values calculated for each; and a correction image generation unit that corrects the feature image based on the correction value for each pixel corrected by the interpolation value correction unit. An apparatus is provided.

  The imaging device includes, for example, an imaging unit, an image data generation unit, and an image processing unit. In addition, the image processing unit includes an edge enhancement processing unit that enhances the edge of the image extracted based on at least the luminance difference between the images. Here, the edge enhancement processing unit can include a feature image generation unit, an interpolation value calculation unit, a correction value calculation unit, an interpolation value correction unit, and a correction image generation unit. With this configuration, it is possible to remove image noise regardless of whether or not the noise is on the edge and without detecting noise.

  According to the present invention, it is possible to remove image noise without detecting noise.

  Exemplary embodiments of the present invention will be described below in detail with reference to the accompanying drawings. In the present specification and drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

  In the following description, an imaging data image that can be obtained by an imaging device using an imaging element is described as an imaging data image (an image represented by RAW image data) obtained by using a single plate type. Here, the single-plate type means that each CCD element is applied with a single color filter of any of the three primary colors of red, blue, and green arranged in a plane according to the order defined in the Bayer array. In this method, only the intensity of color is detected, and color information is obtained in combination with signals from surrounding CCD elements. Needless to say, the embodiment of the present invention is not limited to the above.

  The Bayer array is an array in which three primary colors of red, blue, and green are arranged in a lattice pattern, and one half is green, one quarter is red, and one quarter is blue. Here, in the Bayer array, green is arranged more than red and blue because green contains a lot of luminance components and human eyes are highly sensitive to luminance.

(Flow of image processing in imaging device)
Before describing an image processing apparatus according to an embodiment of the present invention, first, an outline of a flow of image processing performed to obtain a recorded image in an imaging apparatus such as a digital camera will be described. FIG. 1 is an explanatory diagram illustrating an example of the flow of image processing in the imaging apparatus.

  Examples of image processing in the imaging apparatus include a white balance correction process 10, an interpolation process 20, a color tone correction process 30, a gamma correction process 40, a YCbCr conversion process 50, an edge enhancement process 60, and a Jpeg coding process 70.

  The white balance correction processing 10 applies, for example, a gain set in advance for each color of RGB (Red Green Blue) to RAW image data (corresponding to a captured data image before image processing), and each pixel ( pixel) is amplified. For example, the interpolation process 20 generates RGB of all pixels from the Bayer array. For example, the color tone correction process 30 corrects the color tone of the image. The gamma correction process 40 performs, for example, nonlinear conversion of RGB signals to ensure visual linearity. The YCbCr conversion process 50 converts RGB into YCbCr based on a predetermined conversion formula, for example. Here, Y represents luminance, Cb represents color difference (chrominance), and Cr represents color difference (chrominance). For example, the edge enhancement processing 60 detects an edge portion from an image and enhances the lightness and darkness of the image by increasing the luminance of the detected edge portion. The Jpeg coding process 70 converts an image into an image file in Jpeg (Joint Photographic Experts Group) format.

  Through the above processing, the imaging apparatus corrects the RAW image data generated by imaging using an imaging element such as a single-plate CCD, and holds the image captured as a recorded image in Jpeg format. Can do. Note that the flow of image processing shown in FIG. 1 is an example, and the imaging apparatus according to the embodiment of the present invention is not limited to the flow of image processing shown in FIG.

(First Embodiment of Image Processing Device)
Next, an image processing apparatus according to the first embodiment of the present invention will be described. FIG. 2 is an explanatory diagram showing the image processing apparatus 100 according to the first embodiment of the present invention. Here, the image processing apparatus 100 is an image processing apparatus that performs the interpolation process 20 shown in FIG.

  In the following description, the pixel to be interpolated is referred to as a “target pixel”, and among the pixels indicating RGB as the target pixel, “R (Red) pixel” or “B (Blue) pixel” will be described. This is because, for example, in the Bayer array, “G (Green) pixels” are arranged twice as many as “R pixels” and “B pixels”. This is because various processes such as calculation (described later) and feature value calculation (described later) can be easily performed. Therefore, a pixel adjacent to the target pixel is a “G pixel”. Needless to say, the target pixel according to the embodiment of the present invention is not limited to the “R pixel” or the “B pixel”, and may be, for example, a “G pixel”.

  Referring to FIG. 2, the image processing apparatus 100 according to the first embodiment of the present invention includes a feature image generation unit 102, an interpolation value calculation unit 104, a correction value calculation unit 106, an interpolation value correction unit 108, A correction image generation unit 110.

  The feature image generation unit 102 generates a color difference image obtained by removing the luminance signal from the input image. Here, the generation of the color difference image in the feature image generation unit 102 can be performed by calculating a color difference signal for each pixel. The calculation of the color difference signal according to the embodiment of the present invention will be described below.

[Calculation of Color Difference Signal According to First Embodiment of the Present Invention]
FIG. 3 is an explanatory diagram for explaining the calculation of the color difference signal according to the first embodiment of the present invention. Here, the explanatory diagram shown in FIG. 3 shows a Bayer array, and each symbol from a to p represents a pixel value of a pixel. In particular, a pixel having a pixel value e indicates a target pixel to be interpolated. In addition, pixels having pixel values b, d, h, and f, which are adjacent to the pixel having the pixel value e, are G pixels.

  The calculation of the color difference signal according to the first embodiment of the present invention can be performed using Equations 1 to 4. Here, B calculated by Equation 1 indicates a color difference signal in a pixel having a pixel value b. Similarly, D calculated by Equation 2 is a color difference signal in a pixel having a pixel value d, H calculated by Equation 3 is a color difference signal in a pixel having a pixel value h, and F calculated by Equation 4 is The color difference signals in the pixels having the pixel value f are respectively shown.

B = −b + (e + k) / 2 (Formula 1)
D = −d + (e + p) / 2 (Expression 2)
H = −h + (e + n) / 2 (Formula 3)
F = −b + (e + m) / 2 (Formula 4)

  Here, Equations 1 to 4 are obtained by subtracting “pixel value of G pixel” from “pixel value of R pixel” or “pixel value of B pixel” from “pixel value of G pixel”. This corresponds to subtracting the “pixel value”.

  The feature image generation unit 102 calculates the color difference signal for all pixels for which the color difference signal can be calculated by changing the target pixel. Note that the calculation of the color difference signal is not limited to all pixels for which the color difference signal can be calculated. For example, only the R pixel, only the B pixel, or one skipped pixel is used. It can be carried out. Further, the target pixel is not limited to change so that the pixels for which the color difference signals are calculated do not overlap, and for example, the pixels for which the color difference signals are calculated can be overwritten and updated as appropriate.

  Further, the color difference signal calculated by the feature image generation unit 102 may be held in a storage unit included in the feature image generation unit 102, for example. Here, as the storage means that the feature image generation unit 102 has, for example, a magnetic recording medium such as a hard disk or a magnetic tape, a flash memory, a MRAM (Magnetoresistive Random Access Memory), a FeRAM (Ferroelectric) Of course, non-volatile memories such as Random Access Memory), PRAM (Phase change Random Access Memory), magneto-optical disks, etc. are not limited to the above. The holding of the color difference signal is not limited to the above. For example, the image processing apparatus 100 includes a storage unit (not shown), and the storage unit holds the color difference signal calculated by the feature image generation unit 102. You can also.

  The interpolation value calculation unit 104 uses the color difference image generated by the feature image generation unit 102, that is, the color difference signal at each pixel calculated by changing the target pixel, to calculate an interpolation value for interpolating the pixel value. Multiple calculations are made for each.

  Here, the interpolation value calculation unit 104 passes, for example, only a color difference signal having a frequency equal to or lower than the cutoff frequency and attenuates a color difference signal having a frequency higher than the cutoff frequency, or a specific frequency. A filter such as a band-pass filter (Band-Pass Filter) that passes only the color difference signals in the band and attenuates the color difference signals in the other bands can be provided. The filter can be a digital filter, for example.

  For example, in the image processing apparatus 100 according to the first embodiment of the present invention, the interpolation value calculation unit 104 includes a first interpolation value calculation unit 112 and a second interpolation value calculation unit 114, and provides two interpolation values. calculate. Hereinafter, calculation of the interpolation value according to the first embodiment of the present invention will be described.

[Calculation of Interpolated Value According to First Embodiment of the Present Invention]
FIG. 4 is an explanatory diagram illustrating calculation of an interpolation value according to the first embodiment of the present invention. Here, FIG. 4A is an explanatory diagram for explaining the calculation of the interpolation value in the first interpolation value calculation unit 112, and FIG. 4B is the diagram of the interpolation value in the second interpolation value calculation unit 114. It is explanatory drawing explaining calculation.

  Calculation of the interpolation value according to the first embodiment of the present invention can be performed using Equation 5 and Equation 6 using the color difference signal of the pixel adjacent to the target pixel calculated by the feature image generation unit 102.

VV = (B + H) / 2 (Formula 5)
VH = (D + F) / 2 (Formula 6)

  Here, Formula 5 indicates the interpolation value VV calculated by the first interpolation value calculation unit 112. As shown in FIG. 4A, the interpolation value VV is the average value of the color difference signals of pixels adjacent in the vertical direction to the target pixel using the color difference signal calculated for each target pixel by the feature image generation unit 102. Can be derived by calculating.

  Equation 6 represents the interpolation value VH calculated by the second interpolation value calculation unit 114. As shown in FIG. 4B, the interpolation value VH is the average value of the color difference signals of pixels adjacent in the horizontal direction to the target pixel using the color difference signal calculated by the feature image generation unit 102 for each target pixel. Can be derived by calculating.

  Equations 5 and 6 represent calculation of an arithmetic average of color difference signals of pixels adjacent in the vertical direction or horizontal direction with respect to the pixel of interest, but according to the first embodiment of the present invention. The calculation of the interpolation value is not limited to the above, and can be calculated using, for example, a geometric average or a weighted average.

  Further, the “vertical direction” and the “horizontal direction” according to the embodiment of the present invention are not limited to the directions shown in FIG. 4A and FIG. It goes without saying that the “horizontal direction” can be determined with the “vertical direction” defined as the reference.

  The interpolation value calculation unit 104 performs the calculation shown in Equation 5 and Equation 6 for each pixel of interest, and calculates the interpolation value VV and the interpolation value VH for each pixel of interest.

  In addition, the interpolation value VV and the interpolation value VH for each pixel of interest calculated by the interpolation value calculation unit 104 can be held in a storage unit included in the interpolation value calculation unit 104, for example. Here, examples of the storage means included in the interpolation value calculation unit 104 include a magnetic recording medium such as a hard disk, a non-volatile memory such as a flash memory, and a magneto-optical disk, but are not limited thereto. Needless to say. The holding of the interpolation value VV and the interpolation value VH for each pixel of interest is not limited to the above. For example, the image processing apparatus 100 includes a storage unit (not shown), and the interpolation value calculation unit 104 calculates the storage unit. It is also possible to hold the interpolation value VV and the interpolation value VH for each target pixel.

  The correction value calculation unit 106 includes a feature value calculation unit 116 and a correction value setting unit 118. The feature value calculation unit 116 calculates a feature value for each target pixel from the input image. The correction value setting unit 118 also uses the interpolation value VV for each pixel of interest calculated by the first interpolation value calculation unit 112 and the second interpolation value calculation unit 114 based on the feature values calculated by the feature value calculation unit 116. A correction value for correcting the interpolated value VH for each pixel of interest calculated in is calculated and set. Hereinafter, calculation of the feature value according to the first embodiment of the present invention and calculation of the correction value according to the first embodiment of the present invention will be described.

[Calculation of feature values according to the first embodiment of the present invention]
First, calculation of feature values according to the first embodiment of the present invention will be described. FIG. 5 is an explanatory diagram illustrating calculation of feature values according to the first embodiment of the present invention. Here, FIG. 5A is an explanatory diagram for explaining the calculation of the first feature value in the feature value calculation unit 116, and FIG. 5B is the second feature in the feature value calculation unit 116. It is explanatory drawing explaining calculation of a value.

  The calculation of the feature value according to the first embodiment of the present invention can be performed by Expression 7 and Expression 8 using the pixel value of each pixel.

M1x = | b−h | (Formula 7)
M2x = | d−f | (Formula 8)

  Here, Equation 7 represents the first feature value M1x calculated by the feature value calculation unit 116. As shown in FIG. 5A, the first feature value M1x can be derived by calculating a change amount (Vertical Gradient) of a pixel value of a pixel adjacent to the target pixel in the vertical direction.

  Equation 8 represents the second feature value M2x calculated by the feature value calculation unit 116. As shown in FIG. 5B, the second feature value M2x can be derived by calculating the amount of change (Horizontal Gradient) of the pixel value of the pixel adjacent to the target pixel in the horizontal direction.

  Note that Equation 7 and Equation 8 represent calculation of the amount of change in the pixel value of a pixel adjacent in the vertical direction or horizontal direction with respect to the target pixel, but the feature according to the first embodiment of the present invention. It goes without saying that the calculation of values is not limited to the above.

[Calculation of Correction Value According to First Embodiment of the Present Invention]
Next, calculation of a correction value according to the first embodiment of the present invention will be described. Calculation of the correction value according to the first embodiment of the present invention can be performed using fuzzy inference. More precisely, the correction value according to the first embodiment of the present invention is calculated by defuzzify, in which the inference result is replaced with a numerical value using a predetermined membership function. The Here, the membership function is a function representing the degree to which elements of a fuzzy set belong to the set. The value of the membership function in the vertical axis direction is called a grade. A fuzzy set is a set that allows an intermediate state between when an element belongs to a set and when an element does not belong.

  A specific example of calculating the correction value according to the first embodiment of the present invention will be shown below. Note that the membership function relating to the calculation of the correction value according to the first embodiment of the present invention is an arbitrarily settable function, and the membership function used in the following description is the first embodiment of the present invention. The calculation of the correction value according to is not limited.

[First Calculation Example of Correction Value According to First Embodiment of the Present Invention]
FIG. 6 is an explanatory diagram showing a first example using the membership function according to the first embodiment of the present invention. Here, the first example shown in FIG. 6 shows a case where the direction of the image has a feature in the vertical direction, for example, when the image represents a vertical line.

  Referring to FIG. 6, the correction value setting unit 118 performs FIG. 6 () for each of the first feature value M1 related to the vertical change amount and the second feature value M2 related to the horizontal change amount. Four rules of a) to FIG. 6D are set.

Here, the relationship between the membership functions in FIGS. 6A to 6D is as follows.
(1) Membership function in FIG. 6 (a): When both vertical variation (Vertical Gradient) and horizontal variation (Horizontal Gradient) are small (2) Membership function in FIG. 6 (b): When the vertical variation (Vertical Gradient) is small and the horizontal variation (Horizontal Gradient) is large (3) Membership function in Fig. 6 (c): Horizontal variation (Horizontal Gradient) is small and vertical When the amount of change in direction (Vertical Gradient) is large (4) Membership function in Fig. 6 (d): When the amount of change in vertical direction (Vertical Gradient) and the amount of change in horizontal direction (Horizontal Gradient) are both large 6 (a) will be described as pattern 0, FIG. 6 (b) as pattern 1, FIG. 6 (c) as pattern 2, and FIG. 6 (d) as pattern 3.

  Also, KH_v, KL_v, KH_h, and KL_h set in the membership functions of FIGS. 6A to 6D are the same values in FIGS. 6A to 6D. Here, KH_v, KL_v, KH_h, and KL_h can be arbitrarily set. In the correction value setting unit 118 according to the first embodiment of the present invention, the membership function can be arbitrarily set by arbitrarily setting the KH_v, KL_v, KH_h, and KL_h. Further, the correction value setting unit 118 can set the weighting coefficient V_Rn for the vertical direction and the weighting coefficient H_Rn for the horizontal direction in each of the patterns 0 to 3. In addition, the weighting coefficient V_Rn and the weighting coefficient H_Rn set in each of the patterns 0 to 3 shown below are examples, and the weighting coefficient V_Rn and the weighting coefficient H_Rn according to the embodiment of the present invention are not limited. Needless to say.

<Pattern 0>
Referring to FIG. 6A, in the pattern 0, the grade value M1 in the first feature value M1x is M1 = 1, and the grade value M2 in the second feature value M2x is M2 = 0. Therefore, the minimum grade value Rn (where n is a non-negative integer, the same applies hereinafter) in pattern 0 is R0 = 0.

  In addition, since pattern 0 is a case where both the vertical change amount and the horizontal change amount are small, it is desirable that the weighting coefficient V_R0 for the vertical direction and the weighting coefficient H_R0 for the horizontal direction are set to be the same. .

<Pattern 1>
Referring to FIG. 6B, in pattern 1, the grade value M1 in the first feature value M1x is M1 = 1, and the grade value M2 in the second feature value M2x is also M2 = 1. Therefore, the minimum grade value Rn in the pattern 1 is R1 = 1.

  Also, since the pattern 1 has a small amount of change in the vertical direction and a large amount of change in the horizontal direction, the weighting coefficient V_R1 for the vertical direction and the weighting coefficient H_R1 for the horizontal direction are larger in the vertical direction than in the horizontal direction. It is desirable to set.

<Pattern 2>
Referring to FIG. 6C, in pattern 2, the grade value M1 for the first feature value M1x is M1 = 0, and the grade value M2 for the second feature value M2x is M2 = 1. Therefore, the minimum grade value Rn in the pattern 2 is R2 = 0.

Also, since pattern 2 has a small amount of change in the horizontal direction and a large amount of change in the vertical direction, the weighting coefficient V_R2 for the vertical direction and the weighting coefficient H_R2 for the horizontal direction are larger in the horizontal direction than in the vertical direction. It is desirable to set.
<Pattern 3>
Referring to FIG. 6D, in pattern 3, the grade value M1 in the first feature value M1x is M1 = 0, and the grade value M2 in the second feature value M2x is M2 = 1. Therefore, the minimum grade value Rn in the pattern 3 is R3 = 0.

  Further, since the pattern 3 is a case where both the amount of change in the vertical direction and the amount of change in the horizontal direction are large, it is desirable that the weighting coefficient V_R3 for the vertical direction and the weighting coefficient H_R3 for the horizontal direction are set to be the same. .

  FIG. 7 is an explanatory diagram showing an example of values set for calculating a correction value according to the embodiment of the present invention in the first example shown in FIG. Here, in FIG. 7, the sum of the weighting coefficient V_Rn for the vertical direction and the weighting coefficient H_Rn for the horizontal direction is set to be 16. This is because, for example, the correction value calculation process in the correction value setting unit 118 is facilitated by expressing it in bits. Therefore, in FIG. 7, 16 is set to represent 4 bits, but the setting of the weighting coefficient V_Rn for the vertical direction and the weighting coefficient H_Rn for the horizontal direction according to the embodiment of the present invention is limited to the above. It goes without saying that there is nothing.

  Furthermore, the correction value setting unit 118 uses the values (R0 to R3, V_R0 to V_R3, and H_R0 to H_R3) set using the membership function shown in FIG. A correction value Wgt_v for correcting the calculated interpolation value VV can be calculated by Expression 9.

  From Equation 9, the correction value Wgt_v for correcting the interpolation value VV in the first calculation example is Wgt_v = 16 as shown in Equation 10.

  Further, the correction value setting unit 118 can calculate the correction value Wgt_h for correcting the interpolation value VH calculated by the second interpolation value calculation unit 114 by using Equation 11.

  From Equation 11, the correction value Wgt_h for correcting the interpolation value VH in the first calculation example is Wgt_h = 0 as shown in Equation 12.

  In the first example shown in FIG. 6, the correction value setting unit 118 can calculate the correction value Wgt_v shown in Equation 10 and the correction value Wgt_h shown in Equation 12. Next, a second calculation example of the correction value according to the first embodiment of the present invention will be described.

[Second Example of Calculation of Correction Value According to First Embodiment of the Present Invention]
FIG. 8 is an explanatory diagram showing a second example using the membership function according to the first embodiment of the present invention. Here, the second example shown in FIG. 8 shows a case where the direction of the image is unclear.

  Referring to FIG. 8, as in the first example shown in FIG. 6, in the correction value setting unit 118, the first feature value M1x related to the vertical change amount and the second feature value related to the horizontal change amount. Four rules of FIG. 8A to FIG. 8D are set for each feature value M2x.

Here, the relationship between the membership functions in FIGS. 8A to 8D is as follows.
(1) Membership function in FIG. 8 (a): When both vertical variation (Vertical Gradient) and horizontal variation (Horizontal Gradient) are small (2) Membership function in FIG. 8 (b): When the vertical variation (Vertical Gradient) is small and the horizontal variation (Horizontal Gradient) is large (3) Membership function in FIG. 8 (c): Horizontal variation (Horizontal Gradient) is small and vertical When the change in direction (Vertical Gradient) is large (4) Membership function in Fig. 8 (d): When the change in vertical direction (Vertical Gradient) and the change in horizontal direction (Horizontal Gradient) are both large As in the first example shown in FIG. 6, FIG. 8 (a) is pattern 0, FIG. 8 (b) is pattern 1, FIG. 8 (c) is pattern 2, and FIG. 8 (d) is pattern 3. To do.

<Pattern 0>
Referring to FIG. 8A, in the pattern 0, the grade value M1 in the first feature value M1x is M1 = 0.75, and the grade value M2 in the second feature value M2x is M2 = 0.25. Therefore, the minimum grade value Rn in the pattern 0 is R0 = 0.25.

  Similarly to the first example shown in FIG. 6, the pattern 0 is a case where both the amount of change in the vertical direction and the amount of change in the horizontal direction are small. Therefore, the weighting coefficient V_R0 for the vertical direction and the weighting for the horizontal direction are used. The coefficients H_R0 are preferably set to be the same.

<Pattern 1>
Referring to FIG. 8B, in pattern 1, the grade value M1 for the first feature value M1x is M1 = 0.75, and the grade value M2 for the second feature value M2x is also M2 = 0.75. Therefore, the minimum grade value Rn in pattern 1 is R1 = 0.75.

  Similarly to the first example shown in FIG. 6, the pattern 1 is a case in which the amount of change in the vertical direction is small and the amount of change in the horizontal direction is large. Therefore, the weighting coefficient V_R1 for the vertical direction and the weighting for the horizontal direction The coefficient H_R1 is desirably set larger in the vertical direction than in the horizontal direction.

<Pattern 2>
Referring to FIG. 8C, in pattern 2, the grade value M1 in the first feature value M1x is M1 = 0.25, and the grade value M2 in the second feature value M2x is M2 = 0.75. Accordingly, the minimum grade value Rn in pattern 2 is R2 = 0.25.

Similarly to the first example shown in FIG. 6, the pattern 2 is a case where the amount of change in the horizontal direction is small and the amount of change in the vertical direction is large. Therefore, the weighting coefficient V_R2 for the vertical direction and the weighting for the horizontal direction The coefficient H_R2 is desirably set larger in the horizontal direction than in the vertical direction.
<Pattern 3>
Referring to FIG. 8D, in pattern 3, the grade value M1 of the first feature value M1x is M1 = 0.25, and the grade value M2 of the second feature value M2x is M2 = 0.75. Therefore, the minimum grade value Rn in the pattern 3 is R3 = 0.25.

  Similarly to the first example shown in FIG. 6, the pattern 3 is a case where both the vertical change amount and the horizontal change amount are large. Therefore, the weighting coefficient V_R3 for the vertical direction and the weighting for the horizontal direction are used. The coefficients H_R3 are preferably set to be the same.

  FIG. 9 is an explanatory diagram showing an example of values set for calculating the correction value according to the embodiment of the present invention in the second example shown in FIG. Here, in FIG. 9, the sum of the weighting coefficient V_Rn for the vertical direction and the weighting coefficient H_Rn for the horizontal direction is set to 16 as in the first example shown in FIG.

  Further, similarly to the first example shown in FIG. 6, the correction value setting unit 118 sets values (R0 to R3, V_R0 to V_R3, and H_R0 to H_R3), for example, using the membership function shown in FIG. The correction value Wgt_v for correcting the interpolation value VV and the correction value Wgt_h for correcting the interpolation value VH can be calculated by Expressions 9 and 11.

  From Equation 9, the correction value Wgt_v for correcting the interpolation value VV in the second calculation example is Wgt_v = 11 as shown in Equation 13.

  Further, from Equation 11, the correction value Wgt_h for correcting the interpolation value VH in the second calculation example is Wgt_h = 5 as shown in Equation 14.

  Here, in Formula 13 and Formula 14, the correction value is converted to an integer by rounding off. However, the calculation of the correction value according to the embodiment of the present invention is not limited to the above, and may be, for example, rounded down after the decimal point.

  In the second example shown in FIG. 8, the correction value setting unit 118 can calculate the correction value Wgt_v shown in Equation 13 and the correction value Wgt_h shown in Equation 14.

  As described above, the correction value setting unit 118 can calculate the correction values for the two directions of the vertical direction and the horizontal direction in each pixel of interest using fuzzy inference. By using fuzzy inference, the correction value setting unit 118 can calculate optimum correction values for the two directions of the vertical direction and the horizontal direction for each pixel of interest, regardless of the input image.

  The correction value setting unit 118 derives two correction values using four rules when applying fuzzy inference, as shown in the first example and the second example described above. However, the correction value setting unit according to the embodiment of the present invention is not limited to the above. For example, the number of correction values calculated by a fuzzy rule, a membership function, and the membership function can be increased.

  Further, the correction value setting unit 118 can store the correction values for the two directions of the vertical direction and the horizontal direction calculated for each pixel of interest in, for example, a storage unit included in the correction value setting unit 118. Here, examples of the storage means included in the correction value setting unit 118 include a magnetic recording medium such as a hard disk, a non-volatile memory such as a flash memory, and a magneto-optical disk, but are not limited thereto. Needless to say. The holding of the correction value is not limited to the above. For example, the image processing apparatus 100 includes a storage unit (not shown), and the correction value calculated and set by the correction value setting unit 118 is held in the storage unit. It can also be made.

  As described above, the correction value calculation unit 106 includes the feature value calculation unit 116 and the correction value setting unit 118 to calculate a feature value for each pixel from the input image, and based on the feature value, the target pixel It is possible to calculate and set a correction value Wgt_v for correcting the interpolation value VV and a correction value Wgt_h for correcting the interpolation value VH.

  The interpolation value correction unit 108 includes a first correction unit 120 and a second correction unit 122. The first correction unit 120 corrects the interpolation value VV calculated by the first interpolation value calculation unit 112 for each target pixel based on the correction value Wgt_v set by the correction value calculation unit 106. Here, in the first correction unit 120, for example, the interpolation value VV can be corrected by multiplying the interpolation value VV by the correction value Wgt_v (at this time, the correction value Wgt_v is a coefficient). The means for correcting the interpolation value VV in the first correction unit 120 is not limited to the above, and for example, the correction value Wgt_v may be added to or subtracted from the interpolation value VV.

  In addition, the second correction unit 122 corrects the interpolation value VH calculated by the second interpolation value calculation unit 114 for each target pixel based on the correction value Wgt_h set by the correction value calculation unit 106. Here, in the second correction unit 122, similarly to the first correction unit 120, for example, the interpolation value VH is multiplied by the correction value Wgt_h (at this time, the correction value Wgt_h becomes a coefficient), thereby the interpolation value VH. Can be corrected. Needless to say, the means for correcting the interpolation value VH in the second correction unit 122 is not limited to the above.

  The interpolation value VV and the interpolation value VH are corrected by the interpolation value correction unit 108, for example, when the interpolation value correction unit 108 appropriately reads the interpolation value VV and the interpolation value VH from the storage unit included in the interpolation value calculation unit 104. The interpolation value correction unit 108 can read out the correction value Wgt_v and the correction value Wgt_h from the storage unit included in the correction value calculation unit 106 as appropriate.

  Note that the means for reading out the interpolation value VV and the interpolation value VH, the correction value Wgt_v, and the correction value Wgt_h in the embodiment of the present invention is not limited to the above. For example, the interpolation value correction unit 108 includes a storage unit, and the interpolation value calculation unit 104 and the correction value calculation unit 106 respectively store the interpolation value VV and the interpolation value VH for each pixel of interest, the correction value Wgt_v, and the correction value Wgt_h. It can also be stored in a storage means included in the interpolation value correction unit 108. In the above case, the interpolation value correction unit 108 reads out the interpolation value VV and the interpolation value VH for each pixel of interest, the correction value Wgt_v and the correction value Wgt_h from the storage means included in the interpolation value correction unit 108, and performs interpolation for each pixel of interest. The value VV and the interpolation value VH can be corrected.

  Here, examples of the storage means included in the interpolation value correction unit 108 include a magnetic recording medium such as a hard disk, a non-volatile memory such as a flash memory, a magneto-optical disk, and the like. Needless to say.

  In addition, for example, when the image processing apparatus 100 includes a storage unit (not shown), the interpolation value correction unit 108 receives each pixel of interest from the storage unit (not shown) included in the image processing apparatus 100. It is also possible to read out the interpolation value VV and the interpolation value VH, the correction value Wgt_v and the correction value Wgt_h, and correct the interpolation value VV and the interpolation value VH for each pixel of interest.

  The corrected image generation unit 110 corrects the input image for each target pixel based on the interpolation value VV ′ and the interpolation value VH ′ corrected by the interpolation value correction unit 108. The corrected pixel value Pout calculated for each target pixel in the corrected image generation unit 110 can be expressed by Equation 15, for example. Here, E in Equation 15 represents the pixel value of the target pixel. Further, “16” set in the denominator of the second term of Expression 15 is a value set for easy expression in bits, and can be arbitrarily set.

  As shown in Expression 15, the corrected image generation unit 110 calculates the corrected interpolation value VV ′ (that is, “Wgt_v × VV”) and the interpolation value VH ′ (that is, “Wgt_h × VH” from the pixel value of the target pixel. The pixel value is replaced for each target pixel by subtracting the average value of “.)”. Therefore, the image output from the corrected image generation unit 110 is a corrected image obtained by correcting the input image.

  In Expression 15, the corrected pixel value Pout is calculated using the weighted average value related to the two interpolation values of the corrected interpolation value VV ′ and the interpolation value VH ′. However, the corrected pixel value according to the embodiment of the present invention is calculated. The calculation method of the pixel value Pout is not limited to the above. For example, the corrected pixel value Pout can be derived from an interpolation value having a weighted average value of 3 or more. When the weighted average value is derived from three or more interpolation values, for example, the number of fuzzy rules, membership functions, and correction values calculated by the membership functions may be increased. Needless to say, the method of calculating the corrected pixel value Pout according to the embodiment of the present invention is not limited to Equation 15.

  As described above, the image processing apparatus 100 according to the first embodiment of the present invention generates a color difference image from an input image, and obtains a plurality of interpolation values in a plurality of directions (vertical direction and horizontal direction) for each target pixel. calculate. Further, the image processing apparatus 100 calculates a plurality of feature values in a plurality of directions (vertical direction and horizontal direction) for each pixel of interest, and performs a plurality of interpolations for each pixel of interest based on the feature values and fuzzy inference. Set a correction value to correct the value. Then, the image processing apparatus 100 corrects the interpolation value for each pixel of interest using the set correction value, and outputs a corrected image in which the input image is corrected.

  Further, the image processing apparatus 100 according to the first embodiment of the present invention can set correction values for correcting a plurality of interpolation values for each pixel of interest using fuzzy inference. First, an appropriate correction value can be set for each target pixel. Therefore, the image processing apparatus 100 according to the embodiment of the present invention replaces the pixel value for each pixel of interest without performing noise detection even if the input image is an image having noise on the edge, Image noise can be removed.

  In the first embodiment of the present invention, the image processing apparatus 100 has been described. However, the embodiment of the present invention is not limited to such an embodiment, and for example, an imaging apparatus such as a digital camera or a digital camera function. The present invention can be applied to a communication device having an imaging function such as an attached mobile phone. The application to the imaging device will be described later.

(Second Embodiment of Image Processing Device)
As the image processing apparatus according to the embodiment of the present invention, the image processing apparatus 100 according to the first embodiment has been described with respect to the image processing apparatus that performs the interpolation process 20 illustrated in FIG. However, processing that can be performed by the image processing apparatus according to the embodiment of the present invention is not limited to interpolation processing. Therefore, an image processing apparatus according to the second embodiment of the present invention will be described next.

  FIG. 10 is a block diagram showing an outline of processing in the image processing apparatus according to the second embodiment of the present invention. Here, the image processing device 200 is an image processing device that performs the edge enhancement processing 60 shown in FIG.

  Referring to FIG. 10, in the image processing apparatus 200, a noise reduction / edge extraction process 202, an edge enhancement gain setting process 204, a correction process 206, and a corrected image generation process 208 are performed. The image (input image) input to the image processing apparatus 200 is, for example, a luminance image output from the YCbCr conversion process 50 in FIG.

  The noise reduction / edge extraction process 202 is a process corresponding to the process performed in the feature image generation unit 102 and the interpolation value calculation unit 104 shown in FIG. 2, and extracts an edge for each target pixel from the input luminance image. Calculate the interpolation value. Here, the edge extraction in the noise reduction / edge extraction processing 202 can be performed by, for example, a Laplacian filter.

  The edge enhancement gain setting process 204 is a process corresponding to the process performed in the correction value calculation unit 106 illustrated in FIG. 2, and attention is paid from the input luminance image as in the image processing apparatus 100 according to the first embodiment. A feature value is calculated for each pixel, and a correction value for correcting the interpolation value for each target pixel is set using fuzzy inference.

  The correction process 206 is a process corresponding to the process performed in the interpolation value correction unit 108 illustrated in FIG. 2, and corrects the interpolation value for each target pixel as in the image processing apparatus 100 according to the first embodiment.

  The corrected image generation process 208 is a process corresponding to the process performed in the corrected image generation unit 110 illustrated in FIG. 2, and the pixel value is replaced for each target pixel as in the image processing apparatus 100 according to the first embodiment. . Therefore, the corrected image generation process 208 outputs a corrected image obtained by correcting the input luminance image.

  As described above, the image processing apparatus 200 according to the second embodiment of the present invention can set correction values for correcting a plurality of interpolation values for each pixel of interest using fuzzy inference. An optimum correction value can be set for each target pixel regardless of the image. Therefore, the image processing apparatus 200 according to the second embodiment of the present invention sets the pixel value for each pixel of interest without performing noise detection even if the input image is an image having noise on the edge. It can be replaced to remove image noise.

  In the second embodiment of the present invention, the image processing apparatus 200 has been described. However, the embodiment of the present invention is not limited to such an embodiment, and for example, an imaging apparatus such as a digital camera or a digital camera function. The present invention can be applied to a communication device having an imaging function such as an attached mobile phone.

(Program for image processing)
Even if the image to be processed is an image having noise on the edge, the program for causing the image processing apparatus shown in the first and second embodiments of the present invention to function as a computer detects noise. It is possible to remove the noise of the image by replacing the pixel value for each pixel of interest without performing it.

(Image processing method)
Next, an image processing method according to an embodiment of the present invention will be described. FIG. 11 is a flowchart showing an image processing method according to the embodiment of the present invention. In the following description, it is assumed that the image to be processed is an image obtained from an image sensor according to the order defined in the Bayer array. However, the image processing method according to the embodiment of the present invention is described above. Not limited.

  First, a target pixel is determined (S100). Here, the target pixel in step S100 can be, for example, a B or R pixel.

  An interpolation value for interpolating the target pixel is calculated, and the interpolation value is corrected based on the characteristics of the input image (image to be processed) (S102). Hereinafter, the process performed in step S102 will be described in more detail.

[Process 1 in Step S102]
A feature signal is generated (S104). Here, examples of the feature signal include a color difference signal and an edge signal.

  Using the feature signal generated in step S104, a plurality of interpolation values in a plurality of directions are calculated for the target pixel (S106). Here, the plurality of directions in step S106 include, for example, two directions of a vertical direction and a horizontal direction. However, the interpolation values calculated in step S106 are not limited to those calculated in the two directions of the vertical direction and the horizontal direction. For example, the interpolation values are calculated in a total of four directions including the vertical direction, the horizontal direction, and the tanned two directions. You can also.

[Process 2 in Step S102]
Feature values in a plurality of directions are calculated for the target pixel (S108). Here, the feature values in the plurality of directions include, for example, a change amount of the pixel value in the vertical direction and a change amount in the horizontal direction. Needless to say, the feature value calculated in step 108 is not limited to the amount of change in the pixel value in the vertical direction and the amount of change in the horizontal direction.

  Using the feature value calculated in step S108, a correction value for correcting each of the plurality of interpolation values in the plurality of directions calculated in step S106 is set (S110). Here, the correction value can be set using, for example, fuzzy inference. More specifically, in step S110, for example, the correction value can be calculated using the feature value calculated in step S108, a plurality of membership functions set in advance, and equations 9 and 11.

[Process 3 in Step S102]
Each of the plurality of interpolation values is corrected based on the plurality of interpolation values calculated in step S106 and the correction values for correcting the plurality of interpolation values set in step S110 (S112). Here, the correction of the interpolation value in step S112 can be performed, for example, by multiplying the interpolation value calculated in step S106 by the correction value set in step S110, but is not limited thereto.

  As described above, in step S <b> 102, the interpolation value for interpolating the target pixel is calculated by performing the processes 1 to 3 described above, and the interpolation value is corrected based on the characteristics of the input image.

  It is determined whether or not the interpolation value has been corrected for all the interpolable pixels (S114). Here, the determination in step S114 may be performed, for example, based on whether or not the number of pixels to be interpolated and the number of pixels corresponding to the pixel to be interpolated are determined and the interpolation value is corrected by the number of pixels. The determination in step S114 is not limited to the above. For example, the determination can be made based on whether or not the interpolation value is corrected by the number of pixels included in the input image.

  If it is determined in step S114 that the interpolation value has not been corrected for all the interpolable pixels, the target pixel is changed (S116), and the processes in steps S102 to S114 are repeated.

  If it is determined in step S114 that the interpolation values have been corrected for all the interpolable pixels, the pixel values of all the pixels whose interpolation values have been corrected are replaced, and the input image is corrected. It performs (S118).

  As described above, the image processing method according to the embodiment of the present invention can set correction values for correcting a plurality of interpolation values for each pixel of interest using fuzzy inference. An optimal correction value can be set for each pixel of interest. Therefore, the image processing method according to the embodiment of the present invention replaces the pixel value for each pixel of interest without performing noise detection, even if the input image is an image having noise on the edge. Noise can be removed.

(Image pickup apparatus according to an embodiment of the present invention)
Next, an imaging apparatus to which the image processing apparatus according to the embodiment of the present invention shown in FIGS. 2 and 10 is applied will be described. FIG. 12 is a block diagram showing an imaging apparatus 300 according to the embodiment of the present invention.

  Referring to FIG. 12, the imaging apparatus 300 according to the embodiment of the present invention includes at least an imaging unit 302, an image data generation unit 304, and an image processing unit 306.

  The imaging unit 302 can be configured by, for example, a lens (Lens) and an imaging element such as a CCD, and performs imaging.

  The image data generation unit 304 generates a captured data image from the result captured by the imaging unit 302. Here, the image data generation unit 304 includes, for example, an AGC (Automatic Gain Control) circuit and an ADC (Analog to Digital Converter), and can convert an analog signal input from the imaging unit 302 into a digital signal. Further, the image data generation unit 304 can further include a TG (Timing Generator), and can also control the readout timing of the analog signal from the imaging unit 302. Here, the AGC circuit, ADC, and TG are collectively referred to as AFE (Analog Front End).

  The image processing unit 306 can perform image processing on the captured data image generated by the image data generation unit 304 to generate a recorded image. Here, as the image processing performed in the image processing unit 306, for example, the processing shown in FIG. Therefore, the imaging apparatus 300 can include the image processing apparatus 100 according to the first embodiment of the present invention and / or the image processing apparatus 200 according to the second embodiment in the image processing unit 306.

  Therefore, in the image processing unit 306, the image processing apparatus 100 according to the first embodiment of the present invention is applied to the portion that performs the interpolation processing, so that the image to be processed is an image having noise on the edge. However, it is possible to remove image noise without performing noise detection. Further, in the image processing unit 306, the image processing apparatus 200 according to the second embodiment of the present invention is applied to the portion that performs the edge enhancement processing, so that the image to be processed is an image having noise on the edge. However, image noise can be removed without detecting noise.

  Further, the imaging apparatus 300 can further include a control unit 308, a storage unit 310, a display unit 312, and an operation unit 314.

  The control unit 308 is configured by, for example, an MPU (Micro Processing Unit), and can control processing of the entire imaging apparatus 300.

  The storage unit 310 can store the recorded image generated by the image processing unit 306. Here, examples of the storage unit 310 include a magnetic recording medium such as a hard disk, a non-volatile memory such as a flash memory, and a magneto-optical disk. Further, the storage unit 310 may be a unit that can be attached to and detached from the imaging apparatus 300, such as a memory card.

  The display unit 312 displays the recorded image generated by the image processing unit 306 or the image stored in the storage unit 310. An example of the display unit 312 is a TFT (Thin Film Transistor).

  The operation unit 314 is a device for transmitting operation commands for performing various processes to the control unit 308. Examples of the operation unit 314 include a push button switch and a lever switch.

  As described above, the imaging apparatus 300 according to the embodiment of the present invention can perform image processing on the captured data image generated by the image data generation unit 304 by the image processing unit 306. Here, the imaging apparatus 300 includes the image processing apparatus 100 according to the first embodiment of the present invention and / or the image processing apparatus 200 according to the second embodiment of the present invention in the image processing unit 306. Can do.

  Therefore, the imaging apparatus 300 according to the embodiment of the present invention removes noise from an image without detecting noise even if the image processed by the image processing unit 306 is an image having noise on the edge. Therefore, the image is not unnatural like an image corrected by a conventional image processing technique.

  In the embodiment of the present invention, the imaging apparatus 300 has been described. However, the embodiment of the present invention is not limited to such an embodiment, and is applied to an apparatus having an imaging function, such as a digital camera or a mobile phone with a camera function. can do.

  As mentioned above, although preferred embodiment of this invention was described referring an accompanying drawing, it cannot be overemphasized that this invention is not limited to the example which concerns. It will be apparent to those skilled in the art that various changes and modifications can be made within the scope of the claims, and these are naturally within the technical scope of the present invention. Understood.

  For example, in the image processing apparatus 100 according to the first embodiment of the present invention illustrated in FIG. 2, the interpolation value calculation unit 104 includes a first interpolation value calculation unit 112 that calculates a vertical interpolation value and a horizontal interpolation value. However, the present invention is not limited to this configuration, and the image processing apparatus according to the embodiment of the present invention may calculate three or more interpolation values. Good. Even in such a configuration, the fuzzy rules, membership functions, and the number of correction values calculated by the membership functions are set as appropriate (for example, the number of fuzzy rules, membership functions, correction values is increased). Even if the image to be processed is an image having noise on the edge, the noise of the image can be removed without detecting the noise.

  In the image processing apparatus 100 according to the first embodiment of the present invention illustrated in FIG. 2, an input image input to the image processing apparatus 100 is directly input to the corrected image generation unit 110. The image processing apparatus according to the embodiment of the present invention is not limited to the form, and may further include an input image delay unit that delays an input image (signal) input to the image processing apparatus before the correction image generation unit. Even in such a configuration, as in the image processing apparatus 100 according to the first embodiment of the present invention, even if the image to be processed is an image having noise on the edge, the image is not detected. Noise can be removed. Note that the input image delay unit in the above configuration can be configured using a memory such as a RAM (Random Access Memory) or a shift register, but is not limited thereto.

  The configuration described above can be easily changed by those skilled in the art, and should be understood as belonging to the equivalent scope of the present invention.

It is explanatory drawing which shows an example of the flow of the image processing in an imaging device. 1 is a block diagram illustrating an image processing apparatus according to a first embodiment of the present invention. It is explanatory drawing explaining calculation of the color difference signal which concerns on the 1st Embodiment of this invention. It is explanatory drawing explaining calculation of the interpolation value which concerns on the 1st Embodiment of this invention. It is explanatory drawing explaining calculation of the feature value which concerns on the 1st Embodiment of this invention. It is explanatory drawing which shows the 1st example using the membership function which concerns on the 1st Embodiment of this invention. FIG. 7 is an explanatory diagram illustrating an example of values set to calculate a correction value according to the embodiment of the present invention in the first example illustrated in FIG. 6. It is explanatory drawing which shows the 2nd example using the membership function which concerns on the 1st Embodiment of this invention. FIG. 9 is an explanatory diagram illustrating an example of values set for calculating a correction value according to the embodiment of the present invention in the second example illustrated in FIG. 8. It is a block diagram which shows the outline | summary of the process in the image processing apparatus which concerns on the 2nd Embodiment of this invention. It is explanatory drawing which shows the image processing method which concerns on embodiment of this invention. 1 is a block diagram illustrating an outline of an imaging apparatus according to an embodiment of the present invention.

Explanation of symbols

100, 200 Image processing apparatus 102 Feature image generation unit 104 Interpolation value calculation unit 106 Correction value calculation unit 108 Interpolation value correction unit 110 Correction image generation unit 112 First interpolation value calculation unit 114 Second interpolation value calculation unit 116 Feature value calculation unit 118 correction value setting unit 120 first correction unit 122 second correction unit 300 imaging device 302 imaging unit 304 image data generation unit 306 image processing unit 308 control unit 310 storage unit 312 display unit 314 operation unit

Claims (9)

An image processing device that processes an input image pixel by pixel:
A feature image generation unit that generates a predetermined feature image from the input image;
An interpolation value calculation unit that calculates a plurality of interpolation values for each pixel based on the feature image generated by the feature image generation unit;
A correction value calculating unit that calculates a feature value for each pixel from the input image and calculates a correction value for correcting each of a plurality of interpolation values calculated for each pixel based on the feature value;
An interpolation value correction unit that corrects each of the interpolation values calculated for each pixel in the interpolation value calculation unit based on the correction value for each pixel calculated in the correction value calculation unit;
A corrected image generation unit that corrects the feature image based on the interpolation value for each pixel corrected by the interpolation value correction unit;
An image processing apparatus comprising: The feature image generation unit generates a color difference image from the input image,
The interpolation value calculator is
A first interpolation value calculation unit for calculating an average value of pixel values of vertical pixels adjacent to the target pixel for each target pixel for which an interpolation value is calculated;
A second interpolation value calculation unit for calculating an average value of pixel values in pixels in the horizontal direction adjacent to the target pixel;
With
The correction value calculation unit
A feature value calculating unit that calculates the amount of change in the vertical direction and the amount of change in the horizontal direction of the target pixel as feature values;
Correction value setting for calculating and setting a correction value corresponding to the amount of change in the vertical direction calculated for each pixel of interest by the feature value calculation unit and a correction value corresponding to the amount of change in the horizontal direction for each pixel of interest Part;
With
The interpolation value correction unit
Based on a correction value corresponding to the amount of change in the vertical direction for each target pixel set by the correction value setting unit, a first correction for correcting the interpolation value for each target pixel calculated by the first interpolation value calculation unit A correction unit;
A second correction unit configured to correct the interpolation value for each target pixel calculated by the second interpolation value calculation unit based on a correction value corresponding to the amount of change in the horizontal direction for each target pixel set by the correction value setting unit; A correction unit;
With
The corrected image generation unit calculates, for each target pixel, an average value of the interpolation value for each target pixel corrected by the first correction unit and the interpolated value for each target pixel corrected by the second correction unit. The image processing apparatus according to claim 1, wherein a pixel value of each pixel of interest is replaced. The input image is a luminance image;
The image processing apparatus according to claim 1, wherein the feature image generation unit generates an edge image from the input image based on a luminance difference.   The image processing apparatus according to claim 1, wherein the correction value calculation unit calculates the correction value by fuzzy inference using a predetermined membership function. An image processing method for processing an input image pixel by pixel:
Generating a predetermined feature image from the input image;
Calculating a plurality of interpolated values for each pixel based on the feature image generated in the generating step;
Calculating a feature value for each pixel from the input image, and calculating a correction value for correcting each of the plurality of interpolation values calculated for each pixel based on the feature value;
Correcting the interpolation value calculated for each pixel calculated in the step of calculating the interpolation value for each pixel based on the correction value for each pixel calculated in the step of calculating the correction value;
Correcting the feature image based on the interpolation value for each pixel corrected in the step of correcting the interpolation value;
An image processing method characterized by comprising: An imaging apparatus capable of correcting a captured image obtained by imaging on a pixel-by-pixel basis:
An imaging unit for imaging;
An image data generation unit that generates an imaging data image from an imaging result captured by the imaging unit;
An image processing unit that receives the captured data image and corrects the captured data image;
With
The image processing unit
A feature image generation unit for generating a predetermined feature image from the image;
An interpolation value calculation unit that calculates a plurality of interpolation values for each pixel based on the feature image generated by the feature image generation unit;
A correction value calculating unit that calculates a feature value for each pixel from the input image and calculates a correction value for correcting each of a plurality of interpolation values calculated for each pixel based on the feature value;
An interpolation value correction unit that corrects each of the interpolation values calculated for each pixel in the interpolation value calculation unit based on the correction value for each pixel calculated in the correction value calculation unit;
A corrected image generation unit that corrects the feature image based on a correction value for each pixel corrected by the interpolation value correction unit;
An imaging apparatus comprising: The feature image generation unit receives the captured data image, generates a color difference image from the input image,
The interpolation value calculator is
A first interpolation value calculation unit for calculating an average value of pixel values of vertical pixels adjacent to the target pixel for each target pixel for which an interpolation value is calculated;
A second interpolation value calculation unit for calculating an average value of pixel values in pixels in the horizontal direction adjacent to the target pixel;
With
The correction value calculation unit
A feature value calculating unit that calculates the amount of change in the vertical direction and the amount of change in the horizontal direction of the pixel of interest as a feature value;
Correction value setting for calculating and setting a correction value corresponding to the amount of change in the vertical direction calculated for each pixel of interest by the feature value calculation unit and a correction value corresponding to the amount of change in the horizontal direction for each pixel of interest Part;
With
The interpolation value correction unit
Based on a correction value corresponding to the amount of change in the vertical direction for each target pixel set by the correction value setting unit, a first correction for correcting the interpolation value for each target pixel calculated by the first interpolation value calculation unit A correction unit;
A second correction unit configured to correct the interpolation value for each target pixel calculated by the second interpolation value calculation unit based on a correction value corresponding to the amount of change in the horizontal direction for each target pixel set by the correction value setting unit; A correction unit;
With
The corrected image generation unit calculates, for each target pixel, an average value of the interpolation value for each target pixel corrected by the first correction unit and the interpolated value for each target pixel corrected by the second correction unit. The imaging apparatus according to claim 6, wherein pixel values of the respective pixels of interest are replaced.   The imaging device according to claim 6, wherein a luminance image is input to the feature image generation unit, and an edge image is generated from the luminance image based on a luminance difference.   The imaging apparatus according to claim 6, wherein the correction value calculation unit calculates the correction value by fuzzy inference using a predetermined membership function.

Images