JP4880375B2 - Image signal processing apparatus and image signal processing method - Google Patents

Description

  The present invention relates to an image signal processing apparatus and an image signal processing method for suppressing color noise.

  Digital cameras that photograph subjects with CCD or MOS image sensors are widely used. Most of such digital cameras can shoot color images, and output image signals of three colors, for example, red (R), green (G), and blue (B), from the image sensor, An image signal obtained by converting the RGB signal into a luminance signal (Y) and a color difference signal (Cr, Cb) is recorded on a recording medium such as a memory card.

Image sensors are increasing in pixel count and image quality, and image quality degradation due to noise in captured images is a problem. In order to prevent such image quality degradation, it is normal to perform noise reduction processing on the image signal. Known noise reduction processing includes removal of high-frequency components by a low-pass filter and replacement of signal values by a media filter. (See, for example, Patent Document 1).
JP-A-4-235472

  By the way, with respect to color noise caused by including a noise component in a color signal such as a color difference signal, a sufficient effect is not obtained even if the above-described low-pass filter or media filter is used. There is a problem in that the image structure to be broken is destroyed and the image quality is deteriorated.

  The present invention has been made to solve the above problems, and provides an image signal processing apparatus and an image signal processing method capable of obtaining a good image quality with little color noise by performing noise reduction processing on an image signal. With the goal.

In order to achieve the above object, in the image signal processing apparatus according to claim 1, signal conversion means for converting the image signal into a luminance signal and a color difference signal for each pixel, and each pixel around the pixel to be processed The average level calculation means for calculating the average signal level of the color difference signal, the subtraction means for subtracting the average signal level from the signal level of the color difference signal of the pixel to be processed, and the signal level of the color difference signal subtracted by the subtraction means When it is in the vicinity of “0”, the signal level of the color difference signal is converted to “0” and output . For other color difference signals, the absolute value of the signal level is in the vicinity of “0”. level conversion means for performing absolute value of as small to level conversion processing for outputting the threshold for determining whether or not the output from the level conversion means, adding the average signal levels corresponding thereto It is obtained by a that adding means.

The image signal processing apparatus according to claim 2, further comprising: a correlation determining unit that determines the pixel correlation based on the luminance signal, wherein the average level calculating unit is configured to detect the neighboring pixels based on the determination result of the correlation determining unit. The average signal level is calculated using the color difference signals of the pixels having pixel correlation for the pixels to be processed .

4. The image signal processing method according to claim 3, wherein an average level calculating step of calculating an average signal level of the color difference signal for each pixel around the pixel to be processed of the luminance signal and the color difference signal, and a color difference of the pixel to be processed. A subtraction step of subtracting the average signal level from the signal level of the signal, and when the signal level of the subtracted color difference signal is near “0”, the signal level of the color difference signal is converted to “0” and output, For other color difference signals, a level conversion step of outputting the absolute value of the signal level by reducing it by the absolute value of the threshold value for determining whether or not the signal level is near “0”; And an addition step of adding an average signal level corresponding to the color difference signal obtained in the level conversion step .

5. The image signal processing method according to claim 4 , further comprising a correlation determination step for determining pixel correlation based on the luminance signal, and the average level calculation step based on the determination result of the correlation determination step. The average signal level is calculated using the color difference signal of the pixel having pixel correlation with the pixel to be processed.

  According to the present invention, when the color difference signal of the luminance signal and the color difference signal is near “0”, the signal level is converted to “0” and output. When the average signal level of the color difference signals of the surrounding pixels is subtracted from the color difference signal of the target pixel, when the signal level is near “0”, the signal level is converted to “0” and output. Therefore, color noise can be appropriately reduced.

  FIG. 1 shows a main configuration of a digital camera 10 according to the first embodiment of the present invention. An image sensor 12 is disposed behind the taking lens unit 11. The photographic lens unit 11 includes, for example, a photographic lens that automatically adjusts the focus so that the subject is in focus, a diaphragm that adjusts the amount of light incident on the image sensor 12, and the like. A subject image is formed on the light receiving surface of the image sensor 12 by the photographing lens unit 11.

  A large number of pixels (light receiving elements) are arranged in a matrix on the light receiving surface of the image sensor 12, and each pixel has one of red, green, and blue for color imaging. A color filter is provided. Various arrangements such as a Bayer arrangement can be used as the arrangement of the color filters, and the arrangement of the pixels is not limited to a square arrangement but may be a honeycomb arrangement. In this example, a color image is taken by one image sensor 12, but a color image may be taken by a so-called three-plate type in which an image sensor is provided for each color.

  The image sensor 12 photoelectrically converts the subject image formed on the light receiving surface to output the subject image as an electrical analog image signal. The analog signal processing unit 13 includes a correlated double sampling (CDS) circuit for removing noise included in the analog image signal and an amplifier that amplifies the analog image signal. The A / D converter 14 generates a digital image signal obtained by digitally converting the analog image signal processed by the analog signal processing unit 13. The digital image signal from the A / D converter 14 is sent to the data control unit 15. The data control unit 15 controls input / output of digital image signals between the respective units.

  The image processing unit 16 performs image processing such as white balance correction, γ conversion, and color interpolation, and color noise reduction processing for reducing color noise. The red, green, and blue digital image signals input from the data control unit 15 to the image processing unit 16 are subjected to white balance correction processing by increasing or decreasing the digital image signals of two colors at a ratio corresponding to the light source, After this, gamma conversion processing is performed and then color interpolation processing is performed, so that a digital image signal of three colors is obtained for each pixel. The three-color digital image signal subjected to the color interpolation processing is a digital image signal in which the luminance signal Y and the color difference signals Cr and Cb subjected to the color noise reduction processing by the YC conversion unit 17 and the level conversion unit 18 have been processed. Is output to the data control unit 15. Note that such a function of the image processing unit 16 can be configured by, for example, a digital signal processing dedicated processor (DSP) that performs high-speed data processing, a memory that holds data necessary for processing, and the like.

  The display unit 21 includes an LCD (Liquid Crystal Display), a drive circuit that drives the LCD, and the like, and displays a through image or a captured image when a digital image signal is input. The data recording unit 22 is for recording a photographed digital image signal on a recording medium. In addition to the recording medium, the data recording unit 22 compresses and decompresses the digital image signal, and the digital image is recorded on the recording medium. It consists of a media controller that writes and reads signals. Various recording media can be used. For example, a memory card that is detachable from the digital camera 10 is used. The digital image signal may be recorded in a memory built in the digital camera 10.

  The YC conversion unit 17 serving as a signal conversion unit converts a digital image signal composed of three color signals of red, green, and blue into one pixel obtained by color interpolation processing from a luminance signal Y and two color difference signals Cr and Cb. Into a digital image signal. As is well known, the luminance signal Y represents the luminance of the pixel, and the color difference signals Cr and Cb have color information to be expressed and take various positive and negative signal levels depending on the color to be expressed. It is. In order to reduce color noise, the level conversion unit 18 performs level conversion processing for converting the signal level of the color difference signal to “0” when the signal level of the color difference signal is in the vicinity of “0”.

  As shown in FIG. 2, the level converter 18 is preset with a threshold value Th + and a threshold value Th− for the signal levels of the input color difference signals Cr and Cb. The threshold Th + and the threshold Th− are set in the vicinity of the reference level (signal level “0”) of the color difference signal. That is, the threshold Th + has a positive value with respect to the reference level of the color difference signals Cr and Cb, and the threshold Th− has a negative value with respect to the reference level of the color difference signal (signal level “0”). Its absolute value (size) is assumed to be small.

  In the level conversion process, when the signal level of the input color difference signal Cr is in the vicinity of “0”, that is, when it is equal to or lower than the threshold value Th + and equal to or higher than the threshold value Th−, the color difference signal Cr having the signal level “0” is output. Similarly, when the signal level of the input color difference signal Cb is equal to or lower than the threshold value Th + and equal to or higher than the threshold value Th−, the color difference signal Cb having the signal level “0” is output.

  As described above, the signal level of the color difference signals Cr and Cb in the vicinity of the signal level “0” (threshold Th− to threshold Th +) is converted to “0”, thereby preventing the occurrence of conspicuous color noise. The threshold value Th− and the threshold value Th + are preferably set in a range of 1% to 10%, for example, and more preferably in a range of 1% to 5% with respect to the maximum value of the color difference signal. Good.

  In the level conversion process, when the signal levels of the input color difference signals Cr and Cb are not within the range of the threshold value Th− to the threshold value Th +, the absolute values of the signal levels of the color difference signals Cr and Cb are reduced by the absolute value of the threshold value. And output it. That is, when the signal level of the input color difference signals Cr and Cb is larger than the threshold value Th +, the signal level is decreased by the magnitude (absolute value) of the threshold value Th +, and when the signal level is smaller than the threshold value Th−, the signal level is decreased. The threshold value Th− is increased by the magnitude (absolute value). This prevents the gradation from becoming discontinuous.

  The color difference signals Cr and Cb output from the level conversion unit 18 are sent to the data control unit 15 together with the corresponding luminance signal Y of the same pixel.

  Next, the operation of the above configuration will be described. When the photographing mode is selected by an operation unit (not shown), the operation of the image sensor 12 is started, and an analog image signal obtained by photoelectrically converting the subject image is output. The analog image signal is input to the A / D converter 14 via the analog signal processing unit 13 and converted into a digital image signal. The digital image signal is sent from the data control unit 15 to the image processing unit 16, subjected to image processing and color noise reduction processing, and then sent to the display unit 21.

  Since the image sensor 12 photoelectrically converts the subject image repeatedly at a predetermined cycle and outputs an analog image signal, and the digital image signal is sequentially input to the display unit 21, the subject image is displayed on the LCD of the display unit 21. Is displayed. The operator performs framing by observing the through image, and takes a still image by pressing a release button (not shown).

  When the release button is pressed, after the focus of the taking lens unit 11 is readjusted, one frame of exposure is performed by the image sensor 12, and a digital image signal obtained as a result is sent from the data control unit 15 to the image processing unit 16. The image data and the color noise reduction process are performed and then sent to the data recording unit 22 and recorded on the recording medium.

  When displaying a through image as described above or recording on a recording medium, the digital image signal input to the image processing unit 16 is converted into red, green, and blue for each pixel by color interpolation processing. Digital image signal. Then, the digital image signals of these three colors are input to the YC conversion unit 17 and converted into a digital image signal composed of the luminance signal Y and the two color difference signals Cr and Cb as shown in FIG.

  Of the signals output from the YC conversion unit 17, two color difference signals Cr and Cb are respectively input to the level conversion unit 18, and as shown in FIG. 2, the signal level is equal to or lower than the threshold Th + and higher than the threshold Th−. Sometimes, the signal level is converted to “0”. When the signal level is larger than the threshold Th +, the signal level is decreased by the magnitude (absolute value) of the threshold Th +, and when the signal level is smaller than the threshold Th−, The level is greatly increased by the magnitude (absolute value) of the threshold Th-.

  The color difference signals Cr and Cb converted and output in this way by the level conversion unit 18 and the luminance signal Y of the same pixel are displayed via the data control unit 15 as a processed digital image signal of one pixel. Sent to the unit 21 and displayed on the LCD, or sent to the data recording unit 22 and recorded on the recording medium.

  By doing as described above, when the color difference signals Cr and Cb are in the vicinity of the signal level “0”, the color noise is inconspicuous in the low saturation region where the color noise is conspicuous because it is converted to the signal level “0”. Images can be displayed and recorded.

  FIG. 4 shows an example in which the threshold is adjusted according to the condition information. In addition, except being demonstrated below, it is the same as that of 1st embodiment, The same code | symbol is attached | subjected to the substantially same member, and the description is abbreviate | omitted.

  In the level conversion unit 18 in this example, threshold values Th + and Th− as parameters are set by the parameter calculation unit 31. The parameter calculation unit 31 determines the threshold values Th + and Th− based on the condition information, and sets the threshold value Th + and Th− corrected by the image characteristics in the level conversion unit 18.

  As shown in FIG. 5, when processing is performed, the condition information is first acquired by the parameter calculation unit 31. As such condition information, various information that affects color noise can be used. In this example, since the color noise tends to increase as the shooting sensitivity increases, the shooting sensitivity is adopted as the condition information. Note that when shooting in dark places such as night scenes, night scenes, or indoor scenes, or when shooting dark subjects, the shooting sensitivity may increase or the charge accumulation time of the image sensor may increase. There is a tendency for noise to increase, and there is a tendency for color noise to be conspicuous in many portions where the saturation of the subject is low. Therefore, for example, it is also useful to use the shooting mode such as the night scene shooting mode and the slow sync shooting mode, and the presence / absence of strobe light as the condition information. Furthermore, the brightness of the photographing lens, the characteristics of the set aperture value image sensor, and the like can be used as the condition information.

  The parameter calculation unit 31 determines threshold values Th + and Th− based on the acquired condition information. When determining the threshold values Th + and Th−, the parameter calculation unit 31 increases the threshold values Th + and Th− so that the magnitude (absolute value) of the threshold values Th + and Th− increases as the condition information increases the color noise. , Th− respectively.

  In this example, the parameter calculation unit 31 analyzes the characteristics of the digital image signal that is the color noise reduction process by referring to the digital image signal output from the A / D converter 14, and the condition information The threshold values Th + and Th− obtained based on the above are corrected. For example, the amount of color noise presumed to be included in one frame from a digital image signal is examined, and the ratio of the low gamut in one frame is checked. Correction is made to increase Th +, Th−, and correction is performed so that the threshold values Th +, Th− are reduced as the color noise is smaller and the proportion of the low chroma region is lower.

  The threshold values Th + and Th− obtained by the parameter calculation unit 31 as described above are set in the level conversion unit 18, and the level conversion unit 18 performs level conversion processing using the threshold values Th + and Th−. In this way, since the threshold values Th + and Th− are increased or decreased according to the tendency that the color noise is large or small, the color noise can be reduced in a more preferable state.

  FIG. 6 shows an example of controlling the presence / absence of level conversion. In addition, except being demonstrated below, it is the same as that of 1st embodiment, The same code | symbol is attached | subjected to the substantially same member, and the description is abbreviate | omitted.

  The saturation threshold calculation unit 34 receives condition information and calculates a saturation threshold (hereinafter referred to as a saturation threshold) when controlling whether or not to perform level conversion processing based on the input condition information. To do. Since the color noise is more conspicuous as the saturation is lower, as the condition information, various kinds of information that affects the color noise can be used as in the embodiment of FIGS. 5 and 6. As the color noise increases, The saturation threshold is increased so that many level conversions are performed in a wide saturation range. Note that the saturation threshold value may be set to a fixed level instead of being variable.

  The color difference signals Cr and Cb from the YC conversion unit 17 are sent to the level conversion unit 18, the determination unit 35, and the selection unit 36, respectively. The determination unit 35 and the selection unit 36 serve as conversion control means for controlling whether level conversion processing is validated or whether a color difference signal is output as it is. The determination unit 35 obtains the saturation of the pixel using the color difference signals Cr and Cb from the YC conversion unit 17, determines whether or not the saturation is equal to or less than the saturation threshold value, and selects the determination result as the selection unit 36. Send to. In this example, the square root value of the sum of squares of the color difference signals Cr and Cb is obtained as saturation, and the magnitude of this saturation with respect to the saturation threshold is determined.

  As described above, the color difference signals Cr and Cb from the YC conversion unit 17 and the color difference signals Cr and Cb via the level conversion unit 18 are input to the selection unit 36, respectively. The color difference signals Cr and Cb from either one are output to the data control unit 15. Thereby, when the saturation of the pixel is larger than the saturation threshold, the selection unit 36 selects and outputs the color difference signals Cr and Cb from the YC conversion unit 17, and when the saturation is below the saturation threshold, the level conversion unit. The color difference signals Cr and Cb from 17 are output.

  According to this example, as shown in FIG. 7, when the saturation is determined for each pixel and the saturation is larger than the saturation threshold, the luminance signal Y and the level from the YC conversion unit 17 are detected. The color difference signals Cr and Cb that have not been subjected to level conversion processing by the conversion unit 18 are output as digital image signals of the pixels, and when the color difference signals are equal to or lower than the saturation threshold, the luminance signal Y from the YC conversion unit 17 and level conversion are performed. The color difference signals Cr and Cb converted to the signal level “0” when the color difference signals Cr and Cb are in the vicinity of the signal level “0” in the unit 17 are output as digital image signals of the pixels.

  Therefore, the level conversion process is performed by the level conversion unit 17 only when the color noise is conspicuous and the signal level is “0” when the color difference signal is near the signal level “0”. Since the color difference signals Cr and Cb can be output, the color noise can be reduced without degrading the image quality.

  In the above embodiment, saturation is obtained as an index value as to whether or not color noise is noticeable, and the presence or absence of level conversion processing is determined based on the magnitude of the saturation and the saturation threshold. As long as it is an index as to whether or not color noise is noticeable near an achromatic color, it is not limited to saturation, but other appropriate index values, for example, the sum of absolute values of color difference signals Cr and Cb, etc. Can be used. Further, although the saturation is obtained from the color difference signals Cr and Cb, the saturation or an index value instead thereof may be obtained from the RGB image signal before being converted into the luminance signal and the color difference signal.

  Further, for example, an index value for determining whether or not the color noise is conspicuous from the color difference signal Cr is determined, and it is determined whether or not the level conversion processing of the color difference signal Cr is performed, and similarly the color noise is conspicuous from the color difference signal Cb. It may be configured to obtain a value serving as an index as to whether or not to perform level conversion processing of the color difference signal Cb accordingly.

  FIG. 8 shows an example in which color noise reduction processing is performed based on information about each peripheral pixel of the pixel to be processed. In addition, except being demonstrated below, it is the same as that of 1st embodiment, The same code | symbol is attached | subjected to the substantially same member, and the description is abbreviate | omitted.

  The average calculation unit 41 obtains each average signal level of the color difference signals Cr and Cb of each pixel around the pixel to be processed in the level conversion process. For example, 3 × 3 pixels or 5 × 5 pixels centered on the pixel to be processed can be used as the peripheral pixels when obtaining the average signal level. The subtractor 42 obtains a signal level obtained by subtracting the average signal level of the corresponding peripheral pixels from the color difference signals Cr and Cb from the YC conversion unit 17 for each of the color difference signals Cr and Cb. The level conversion circuit 18 performs level conversion processing on each signal level subtracted from the subtractor 42. The adder 43 adds an average signal level corresponding to the subtracted signal level subjected to the level conversion process.

  The level conversion process using the average signal level of the surrounding pixels as described above uses the fact that the color of the pixel to be processed and each of the surrounding pixels are approximate, that is, there is a pixel correlation. When peripheral pixels that do not have pixel correlation with the target pixel are used, appropriate processing may not be performed. For this reason, in this example, a correlation determination unit 44 that determines the pixel correlation between the pixel to be processed and its surrounding pixels is provided, and the average signal level is obtained based on the determination result of the correlation determination unit. It is determined whether or not to use as peripheral pixels.

  Based on the signal level of the luminance signal of the pixel to be processed and the signal level of the luminance signal of the surrounding pixel, the correlation determination unit 44 determines whether each peripheral pixel has pixel correlation with the pixel to be processed. Determine whether or not. Based on the discrimination result of the correlation discriminating unit 44, the average calculating unit 41 obtains each average signal level using only peripheral pixels discriminated to have pixel correlation.

  According to this example, as shown in FIG. 9, the average signal levels of the color difference signals Cr and Cb are obtained based on the luminance signal using peripheral pixels having pixel correlation with the pixel to be processed. Then, a subtracted signal level obtained by subtracting the average signal level of the color difference signal Cr of the peripheral pixels from the signal level of the color difference signal Cr of the pixel to be processed is obtained. Then, the average signal level of the previously obtained color difference signal Cr is added to the signal level obtained by performing the level conversion process on the subtracted signal level, and this is used as the color difference signal Cr in which the color noise is reduced. The color difference signal Cb is similarly processed.

  By doing so, level conversion processing for reducing color noise can be performed not only for pixels with a signal level near “0” but also for pixels with a high signal level of the color difference signal. The color noise can be reduced not only in the low saturation region but also in the high saturation region.

  The configurations described in the above embodiments can be combined with each other without conflicting. In each of the above embodiments, the example in which the image signal processing device of the present invention is incorporated in the digital camera has been described. However, the present invention is not limited to this, and can be used for various devices that handle image signals. It is also possible to take out only the portion for performing the color noise reduction process and make an independent image signal processing apparatus. Further, although cases have been described with the above embodiments where processing is performed using digitally converted image signals, similar processing may be performed on analog image signals.

It is a block diagram which shows the structure of the digital camera which implemented this invention. It is a graph explaining conversion of a level conversion process. It is a flowchart which shows the procedure of a color noise reduction process. It is a block diagram which shows the principal part structure of the example which enabled adjustment of the parameter. It is a flowchart which shows the procedure of the color noise reduction process in the example which enabled adjustment of the parameter. It is a block diagram which shows the principal part structure of the example which determines the presence or absence of a level conversion process according to saturation. It is a flowchart which shows the procedure of the color noise reduction process in the example which determines the presence or absence of a level conversion process according to saturation. It is a block diagram which shows the principal part structure of the example which performs a level conversion process to each pixel using the pixel correlation of a surrounding pixel. It is a flowchart which shows the procedure of the color noise reduction process in the example which performs a level conversion process to each pixel using the pixel correlation of a surrounding pixel.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Digital camera 12 Image sensor 17 YC conversion part 18 Level conversion part 35 Discriminating part 36 Selection part 41 Average calculation part 42 Subtractor 43 Adder 44 Correlation discrimination | determination part

Claims (4)

In an image signal processing apparatus that performs processing for suppressing color noise on an image signal output from an image sensor,
Signal conversion means for converting an image signal into a luminance signal and a color difference signal for each pixel;
Average level calculating means for calculating the average signal level of the color difference signal for each pixel around the pixel to be processed;
Subtracting means for subtracting the average signal level from the signal level of the color difference signal of the pixel to be processed;
When the signal level of the subtraction color difference signals in the subtracting means is near "0", the signal level of the color difference signals and converts to "0", for the other of the color difference signal, that Level conversion means for performing level conversion processing for outputting the absolute value of the signal level by reducing it by the absolute value of the threshold value for determining whether or not the signal level is near “0”;
An image signal processing apparatus comprising an adding means for adding an average signal level corresponding to the output from the level converting means. Correlation determining means for determining pixel correlation based on the luminance signal,
The average level calculating unit calculates an average signal level using a color difference signal of a pixel having pixel correlation with a pixel to be processed among peripheral pixels based on a determination result of the correlation determining unit. The image signal processing apparatus according to claim 1. In an image signal processing method for performing processing for suppressing color noise on an image signal,
An average level calculating step for calculating an average signal level of the color difference signal for each pixel around the pixel to be processed of the luminance signal and the color difference signal;
A subtraction step of subtracting the average signal level from the signal level of the color difference signal of the pixel to be processed;
When the signal level of the subtraction color difference signal is near "0", the signal level of the color difference signals and converts to "0", for the other of the color difference signal, the absolute of the signal level A level conversion step of outputting a value that is reduced by an absolute value of a threshold value for determining whether or not the signal level is near “0”;
An image signal processing method comprising: an addition step of adding an average signal level corresponding to the color difference signal obtained in the level conversion step. A correlation determination step of determining pixel correlation based on the luminance signal;
The average level calculating step calculates an average signal level using a color difference signal of a pixel having pixel correlation with a pixel to be processed among peripheral pixels based on a determination result of the correlation determining step. The image signal processing method according to claim 3 .

Images