JP4599279B2 - Noise reduction device and noise reduction method - Google Patents

Description

  The present invention relates to a noise reduction device that performs noise reduction (NR) processing on RAW image data output from a solid-state imaging device.

  Conventionally, a signal output from a solid-state imaging device has a poor S / N when performing low-sensitivity imaging or the like, so that a sufficiently satisfactory image quality cannot be obtained. In such a case, in addition to normal noise, false color and color noise caused by noise including low-frequency components are a problem, and this is particularly noticeable in dark areas where the luminance is low. there were. Conventionally, various methods for performing NR processing have been proposed (see Patent Documents 1 to 4).

JP 2000-232384 A Japanese Patent Laid-Open No. 2003-150956 JP-A-10-63839 JP 2000-175046 A

  In order to effectively remove noise included in a low-luminance portion of an image, it is effective to perform NR processing on pixel data to be subjected to NR processing using pixel data within a wide range around it. However, when NR processing is performed using a wide range of pixel data, the image quality in the low-luminance part is improved, but the image quality in other parts may be deteriorated.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide a noise reduction device capable of effectively removing noise and improving image quality.

  A noise reduction device according to the present invention is a noise reduction device that performs noise reduction of RAW image data obtained from a solid-state imaging device, and includes a large area having a large size including pixel-of-interest data of the RAW image data, and the large area An average value for calculating an average value of pixel data having the same color component as the target pixel data included in the area (hereinafter referred to as area same color pixel data) for each of a plurality of areas including at least a small area having a smaller size than A calculation unit; a variance calculation unit that calculates a variance of the same color pixel data for each of the plurality of areas based on the average value; and a noise amount estimation unit that estimates a noise amount included in the target pixel data; Based on the variance for each of the plurality of areas, the noise of the pixel-of-interest data is selected from the plurality of areas. Optimal area determination means for determining an optimal area that is the optimal area to be used for the reduction process, and the target pixel data is brought close to the average value calculated for the optimal area according to the estimated noise amount. Noise reduction processing means for performing processing and performing noise reduction processing of the pixel-of-interest data.

  The noise reduction device according to the present invention is a threshold value that is a determination criterion for determining the optimum area, and a threshold value that is determined according to the amount of noise is a first threshold value. The optimum area is determined using the variance for each area, the average value for each of the plurality of areas, and the first threshold value.

  The noise reduction device according to the present invention is a threshold value serving as a determination criterion for determining whether noise reduction processing should be performed on the pixel-of-interest data using the optimal area, and is determined according to the noise amount. Determine whether or not noise reduction should be performed on the pixel-of-interest data using the optimum area using the variance calculated for the optimum area and the second threshold as a second threshold. Noise reduction execution determining means, and output means for outputting either the target pixel data or the target pixel data after the noise reduction processing according to a determination result by the noise reduction execution determination means.

  In the noise reduction device of the present invention, the RAW image data includes pixel data of a red (R) component, a green (G) component, and a blue (B) component, and the target pixel data is the R component or the B component. If there is, the optimum area determination means determines whether the large area is the optimum area based on the dispersion of the R component or B component pixel data included in the large area and the first threshold value. If the large area is not the optimal area, based on the dispersion of the R component or B component pixel data included in the small area and the first threshold, It is determined whether or not a small area is the optimal area, and the noise reduction execution determining means determines that the target pixel data is determined when the large area or the small area is determined as the optimal area. It determines that it should be performing the noise reduction capacitor, wherein when the small area is determined not to be the optimal area, determines not to make any noise reduction on the target pixel data.

  In the noise reduction device according to the present invention, the noise amount estimation unit corresponds to a noise amount included in pixel data of each gradation obtained in advance by RAW image data obtained by imaging a gradation chart with the solid-state imaging device. The amount of noise included in the target pixel data is estimated based on the data to be processed.

  The noise reduction method of the present invention is a noise reduction method for performing noise reduction of RAW image data obtained from a solid-state imaging device, and includes a large area having a large size including target pixel data of the RAW image data, and the large area An average value for calculating an average value of pixel data having the same color component as the target pixel data included in the area (hereinafter referred to as area same color pixel data) for each of a plurality of areas including at least a small area having a smaller size than A calculation step; a variance calculation step for calculating a variance of the same color pixel data for each of the plurality of areas based on the average value; and a noise amount estimation step for estimating a noise amount included in the target pixel data; , The pixel of interest among the plurality of areas based on the variance for each of the plurality of areas An optimal area determination step of determining an optimal area that is an optimal area to be used for the noise reduction processing of the data, and the average value calculated for the optimal area according to the estimated noise amount And a noise reduction processing step of performing a process of bringing the pixel data closer to perform a noise reduction process of the target pixel data.

  ADVANTAGE OF THE INVENTION According to this invention, the noise reduction apparatus which can remove a noise effectively and can improve an image quality can be provided.

  Embodiments of the present invention will be described below with reference to the drawings.

FIG. 1 is a diagram showing a schematic configuration of a main part of a digital camera for explaining an embodiment of the present invention.
A digital camera shown in FIG. 1 includes a solid-state imaging device 1, an A / D converter 2, a buffer 3, a preprocessing circuit 4, a timing generator (TG) 5, a pixel data holding unit 6, a noise reduction ( NR) device 7 and a signal processing unit 8.

  The solid-state imaging device 1 is a CCD type or MOS type color solid-state imaging device having a large number of photoelectric conversion elements arranged in a row direction on a semiconductor substrate and in a column direction perpendicular thereto. As shown in FIG. 2A, the solid-state imaging device 1 has a large number of photoelectric conversion elements 10 arranged in a square lattice pattern in a row direction (X direction) on a semiconductor substrate and a column direction (Y direction) perpendicular thereto. As shown in FIG. 2B, a so-called honeycomb array in which a large number of photoelectric conversion elements 10 are arranged as described in JP-A-10-136391 can be used. In the following description, it is assumed that a large number of photoelectric conversion elements included in the solid-state imaging element 1 are arranged in a honeycomb shape. In this embodiment, data obtained by digitizing an imaging signal output from one photoelectric conversion element is defined as pixel data.

  The solid-state imaging device 1 has a primary color filter, and a number of photoelectric conversion elements include, for example, a photoelectric conversion element that detects red (R), a photoelectric conversion element that detects green (G), and a blue color. And a photoelectric conversion element for detecting (B). The color filter may be a complementary color type. The arrangement of the color filters is not particularly limited as long as it is a regular arrangement (an arrangement in which RGB relative positions are uniquely determined).

  The A / D converter 2 digitally converts a large number of pixel data output from the solid-state image sensor 1. A large number of pixel data digitally converted here becomes so-called RAW image data. The RAW image data is data obtained by digitizing the dot-sequential imaging signal output from the solid-state imaging device 1 as it is, and since this data remains as it is, only pixel data of one color component exists at one imaging point. The color image data cannot be generated.

  The buffer 3 absorbs a driving speed difference between the solid-state imaging device 1 and the signal processing unit 8 and is configured by a general FIFO or the like.

  The preprocessing circuit 4 performs various signal processing such as OB correction, gain adjustment, smear correction, shading correction, and gamma correction on the RAW image data supplied from the buffer 3.

  The pixel data holding unit 6 holds a large number of pixel data output from the preprocessing circuit 4 and supplies the pixel data to the NR device 7. The pixel data holding unit 6 has N−1 line memories 61 and N delays. And an element 62.

  The NR device 7 performs NR processing on the pixel data output from the pixel data holding unit 6 by a processing algorithm described later.

  The signal processing unit 8 performs a synchronization process, a YC conversion process, and the like in which a single imaging point has pixel data of R (red), G (green), and B (blue) from a large number of pixel data after NR processing. . Through these processes, a color image can be reproduced.

  It is assumed that pipeline processing is performed in any of the preprocessing circuit 4, the pixel data holding unit 6, the NR device 7, and the signal processing unit 8, thereby enabling high-speed sequential processing of pixel data. The preprocessing circuit 4 may be provided between the NR device 7 and the signal processing unit 8 as shown in FIG. 3 as long as it can perform signal processing on the RAW image data.

  In the NR device 7, a plurality of areas including target pixel data to be subjected to NR processing are set for a large number of pixel data supplied from the pixel data holding unit 6, and the target pixel data is determined from the plurality of areas. An optimum area, which is an optimum area for use in noise reduction processing, is determined, and NR processing is performed on target pixel data using pixel data included in the optimum area.

FIG. 4 is an image diagram in which the pixel data supplied from the pixel data holding unit 6 shown in FIG. 1 to the NR device 7 is two-dimensionally arranged.
In the NR device 7, for example, as shown in FIG. 4, a large area L having a large size including target pixel data, a small area S having a small size including target pixel data, and a horizontally long horizontal area H including target pixel data, A vertically long vertical area V including the target pixel data is set. Areas S, H, and V are set inside area L.

FIG. 5 is a block diagram showing a schematic configuration of the NR device 7 shown in FIG.
The NR device 7 includes an average calculation unit 71, a mean square calculation unit 72, a determination circuit 73, a memory 74, an NR processing unit 75, and an output unit 76.

  In the NR device 7, the noise amount estimated to be included in the pixel-of-interest data is stored in the memory 74 in advance, so that the circuit scale required for estimating the noise amount is reduced. For example, a flat 256 gray scale chart is prepared, and this is photographed by the solid-state imaging device 1 at each photographing sensitivity from low sensitivity to high sensitivity. Then, an average deviation of pixel data of each color component included in the pixel data obtained by photographing each gradation image of the gray scale chart is obtained. Then, as shown in FIG. 6, a deviation histogram indicating the average of the deviations for each gradation from 0 to 255 is stored in the memory 74. In the memory 74, a deviation histogram as shown in FIG. 6 is stored for each photographing sensitivity for each of the R component, the G component, and the B component. The average deviation of the pixel data of each color component included in the pixel data obtained by photographing an image of a certain gradation corresponds to the noise amount of each color component included in the image of that gradation. That is, by using the deviation histogram shown in FIG. 6, it is possible to estimate the amount of noise included in the pixel data obtained by photographing.

  Note that the memory 74 can be made more compact by dividing the image measure into blocks having a plurality of pixel values and connecting the closest linear lines in each block, as shown by the broken lines in FIG. . Which configuration is used depends on the trade-off between the degree of freedom and the amount of hardware.

  The average calculation unit 71 calculates the average of pixel data of the same color component as the target pixel data included in the area (hereinafter referred to as area same color pixel data) for each area, and outputs the average to the determination circuit 73.

  The mean square calculation unit 72 calculates the mean square of the same color pixel data for each area and outputs the result to the determination circuit 73. If the area same color pixel data is val, the root mean square of the area same color pixel data is

It is obtained by Since each area has an overlapping portion, an efficient circuit can be realized by devising an integration path of image data.
FIG. 7 is a diagram illustrating an example of an accumulation path of G component pixel data when the small area S is 5 rows × 5 columns, the large area L is 13 rows × 13 columns, and the target pixel data is the G component. .
Reference numerals 71a, 71d, and 71f shown in FIG. 7 are delay elements. Reference numerals 71b, 71c, 71e, 71g, 71h, and 71i shown in FIG. 7 are integrators. As shown in FIG. 7, by integrating the integration result in the small area S with the integration result in the large area L, it is possible to avoid duplication calculation and to perform efficient calculation.

  The discriminating circuit 73 includes a variance calculating unit 77, an optimum area determining unit 78, an NR execution determining unit 79, and a noise amount estimating unit 80.

  The variance calculation unit 77 calculates the variance of the area same color pixel data for each area. The distribution of the area same color pixel data can be calculated by the following formula 1 when the area same color pixel data is val and the average of the area same color pixel data is ave. In this embodiment, the distribution is more suitable for calculation. It is assumed that the variance is calculated using the approximate expression of Expression 2. Of course, the variance may be calculated using Equation 1. Since the first term on the right side of Equation 2 is the mean square and the second term on the right side is the mean square, the variance calculating unit 77 is based on the data supplied from the average calculating unit 71 and the mean square calculating unit 72. The variance of the area same color pixel data can be calculated.

  The noise amount estimation unit 80 acquires a deviation corresponding to the pixel value of the target pixel data from the deviation histogram stored in the memory 74, and estimates the acquired deviation as a noise amount included in the target pixel data. A known method may be used as the noise amount estimation method.

  Based on the variance of each area, the optimal area determination unit 78 determines an optimal area that is the optimal area to be used for the noise reduction process of the pixel data of interest among the areas. The determination algorithm will be described later.

  The NR execution determination unit 79 determines whether or not to perform NR processing on the target pixel data, and notifies the NR processing unit 75 and the output unit 76 of the determination result.

  When the NR processing unit 75 receives a notification from the NR execution determination unit 79 that NR processing should be performed, the NR processing unit 75 performs NR processing to reduce noise included in the target pixel data, and outputs the target pixel data after the NR processing To the unit 76. In the present embodiment, the NR process is performed by paying attention to the fact that the noise of the pixel-of-interest data is reduced when the variance of the optimum area is reduced. In order to reduce the variance of the optimum area, the deviation of the area same color pixel data included in the optimum area may be reduced.

  FIG. 8 is a diagram showing an image of processing for reducing the variance of the optimum area. The solid line in FIG. 8 shows the distribution of deviation of the same color pixel data in the optimum area, and the broken line in FIG. 8 shows the distribution of deviation of the same color pixel data in the optimum area after the variance is reduced. Yes. By performing the process of changing the distribution of the solid line in FIG. 8 to the distribution of the broken line, it is possible to reduce the variance of the optimum area without degrading the image quality.

  In order to obtain the distribution shown by the broken line in FIG. 8, the difference between the average value calculated for the optimum area and the pixel data of the same color for each area is multiplied by the coefficient ratio as shown in the following Expression 3. It is sufficient to perform a process of reducing

Equation 3: Coefficient ratio = f + {(f−g) / (sigma × h)} × | valc-aves |
Here, f, g, and h are predetermined coefficients, sigma is a deviation corresponding to the target pixel data, valc is a pixel value of the target pixel data, and aves is an average value calculated for the optimum area. .

  Since the NR processing of the target pixel data needs to be performed according to the amount of noise estimated to be included in the target pixel data, the coefficient ratio becomes a value that changes according to the deviation sigma corresponding to the target pixel data. Yes.

  Actually, the purpose is to reduce the noise contained in the pixel data of interest, so the process of reducing the difference by multiplying the difference between the average value calculated for the optimal area and the same color pixel data of each area by the coefficient ratio May be performed only for the target pixel data. That is, in the actual operation, the NR processing unit 75 calculates the coefficient ratio based on the deviation (sigma) corresponding to the pixel-of-interest data, and calculates the optimum area by the following expression 4 using the coefficient ratio. Performs a process to reduce the difference between the average value and the target pixel data (synonymous with the process of bringing the target pixel data closer to the average value calculated for the optimal area according to the estimated noise amount), and the noise of the target pixel data Perform reduction processing.

  Expression 4: Remarked pixel data after NR processing val_new = aves + (valc-aves) × ratio

  False color and color noise in the low-luminance portion of the image can be effectively reduced by using a wide range of images. That is, it is effective to set the large area L as the optimum area in order to reduce the false color and the color noise in the low luminance part of the image. However, if NR processing is performed using only the large area L, the image quality cannot be maintained. Therefore, in the present embodiment, in addition to the large area L, a small area S, a horizontal area H, and a vertical area V are set, and these areas are selectively used according to the image, thereby improving the image quality. ing.

  Based on the determination result of the NR execution determination unit 79, the output unit 76 outputs either the target pixel data after NR processing by the NR processing unit 75 or the target pixel data that has not been subjected to NR processing, to the signal processing unit. 8 is output. The target pixel data is supplied to the output unit 76 via the determination circuit 73.

Next, the operation of the noise reduction device 7 will be described.
FIG. 9 is a diagram illustrating an operation flow of the noise reduction device illustrated in FIG. 5 when the target pixel data is the G component. The operation when the pixel-of-interest data is the R component or the G component is the same as that in FIG. In FIG. 9, symbols for the large area L, the small area S, the horizontal area H, and the vertical area V are omitted.
When imaging is performed and RAW image data is input, each of the large area L, the small area S, the horizontal area H, and the vertical area V is obtained by the average calculating unit 71, the mean square calculating unit 72, and the variance calculating unit 77. And the variance are calculated (S10).

  Next, the noise amount estimation unit 80 corresponds to the pixel value of the target pixel data from the deviation histogram corresponding to the photographing sensitivity of the photographing performed to obtain the input RAW image data and the color component of the target pixel data. The deviation sigma to be acquired is acquired (S20).

  Next, the optimal area determination unit 78 performs NR processing on the target pixel data using the threshold A and threshold B, which are the first thresholds that are determination criteria for determining the optimal area, and the optimal area from the deviation sigma. A threshold value C, which is a second threshold value that is a criterion for determining whether or not to perform the process, is generated (S30). The threshold A is a value obtained by multiplying the deviation sigma by a predetermined coefficient a. The threshold value B is a value obtained by squaring a value obtained by multiplying the deviation sigma by a predetermined coefficient b. The threshold C is a value obtained by squaring a value obtained by multiplying the deviation sigma by a predetermined coefficient c.

Next, the optimum area determination unit 78 compares the variance and average of the large area L and the small area S (S40). Comparing the dispersion of the large area L and the small area S is because it is considered that there is less change in the image in the area where the dispersion is small, so either the large area L or the small area S is suitable for performing NR processing. This is to determine whether or not
Even if the variance of the small area S is smaller than the variance of the large area L, if the average of the large area L and the small area S is not so different, the image in the large area L and the small area S It seems that there is not much difference in the images. Since the false color and color noise in the low-luminance part have a great influence on the image quality, it is desirable to use the large area L preferentially in such a case. For this reason, the optimum area determination unit 78 compares the average of the large area L and the small area S to determine which of the large area L and the small area S is suitable for performing NR processing.

  As a result of the comparison in S40, the variance of the small area S is greater than the variance of the large area L, or the average difference between the large area L and the small area S is less than a predetermined percentage of the threshold A (S40). : N), the optimal area determination unit 78 determines the large area L as the optimal area (S41). However, the optimal area determined here is only determined to be optimal among the four areas, and it is not clear whether the image in the large area L has little change. If the image in the large area L has little change, a good image can be obtained by performing NR processing. However, when the image in the large area L has many changes such as a mesh, if the NR process is performed, the edges are rounded and the resolution is lowered. Therefore, in the noise reduction device 7, after the large area L is determined as the optimum area, the NR execution determination unit 79 compares the variance of the large area L with the threshold C, and determines whether or not to perform NR processing on the large area L. Is finally determined (S42).

  When the variance of the large area L is larger than the threshold value C (S42: Y), since the image in the large area L is considered to have many changes, the NR execution determination unit 79 performs NR processing on the target pixel data. (S42Y), and notifies the NR processing unit 75 and the output unit 76 to that effect. On the other hand, when the variance of the large area L is equal to or less than the threshold value C (S42: N), since the image in the large area L is considered to have little change, the NR execution determination unit 79 adds the NR to the target pixel data. It is determined that processing is to be performed, and that effect is notified to the NR processing unit 75 and the output unit 76. The NR processing unit 75 uses the average of the optimal area determined by the optimal area determination unit 78, the deviation sigma, and the target pixel data when there is a notification that the target pixel data should be subjected to NR processing. NR processing is performed (S60). The output unit 76 outputs the target pixel data or the target pixel data after the NR process according to the notification from the NR execution determination unit 79.

  As a result of the comparison in S40, the variance in the small area S is smaller than the variance in the large area L, and the average difference between the large area L and the small area S is equal to or greater than a predetermined percentage of the threshold A (S40). : Y), the optimal area determination unit 78 determines the large area L as a non-optimal area.

  Next, the optimum area determination unit 78 determines whether or not the difference between the variance of the vertical area V and the variance of the horizontal area H is equal to or greater than a predetermined percentage of the threshold B. If the difference between the variance of the vertical area V and the variance of the horizontal area H is less than a predetermined percentage of the threshold value B (S43: N), it is considered that the image in the small area S is little changed vertically and horizontally. Therefore, in such a case, the optimum area determination unit 78 determines the small area S as the optimum area (S44).

  After the small area S is determined as the optimum area, the NR execution determination unit 79 compares the dispersion of the small area S with the threshold value C and finally determines whether or not the NR processing should be performed on the small area S ( S45). When the variance of the small area S is larger than the threshold value C (S45: Y), the NR execution determination unit 79 determines that the target pixel data should not be subjected to NR processing (S45Y), and informs that effect. 75 and the output unit 76. On the other hand, when the variance of the small area S is equal to or less than the threshold value C (S45: N), the NR execution determination unit 79 determines that the target pixel data should be subjected to NR processing, and informs the NR processing unit 75 and the output unit. 76 is notified. Thereafter, the process proceeds to S60.

  On the other hand, in S43, when the difference between the variance of the vertical area V and the variance of the horizontal area H is equal to or greater than a predetermined percentage of the threshold value B (S43: Y), the image in the small area S changes vertically and horizontally. Since it is considered to be large, in such a case, the optimum area determination unit 78 determines the small area S as a non-optimal area.

  Next, the optimum area determination unit 78 compares the variances of the horizontal area H and the vertical area V (S46). When the variance of the horizontal area H is larger (S46: Y), the optimal area determination unit 78 determines the vertical area V as the optimal area (S47). After the vertical area V is determined as the optimum area, the NR execution determination unit 79 compares the dispersion of the vertical area V with the threshold value C and finally determines whether or not NR processing should be performed on the vertical area V ( S48). When the variance of the vertical area V is larger than the threshold value C (S48: Y), the NR execution determination unit 79 determines that the target pixel data should not be subjected to NR processing (S48Y), and informs the NR processing unit. 75 and the output unit 76. On the other hand, when the variance of the vertical area V is equal to or less than the threshold value C (S48: N), the NR execution determination unit 79 determines that the target pixel data should be subjected to NR processing, and notifies that to the NR processing unit 75 and the output unit. 76 is notified. Thereafter, the process proceeds to S60.

  In S46, when the variance of the vertical area V is larger (S46: N), the optimal area determination unit 78 determines the horizontal area H as the optimal area (S49). After the horizontal area H is determined as the optimum area, the NR execution determination unit 79 compares the variance of the horizontal area H with the threshold C, and finally determines whether or not the horizontal area H should be subjected to NR processing ( S50). When the variance of the horizontal area H is larger than the threshold value C (S50: Y), the NR execution determination unit 79 determines that the target pixel data should not be subjected to NR processing (S50Y), and informs the NR processing unit. 75 and the output unit 76. On the other hand, when the variance of the horizontal area H is equal to or less than the threshold value C (S50: N), the NR execution determination unit 79 determines that the target pixel data should be subjected to NR processing, and notifies that to the NR processing unit 75 and the output unit. 76 is notified. Thereafter, the process proceeds to S60.

FIG. 10 is a diagram illustrating an image during operation of the noise reduction device illustrated in FIG. 5. Broken lines shown in FIG. 10 indicate a large area and a small area. The image shown in FIG. 10 is obtained when the above-described vertical area and horizontal area are not considered.
When the area is set to a position such as (A), (C), (D), (E), there is not much difference in the variance and average of the large area L and the small area S. Is determined as the optimum area. Further, when the area is set at a position such as (B) or (F), the dispersion of the small area S is smaller than the dispersion of the large area L, and the average of the large area L and the small area S is obtained. Since the difference is large and there is not much difference between the horizontal area H and the vertical area V, the small area S is determined as the optimum area. Even if the area is determined to be the optimum area, in the case of (C) or (D) where boundaries or meshes exist in the optimum area, NR processing is performed in order not to reduce the resolution. Absent.

  As described above, according to the noise reduction device 7, since a plurality of areas are set and NR processing is performed selectively using the areas in accordance with the images contained in each area, the image quality is improved. Can be improved. In addition, since the optimum area is determined by comparing the variance of each area, the image can be determined with a simple algorithm. Further, since the noise amount is estimated from the deviation histogram obtained in advance and the image is discriminated in consideration of this noise amount, high discrimination accuracy can be realized.

  Further, according to the noise reduction device 7, after determining the optimum area by determining the image, it is determined whether the NR processing should be performed using the threshold value C in consideration of the amount of noise. It is possible to suppress a decrease in resolution that occurs due to performing NR processing on an image. Note that the determination processing by the NR execution determination unit 79 may be omitted if the reduction in resolution is ignored. In this case, in FIG. 9, the NR processing unit 75 may start the NR processing immediately after the processing of S41, S44, S47, and S49.

  Further, according to the noise reduction device 7, since it is not necessary to have a circuit for estimating the amount of noise, the circuit scale can be reduced as compared with the prior art.

  Further, the noise reduction device 7 performs the NR processing of the pixel data of interest using the idea of reducing the variance of the optimum area. When NR processing such as filtering is performed on the optimal area, there is a possibility that an unnatural contour may be formed due to switching of a plurality of areas. However, as in this embodiment, the idea of reducing the statistical value of variance is reduced. By using, the generation of an unnatural contour can be prevented. However, if there is too much difference between the size of the large area L and the small area S, the difference in dispersion between the large area L and the small area S becomes large. It becomes an image with. Therefore, it is desirable to keep the size of the large area L to about twice the size of the small area S. Alternatively, it is desirable to take measures such as adding an intermediate size area between the large area L and the small area S.

  In the above description, the average difference between the large area L and the small area S is compared with the threshold A in S40 of FIG. 9 in order to increase the determination accuracy of the optimum area, but this comparison is omitted. Also good. In this case, in S40, only the dispersion of the large area L and the small area S is compared. If the large area L is larger, the process proceeds to S41. If the small area S is larger, the process proceeds to S43. .

  In the above description, four areas are set. However, the area to be set may be at least two of the large area L and the small area S. By setting the horizontal area H and the vertical area V as in the present embodiment, it is possible to finely discriminate an image and to perform highly accurate NR processing. When the horizontal area H and the vertical area V are not set, the operation flow is such that the process proceeds to S44 when the determination in S40 of FIG. 9 is YES.

  In the above description, the same processing is performed by the noise reduction device 7 regardless of whether the target pixel data is the R component, the G component, or the B component. However, the target pixel data is the R component or the B component. In this case, there is no problem even if the noise reduction device 7 is operated with a simpler algorithm. Hereinafter, this simple algorithm will be described.

FIG. 11 is a diagram illustrating an operation flow of the noise reduction device 7 when the target pixel data is an R component or a B component. In FIG. 11, the symbols for the large area L and the small area S are omitted. In FIG. 11, the R component or the B component is expressed as R / B.
When imaging is performed and RAW image data is input, the average calculation unit 71, the mean square calculation unit 72, and the variance calculation unit 77 perform the R component or the B component included in each of the large area L and the small area S. The average and variance and the variance of the G component included in each of the large area L and the small area S are calculated (S70).

  Next, the noise amount estimation unit 80 calculates the deviation sigma corresponding to the pixel value of the target pixel data from the imaging sensitivity of the imaging performed to obtain the input RAW image data and the deviation histogram corresponding to the target pixel data. One of the G component pixel data adjacent to the target pixel data from the deviation histogram corresponding to the photographing sensitivity and G component of the photographing performed to obtain (R / B) and obtain the input RAW image data The deviation sigma (G) corresponding to is acquired (S80).

  The optimal area determination unit 78 generates a threshold D, which is a first threshold serving as a determination criterion for determining the optimal area, from the deviation sigma (R / B), and determines the optimal area from the deviation sigma (G). A threshold value E, which is a first threshold value that is a determination criterion for the determination, is generated (S90). The threshold value D is a value obtained by squaring a value obtained by multiplying the deviation sigma (R / B) by a predetermined coefficient d. The threshold E is a value obtained by squaring a value obtained by multiplying the deviation sigma (G) by a predetermined coefficient e.

  Next, the optimum area determination unit 78 compares the variance of the R component or B component included in the large area L with the threshold D, and compares the variance of the G component included in the large area L with the threshold E (S100). ). Here, the reason why the variance of the G component is compared with the threshold value E is that only the R component or the B component has low sensitivity to luminance and it is difficult to discriminate an image. When both the dispersion of the R component or B component included in the large area L and the dispersion of the G component included in the large area L are less than the threshold values D and E (S100: N), in this case, Since it is considered that there is little change in the image in the area L, the optimal area determination unit 78 determines the large area L as the optimal area (S100N).

  On the other hand, when either the dispersion of the R component or B component included in the large area L or the dispersion of the G component included in the large area L is equal to or greater than the thresholds D and E (S100: Y), Since the image in the large area L is considered to have many changes, the optimal area determination unit 78 determines the large area L as a non-optimal area, the variance of the R component or B component contained in the small area S and the threshold D And the variance of the G component included in the small area S and the threshold E are compared (S120).

  When both the dispersion of the R component or B component included in the small area S and the dispersion of the G component included in the small area S are less than the threshold values D and E (S120: N), Since it is considered that there is little change in the image in the area S, the optimal area determination unit 78 determines the small area S as the optimal area (S120N).

  In the operation when the pixel-of-interest data is the G component, the NR execution determination unit 79 determines whether or not to perform NR processing after determining the optimal area in S100N or S120N. However, since most of false color and color noise in the low luminance part are caused by the R component and the B component, and in order to balance the G component, it is better to perform NR processing as much as possible. . Further, since the R component or the B component does not contribute to the resolution as much as the G component, the image quality deterioration due to the NR processing is not so great. For this reason, the NR execution determination unit 79 determines that NR processing should be performed after the optimal area is determined in S100N or S120N. Thus, NR processing is always performed after the optimum area is determined (S60), and the target pixel data after NR processing is output from the output unit 76.

  In S120, when either the dispersion of the R component or B component included in the small area S or the dispersion of the G component included in the small area S is equal to or greater than the thresholds D and E (S120: Y), in this case Since it is considered that there are many changes in the image in the small area S, the optimal area determination unit 78 determines the small area S as a non-optimal area. When the small area S is determined as a non-optimal area, the NR execution determination unit 79 determines that the target pixel data should not be subjected to NR processing, and notifies the NR processing unit 75 and the output unit 76 to that effect (S120Y). ). Thereby, the target pixel data that has not been subjected to NR processing is output from the output unit 76. The R component or the B component does not contribute to the resolution as much as the G component, and the image quality can be maintained without determining the optimum area after S120Y. For this reason, when the target pixel data is the R component or the B component, a simple process as shown in FIG. 11 is sufficient.

  According to the processing as shown in FIG. 11, since the determination as to whether or not the large area L is the optimum area is prioritized, false color and color noise are preferentially removed from the low-luminance portion. Can lead to improved image quality. Further, the algorithm can be simplified as compared with the case where the target pixel data is the G component.

  In S100 and S120 of FIG. 11, the comparison of the variance of the G component and the threshold value E is not necessarily indispensable, as described above, in order to facilitate image discrimination. When the comparison of the variance of the G component and the threshold E is not performed, the process proceeds to S100N when the variance of the R component or the B component of the large area L is less than the threshold D in S100 of FIG. When the variance of the R component or B component is equal to or greater than the threshold value D, the process may be shifted to S120. In S120 of FIG. 11, when the dispersion of the R component or B component in the small area S is less than the threshold D, the process proceeds to S120N, and when the dispersion of the R component or B component in the small area S is greater than or equal to the threshold D. The process may be shifted to S120Y.

The figure which shows the principal part schematic structure of the digital camera for describing embodiment of this invention The figure which shows the example of an arrangement | sequence of the photoelectric conversion element of the solid-state image sensor shown in FIG. The figure which shows the modification of the principal part schematic structure of the digital camera for describing embodiment of this invention 2 is an image diagram in which pixel data supplied from the pixel data holding unit shown in FIG. 1 to the NR device is two-dimensionally arranged. The block diagram which shows schematic structure of the NR apparatus shown in FIG. Diagram showing deviation histogram The figure which shows an example of the integration | accumulation path | route of the pixel data of G component when the small area S is 5 rows x 5 columns, the large area L is 13 rows x 13 columns, and the attention pixel data is G component. The figure which shows the image of the processing which reduces the variance of the optimal area The figure which shows the operation | movement flow of the noise reduction apparatus shown in FIG. The figure which shows the image at the time of operation | movement of the noise reduction apparatus shown in FIG. The figure which shows the operation | movement flow of a noise reduction apparatus in case attention pixel data is R component or B component.

Explanation of symbols

DESCRIPTION OF SYMBOLS 1 Solid-state image sensor 2 A / D conversion part 3 Buffer 4 Preprocessing circuit 5 TG
6 Pixel data holding unit 7 Noise reduction device 8 Signal processing unit 71 Average calculating unit 72 Mean square calculating unit 73 Discriminating circuit 74 Memory 75 Noise reduction processing unit 76 Output unit 77 Variance calculating unit 78 Optimal area determining unit 79 Noise reduction execution determining unit 80 Noise amount estimation unit

Claims (6)

A noise reduction device that performs noise reduction of RAW image data obtained from a solid-state imaging device,
The same color as the target pixel data included in the area for each of a plurality of areas including at least a large area having a large size and a small area having a size smaller than the large area, including the target pixel data of the RAW image data Average value calculating means for calculating an average value of component pixel data (hereinafter referred to as area same color pixel data);
Based on the average value, for each of the plurality of areas, a variance calculation unit that calculates a variance of the area same color pixel data;
A noise amount estimating means for estimating a noise amount included in the target pixel data;
Based on the variance for each of the plurality of areas, an optimal area determination unit that determines an optimal area that is optimal for use in noise reduction processing of the pixel-of-interest data among the plurality of areas;
Noise comprising noise reduction processing means for performing a process of bringing the target pixel data closer to the average value calculated for the optimum area in accordance with the estimated noise amount and performing a noise reduction process on the target pixel data Reduction device. The noise reduction device according to claim 1,
It is a threshold value that is a determination criterion for determining the optimum area, and a threshold value that is determined according to the amount of noise is a first threshold value,
The optimal area determination unit is a noise reduction device that determines the optimal area using the variance for each of the plurality of areas, the average value for each of the plurality of areas, and the first threshold value. The noise reduction device according to claim 1 or 2,
A threshold value used as a criterion for determining whether or not to perform noise reduction processing on the pixel-of-interest data using the optimum area, and a threshold value determined according to the amount of noise is a second threshold value,
Using the variance calculated for the optimal area and the second threshold, noise reduction execution determining means for determining whether to perform noise reduction on the pixel-of-interest data using the optimal area;
A noise reduction device comprising: output means for outputting either the target pixel data or the target pixel data after the noise reduction process according to a determination result by the noise reduction execution determination means. The noise reduction device according to claim 3,
The raw image data includes pixel data of a red (R) component, a green (G) component, and a blue (B) component,
When the target pixel data is the R component or the B component,
The optimum area determining means determines whether or not the large area is the optimum area based on a dispersion of pixel data of the R component or the B component included in the large area and the first threshold value. When it is determined that the large area is not the optimum area, the small area is determined based on the dispersion of the R component or B component pixel data included in the small area and the first threshold value. Determine whether it is the optimal area,
The noise reduction execution determination unit determines that noise reduction should be performed on the target pixel data when the large area or the small area is determined as the optimal area, and the small area is not the optimal area. A noise reduction device that, when determined, determines that noise reduction should not be performed on the pixel-of-interest data. The noise reduction device according to any one of claims 1 to 4,
The noise amount estimation means is based on data corresponding to a noise amount included in pixel data of each gradation, which is obtained in advance from RAW image data obtained by imaging a gradation chart with the solid-state imaging device. A noise reduction device that estimates the amount of noise contained in pixel data. A noise reduction method for performing noise reduction of RAW image data obtained from a solid-state imaging device,
The same color as the target pixel data included in the area for each of a plurality of areas including at least a large area having a large size and a small area having a size smaller than the large area, including the target pixel data of the RAW image data An average value calculating step for calculating an average value of component pixel data (hereinafter referred to as area same color pixel data);
A variance calculating step for calculating a variance of the same color pixel data for each of the plurality of areas based on the average value;
A noise amount estimating step for estimating a noise amount included in the target pixel data;
Based on the variance for each of the plurality of areas, an optimal area determination step of determining an optimal area that is the most suitable area to use for noise reduction processing of the pixel-of-interest data among the plurality of areas;
A noise reduction processing step of performing a process of bringing the target pixel data closer to the average value calculated for the optimal area according to the estimated noise amount, and performing a noise reduction process of the target pixel data Reduction method.

Images