JP2003101815A - Signal processor and method for processing signal - Google Patents

Abstract

PROBLEM TO BE SOLVED: To obtain a signal processor and a signal processing method capable of effectively reducing noise in an area from a low luminance part up to a high luminance part. SOLUTION: A noise reduction part 30 for applying noise reduction processing to a digital signal is arranged between an A/D converter 20 for converting an analog signal (picture signal) obtained by image pickup using an image pickup element 16 into a digital signal and a signal processing part 32 for applying various kinds of signal processing other than the noise reduction processing to the digital signal.

Description

Detailed Description of the Invention

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to a signal processing device and a signal processing method, and more particularly to a signal processing device and a signal processing method for performing noise reduction processing on a signal output from an image sensor.

[0002]

2. Description of the Related Art In recent years, CCD (Charge Coupled Devic
e), CMOS (Complementary Metal Oxide Semicondu)
Demand for digital cameras such as digital still cameras and digital video cameras is rapidly increasing with the increase in resolution of image sensors such as image sensors.

By the way, in such a digital camera, in order to improve the quality of an image obtained by photographing, an image signal representing a subject image output from an image pickup device is subjected to analog / digital conversion (hereinafter referred to as "A / D conversion "
Say. ) To obtain a digital signal, various digital signal processing such as γ correction processing, YC conversion processing, and edge enhancement processing is performed on the digital signal.

However, in the image signal obtained by the image pickup by the image pickup device, white noise such as shot noise and thermal noise generated in the image pickup device and its periphery,
There is a problem that noise such as jump-in noise inside and outside the system may be included, and an S / N ratio (Signal to Noise Ratio) of an image (subject image) obtained based on the image signal may be deteriorated. there were.

In order to solve this problem, conventionally, after converting an analog signal (image signal) output from an image pickup device into a digital signal and performing various signal processing such as γ correction processing on the digital signal, Noise reduction processing was performed (see Japanese Patent Laid-Open No. 2000-354180).

That is, in the conventional noise reduction technique, as shown in FIG. 11 as an example, the subject image formed on the image pickup device 106 constituted by the CCD after passing through the lens 102 and the optical low-pass filter 104 is picked up by the image pickup device. 10
It is photoelectrically converted by 6 to be an image signal. Then, after the image signal is amplified by the amplifier 108, analog /
The signals are converted into digital signals by the digital converter 110, and further, signals for each color of R (red), G (green), B (blue) (hereinafter, “R signal”, “G signal”, “ After being separated into “B signals”), the R, G, and B signals are subjected to various signal processing such as γ correction processing in the signal processing unit 114. After that, the noise reduction unit 1 applies to the digital signal processed by the signal processing unit 114.
Noise reduction processing was performed in 16.

[0007]

However, in the above-mentioned conventional technique, the noise reduction processing is performed on the image signal after various signal processing such as γ correction processing is performed. Changes in properties, one-dimensional or two-dimensional for still images, 3 for moving images
There is a problem in that the noise reduction effect may not be exerted in some cases because of the dimensional spread. Hereinafter, this point will be specifically described with reference to the drawings. In addition,
Here, a luminance signal (hereinafter, referred to as “Y signal”) is generated based on each of the R, G, and B signals, and the signal processing unit 114 in FIG. 11 generates a low pass filter (Low Pass Filter) for the Y signal. ) A case of performing processing will be described.

As shown in FIG. 12, as an example, there is a scratch at the image pickup position corresponding to the pixel in the second row and third column (pixel surrounded by ◯ in the same figure) in the image pickup area of 5 × 5 pixels of the image pickup device. Yes, the Y signal value corresponding to a pixel having no flaw in the row including this pixel (the row surrounded by the broken line in the figure) is “1”.
And assuming that the Y signal value corresponding to the defective pixel is “0”, the Y signal value of each pixel in this row is 1:
When a 2: 1 one-dimensional low-pass filter is applied, the Y signal values in this row are '1', '3 /
It becomes 4 ',' 1/2 ',' 3/4 ', and'1'.

In other words, in this case, although the flaw of the image pickup device is only in the one-pixel region, the influence of the flaw has spread to three-dimensional one-dimensional pixels. Similarly, when the processing using the two-dimensional low-pass filter is performed, the noise spreads two-dimensionally, and when the processing using the three-dimensional low-pass filter is performed, the noise spreads three-dimensionally.

As described above, even if the noise reduction processing is performed on the image signal in which the influence of noise spreads in the direction according to the dimensionality of the filter, the noise reduction effect may not be exhibited.

On the other hand, in the above-mentioned conventional technique, noise is magnified in the low-luminance portion and reduced in the high-luminance portion due to the influence of the non-linear signal processing such as γ correction processing, so that the noise from the low-luminance portion is increased. There is also a problem that a uniform noise reduction effect cannot be obtained up to the luminance part. Hereinafter, this point will be specifically described with reference to the drawings. In addition, here
The case where the noise reduction process is performed after the γ correction process is described as an example.

As an example, as shown in FIG. 13, when the correction processing is linearly performed, the same input change amount ('a' in FIG.
When the noise reduction process is performed on the image signal after the correction process, since the output change amount (corresponding to It is possible to obtain a constant level of noise reduction effect from the low brightness portion to the high brightness portion. On the other hand, since the γ correction process is performed non-linearly, the output change amount (corresponding to'c 'and'd' in the figure) for the same input change amount is large in the low luminance part and the high luminance part. It will be different. Therefore, when the noise reduction processing is performed on the image signal after the γ correction processing, the noise reduction effect varies depending on the level of the brightness.

The present invention has been made to solve the above problems, and provides a signal processing apparatus and a signal processing method capable of effectively reducing noise from a low luminance portion to a high luminance portion. The purpose is to

[0014]

In order to achieve the above object, a signal processing apparatus according to a first aspect of the present invention comprises an A / D conversion means for converting an analog signal output from an image pickup device into a digital signal, and the digital signal. Noise reduction processing means for performing noise reduction processing for reducing noise on a signal, and other processing means for performing signal processing other than the noise reduction processing on a digital signal subjected to noise reduction processing by the noise reduction processing means And are equipped with.

According to the signal processing device of the first aspect,
The analog signal output from the image sensor is converted into a digital signal by the A / D conversion means, and noise reduction processing for reducing noise is performed on the digital signal by the noise reduction processing means, and the noise reduction processing is performed. The signal processing other than the noise reduction processing is performed on the digital signal by other processing means. The signal processing other than the noise reduction processing includes γ correction processing, YC conversion processing for converting each R, G, and B signal into a luminance signal and a color difference signal, edge enhancement processing, and the like. Further, the image pickup device includes a C
It includes a solid-state image sensor such as a CD or a CMOS image sensor.

That is, in the present invention, the noise reduction processing is performed on the digital signal converted by the A / D conversion means before the signal processing other than the noise reduction processing is performed. The noise reduction processing can be performed at the same level, and as a result, noise can be effectively reduced from the low luminance portion to the high luminance portion.

As described above, according to the signal processing device of the first aspect, the analog signal output from the image pickup device is converted into a digital signal, and the noise reduction process for reducing noise is performed on the digital signal. Since signal processing other than the noise reduction processing is performed on the digital signal subjected to the noise reduction processing, noise can be effectively reduced from the low luminance portion to the high luminance portion.

Further, according to the present invention, since the noise reduction processing is performed on the digital signal obtained by the A / D conversion means, the noise reduction processing is directly performed on the analog signal output from the image pickup device. The processing can be performed more easily than when it is performed.

By the way, as signal processing for the image signal output from the image pickup device, a black level correction process (so-called dark correction process) for reducing the influence of dark current generated in the image pickup device, and color balance adjustment. There is a gain correction process (so-called white balance correction process) for performing the above. However, these correction processes do not change the nature of noise, but rather are processes for making the image signal proper.

Therefore, the signal processing apparatus according to claim 2 is
The invention according to claim 1, further comprising correction processing means for performing at least one correction processing of a black level correction processing and a gain correction processing on the digital signal converted by the A / D conversion means, and the noise reduction processing. The means performs the noise reduction processing on the digital signal corrected by the correction processing means.

According to the signal processing device of the second aspect,
The correction processing means performs at least one correction processing of the black level correction processing and the gain correction processing on the digital signal converted by the A / D conversion means of the present invention, and the noise reduction processing means performs the correction processing means. The noise reduction process is performed on the digital signal that has been subjected to the correction process.

As described above, according to the signal processing apparatus of the second aspect, at least one of the black level correction processing and the gain correction processing is performed on the digital signal,
Since noise reduction processing is performed on the corrected digital signal, noise reduction processing can be performed on the optimized digital signal, and noise can be reduced more effectively. .

On the other hand, as typical filters capable of reducing signal noise, there are linear filters such as recursive filters and non-recursive filters, and non-linear filters such as median filters and Wiener filters. Has advantages and disadvantages depending on the state of noise or the state of an image to be processed.

Therefore, the signal processing apparatus according to claim 3 is
In the invention of claim 1 or 2, the noise reduction processing means performs the noise reduction processing based on at least one of a linear filter and a non-linear filter.

As described above, according to the signal processing device of the third aspect, since the noise reduction processing is performed based on at least one of the linear filter and the non-linear filter,
It is possible to perform noise reduction processing based on a suitable filter according to the state of the processing target, and as a result, noise can be reduced more effectively.

By the way, the non-linear filter can effectively reduce the noise for the edge of the image (the boundary portion of the color of the image) indicated by the image signal output from the image sensor, as compared with the linear filter. On the contrary, the linear filter can effectively reduce the noise with respect to the non-edge of the image as compared with the non-linear filter.

For example, when a low-pass filter, which is a kind of linear filter, is used to uniformly apply a low-pass filter to an input image signal showing an image having flat portions and edge portions shown in FIG. 10, an output image signal As shown in the figure, the noise generated in the flat part (the part surrounded by ◯ in the broken line in the figure) can be effectively removed, but the noise reduction effect is too strong at the edge part and the edge sharpness is sharpened. Will be reduced. On the other hand, when a median filter, which is a type of non-linear filter, is used, the noise reduction effect of the filter is weaker than that of a low-pass filter, and thus noise can be reduced without impairing the sharpness of edges.

Therefore, the signal processing apparatus according to claim 4 is
The invention according to claim 3, further comprising edge detection means for detecting an edge of an image indicated by the digital signal converted by the A / D conversion means, wherein the noise reduction processing means is detected by the edge detection means. The digital signal corresponding to the edge is subjected to the noise reduction processing based on the nonlinear filter, and the other digital signals are subjected to the noise reduction processing based on the linear filter.

According to the signal processing device of the fourth aspect,
The edge detection means detects the edge of the image represented by the digital signal converted by the A / D conversion means,
The noise reduction processing unit performs noise reduction processing based on the nonlinear filter on the digital signal corresponding to the edge detected by the edge detection unit, and performs noise reduction processing based on the linear filter on other digital signals. .

As described above, according to the signal processing device of the fourth aspect, the edge of the image indicated by the digital signal is detected, and the noise reduction based on the nonlinear filter is performed on the digital signal corresponding to the detected edge. Noise reduction processing based on a linear filter is performed for other digital signals, so noise is effectively reduced for digital signals corresponding to both the edge and non-edge regions of the image. can do.

On the other hand, when the image signal is divided into the two regions of the edge and the non-edge in this way and the noise reduction process is performed using a suitable filter for each region, the image indicated by the image signal after the noise reduction process is performed. In the boundary area between the edge and the non-edge, processing results obtained by different filters are adjacent to each other, so that the boundary area may be uncomfortable.

In view of this point, in the signal processing apparatus according to the fifth aspect of the present invention, in the invention according to the third aspect, the edge detection for detecting the edge of the image indicated by the digital signal converted by the A / D conversion means. The noise reduction processing means further comprises means for processing the digital signal processing result of the non-linear filter for each pixel and the linear filter processing result of the digital signal to be processed by the edge detection means. The noise reduction process is performed by increasing the mixing ratio of the processing result of the non-linear filter and mixing as the detected edge approaches.

According to the signal processing device of the fifth aspect,
The edge detection means detects the edge of the image represented by the digital signal converted by the A / D conversion means,
By the noise reduction processing means, the processing result of the non-linear filter for each pixel and the processing result of the linear filter with respect to the digital signal are nonlinear filters as the pixel position of the digital signal to be processed approaches the edge detected by the edge detection means. The noise reduction processing is performed by increasing the mixing ratio of the processing result by and mixing.

As a specific example of the noise reduction processing at this time, a method of deriving the filtering result R for each pixel by applying the following expression (1) can be exemplified.

R = ki × LNR + kj × NNR (1) Here, LNR is the filtering result by the linear filter, ki is the mixing coefficient of the filtering result, N
NR is the filtering result by the nonlinear filter, k
j represents the mixing coefficient of the filtering result, and the mixing coefficients ki and kj are determined so that ki + kj = 1.

That is, in this case, the mixing coefficients ki and kj may be determined so that the mixing coefficient kj increases as the edge detected by the edge detecting means approaches.

As described above, according to the signal processing device of the fifth aspect, the edge of the image represented by the digital signal is detected, and the processing result by the non-linear filter and the processing result by the linear filter for each pixel with respect to the digital signal. Since noise reduction processing is performed by increasing the mixing ratio of the processing result of the nonlinear filter as the pixel position of the digital signal to be processed approaches the detected edge, edge and non-edge It is possible to obtain an image signal capable of reproducing an image without a sense of discomfort in a boundary area between and.

By the way, when the image pickup device for picking up an image of a subject in color has a single plate structure, a color filter may be intermittently arranged for pixels of at least one color of R, G and B. For example, when the color filters of the image sensor are arranged in a Bayer array, as shown in FIG.
Although G is arranged in each row and each column, R and B are arranged intermittently in every two rows or every two columns. Therefore, compared with the image signal corresponding to G, the image signals corresponding to R and B are intermittently output from this image sensor in time series, and the image supplied in such a state. Even if an edge is detected using a signal, it is not possible to perform highly accurate detection.

In view of this point, in the signal processing device according to claim 6, in the invention according to claim 4 or 5, the luminance signal is generated based on the digital signal converted by the A / D conversion means. A luminance signal generating means is further provided, and the edge detecting means detects the edge based on the luminance signal.

According to the signal processing device of the sixth aspect,
The brightness signal generation means generates a brightness signal based on the digital signal converted by the A / D conversion means, and the edge detection means detects the edge based on the brightness signal.

That is, in the present invention, the edge detection is performed by applying a luminance signal which is a signal generated for all pixels based on each signal of R, G and B, and which does not generate an intermittent portion in the pixel. The edge is detected with high accuracy.

As described above, according to the signal processing device of the sixth aspect, since the luminance signal is generated based on the digital signal and the edge is detected based on the luminance signal, the edge detection accuracy is improved. As a result, noise can be reduced more effectively.

By the way, since the sensitivities of R and B are generally lower than G, the correction gain for R and B becomes larger than G, which often causes deterioration of the S / N ratio. In this case, it may be possible to improve the image quality as a result by performing the noise reduction processing on only the R and B signals instead of performing it on all the R, G, and B signals. Similarly, depending on the state of the subject image and the state of noise, R,
In some cases, it may be preferable to perform the noise reduction processing on only one signal of G and B.

Therefore, the signal processing apparatus according to claim 7 is
The invention according to any one of claims 1 to 6, wherein the image pickup device separates a subject into a plurality of colors to capture an image, and the noise reduction processing unit includes at least one of the plurality of colors. The noise reduction processing is selectively performed for each color.

According to the signal processor of claim 7,
The image pickup device according to the present invention is configured to separate an image of a subject into a plurality of colors and pick up an image, and the noise reduction processing unit selectively performs noise reduction processing on at least one of the plurality of colors. In addition, R, G, and B are included in the plurality of colors.
In addition to the three primary colors, Y (yellow) and C, which are complementary colors to these
(Cyan) and M (magenta) are included.

As described above, according to the signal processing apparatus of the seventh aspect, the image pickup device according to the present invention separates an image of a subject into a plurality of colors and picks up an image, and at least one of the plurality of colors is picked up. Since the noise reduction process is selectively performed on the noise reduction process, the noise reduction process can be performed only on the color in which noise is likely to occur, and as a result, the noise can be effectively reduced.

On the other hand, in order to achieve the above object, the signal processing method according to the eighth aspect is a noise reduction for converting an analog signal output from an image pickup device into a digital signal and reducing noise with respect to the digital signal. Processing is performed, and signal processing other than the noise reduction processing is performed on the digital signal subjected to the noise reduction processing.

Therefore, according to the signal processing method of the eighth aspect, since it operates in the same manner as the invention of the first aspect, from the low luminance portion to the high luminance portion, as in the invention of the first aspect. Can effectively reduce the noise.

According to a ninth aspect of the present invention, in the signal processing method according to the eighth aspect, at least one of black level correction processing and gain correction processing is performed on the digital signal, and the correction processing is performed. The noise reduction processing is performed on the performed digital signal.

Therefore, the signal processing method according to the ninth aspect operates in the same manner as the invention according to the second aspect. Therefore, as with the invention according to the second aspect, the optimized digital signal is processed. Noise reduction processing can be performed, and noise can be reduced more effectively.

The signal processing method according to claim 10 is:
In the invention of claim 8 or claim 9, the noise reduction processing is performed based on at least one of a linear filter and a non-linear filter.

Therefore, according to the signal processing method of the tenth aspect, it operates in the same manner as the invention of the third aspect.
Similar to the invention described in claim 3, it is possible to perform noise reduction processing based on a suitable filter according to the state of the processing target, and as a result, noise can be reduced more effectively.

The signal processing method according to claim 11 is
11. The invention according to claim 10, wherein an edge of an image indicated by the digital signal is detected, noise reduction processing based on the nonlinear filter is performed on the digital signal corresponding to the detected edge, and the digital signal corresponding to another digital signal is detected. Noise reduction processing based on the linear filter.

Therefore, according to the signal processing method of the eleventh aspect, it operates in the same manner as the invention of the fourth aspect.
Similarly to the invention described in claim 4, noise can be effectively reduced for digital signals corresponding to both the edge and non-edge regions of an image.

The signal processing method according to claim 12 is:
11. The digital signal according to claim 10, wherein an edge of an image represented by the digital signal is detected, and a processing result of the nonlinear filter for each pixel with respect to the digital signal and a processing result of the linear filter are processed. The noise reduction processing is performed by increasing the mixing ratio of the processing result by the non-linear filter and performing mixing as the pixel position approaches the detected edge.

Therefore, the signal processing method according to the twelfth aspect operates in the same manner as the invention according to the fifth aspect.
Similar to the fifth aspect of the invention, it is possible to obtain an image signal capable of reproducing an image without a sense of discomfort in a boundary region between an edge and a non-edge.

The signal processing method according to claim 13 is
In the invention of claim 11 or 12, a brightness signal is generated based on the digital signal, and the edge is detected based on the brightness signal.

Therefore, according to the signal processing method of the thirteenth aspect, it operates in the same manner as the invention of the sixth aspect.
Similar to the sixth aspect of the invention, the edge detection accuracy can be improved, and as a result, noise can be reduced more effectively.

Further, the signal processing method according to claim 14 is:
The invention according to any one of claims 8 to 13, wherein the image pickup device separates an image of a subject into a plurality of colors to pick up an image, and the noise is selectively applied to at least one of the plurality of colors. The reduction process is performed.

Therefore, according to the signal processing method of the fourteenth aspect, since it operates in the same manner as the invention of the seventh aspect,
Similar to the invention described in claim 7, the noise reduction process can be performed only for the color in which noise is likely to occur, and as a result, the noise can be effectively reduced.

[0061]

BEST MODE FOR CARRYING OUT THE INVENTION Embodiments of the present invention will be described in detail below with reference to the drawings. In addition, here
A case where the present invention is applied to a digital camera will be described.

[First Embodiment] First, referring to FIG.
The configuration of the digital camera 10 according to this embodiment will be described.

As shown in the figure, the digital camera 10 according to the present embodiment is generated when a lens 12 for forming a subject image and an R, G, B sampling by an image sensor 16 described later are performed. An optical low-pass filter (hereinafter referred to as “optical LPF”) 14 arranged on the optical axis of the lens 12 for reducing frequency interference (so-called aliasing), and arranged behind the lens 12 on the optical axis. Image pickup device 16, an image signal processing unit 10A that performs various kinds of signal processing on the output signal from the image pickup device 16, and a system control unit 50 that controls the overall operation of the digital camera 10.
And a mode changeover switch 60 used for selecting a color targeted for noise reduction processing.

The image signal processing unit 10A has one chip L.
The digital camera 10 is configured as an SI (Large Scale Integrated circuit), which is intended to reduce the size and weight of the digital camera 10, improve reliability, and reduce cost.

On the other hand, the image signal processing unit 10A has an amplifier 18 for amplifying and outputting an input analog signal, and an analog / digital converter (hereinafter, referred to as "the analog signal") for converting the input analog signal into a digital signal and outputting it. A / D converter ") 20 and the input digital signal into R, G, B
The RGB separation unit 22 that separates and outputs each color, the noise reduction unit 30 that performs noise reduction processing on the input digital signal, and the noise reduction processing other than the noise reduction processing on the input digital signal. And a signal processing unit 32 that performs signal processing and outputs.

An output end of the image pickup device 16 (an output end for outputting an analog signal indicating a subject image obtained by image pickup) is connected to an input end of an RGB separation section 22 via an amplifier 18 and an A / D converter 20 in this order. Has been done. Therefore, the RGB separation unit 22 is sequentially output from the image pickup device 16, and the amplifier 1
After being amplified by 8, the image signal converted into a digital signal by the A / D converter 20 is sequentially separated for each color of R, G, B and output in time series.

The output end of the RGB separation unit 22 is connected to the input end of the noise reduction unit 30, and the output end of the noise reduction unit 30 is connected to the input end of the signal processing unit 32. Therefore, the noise reduction unit 30
Noise reduction processing is performed on each of the R, G, and B digital signals output from the GB separation unit 22 as necessary, and the signal processing unit 32 outputs each of the R, G, and B output from the noise reduction unit 30. If necessary, the digital signal is subjected to signal processing other than noise reduction processing and output.

The signal processing performed by the signal processing unit 32 includes γ correction processing, YC conversion processing, edge enhancement processing, black level correction processing, and RGB gain correction processing. Therefore, the output signal from the signal processing unit 32 is
The luminance signal (Y signal) and the color difference signal (R) for each of the R, G, and B signals and when the YC conversion process is performed.
-Y signal and BY signal).

Here, the system control unit 50 is connected to each component of the image signal processing unit 10A, and the operation of each component of the image signal processing unit 10A is controlled by the system control unit 50. Further, a mode changeover switch 60 provided outside the digital camera 10 is connected to the system control unit 50, and the system control unit 50 allows the mode changeover switch 60 operated by the user of the digital camera 10.
The switching setting status of is always known.

In the present embodiment, R, G, and B are set as modes that can be set by the mode changeover switch 60.
There are two types of modes, namely, a first mode in which noise reduction processing is performed on all signals in 1) and a second mode in which noise reduction processing is performed on only R and B signals. Mode switch 60,
Only one of these modes can be set.

The image pickup device 16 in the present embodiment is composed of a CCD in which color filters of R, G and B colors are provided in a predetermined arrangement on the light receiving surface of each light receiving device. Here, as the above-mentioned predetermined array, as shown in FIG.
Bayer array shown in Fig. 2, G stripe R shown in Fig. 2B
The / B perfect checkered array and the honeycomb array shown in FIG. 2C can be exemplified. In each of these color filter arrays, the number of G color filters is twice as large as that of R and B, and the image sensor 16 outputs the R and B analog signals in chronological order. It is output intermittently.

On the other hand, FIG. 3 shows the configuration of the noise reduction section 30 according to the present embodiment. As shown in the figure, the noise reduction unit 30 according to the present embodiment uses an interpolation unit 30A that interpolates and outputs the missing pixel values of the R signal and the B signal, and a signal for performing noise reduction processing. It is configured to include a switching unit 30B and a filter unit 30C that actually performs noise reduction processing.

The input end of the interpolation unit 30A is the RGB separation unit 22.
To the input end of the switching unit 30B, the output end of the switching unit 30B to either one of the input end and the output end of the filter unit 30C, and the output end of the filter unit 30C to the signal end. Each is connected to the input end of the processing unit 32.

The switching unit 30B receives the R, G, and B signals after the interpolation processing, which are input from the interpolation unit 30A, from the filtering unit 3B.
0C for noise reduction processing or bypassing the filter unit 30C for output without noise reduction processing is switched, and the switching control is performed by the system control unit 50. The switching unit 3
0B is configured so that it can be switched independently for each of R, G, and B colors.

On the other hand, FIG. 4 shows a structural example of a filter provided in the filter section 30C for performing the noise reduction processing.

The filter shown in FIG. 9A is a configuration example of a one-dimensional acyclic filter which is a kind of linear filter, and four delay elements D refer to an input signal for five horizontal pixels, Noise reduction of the pixel located at the center of the 5 pixels by multiplying the signal values of the 5 pixels by the coefficients C ₀ , C ₁ , C ₂ , C ₃ and C ₄ and then adding these values. The latter signal value is output.

Here, in order to make the filter function as a low-pass filter, the coefficients C ₀ , C ₁ , C ₂ , C ₃ and C _{4 are used.}
Is determined so that C ₀ + C ₁ + C ₂ + C ₃ + C ₄ = 1. As an example, the coefficient C ₀ = 0.2, the coefficient C ₁ = 0.2,
Coefficient C ₂ = 0.2, coefficient C ₃ = 0.2, coefficient C ₄ = 0.2
Then, the signal values of the central pixel are uniformly averaged, and the coefficient C
_{If 0} = 0.1, coefficient C ₁ = 0.2, coefficient C ₂ = 0.4, coefficient C ₃ = 0.2, coefficient C ₄ = 0.1, the signal value of the central pixel is Pixels closer to are obtained as weighted.

When the one-dimensional acyclic filter is provided in the filter section 30C, one for each of the R, G, and B signals input from the switching section 30B, one for each.
There will be a total of three.

On the other hand, the filter shown in FIG. 9B is a configuration example of a one-dimensional median filter which is a kind of non-linear filter, and the four delay elements D refer to the input signals of five horizontal pixels, The median value determination circuit detects the median value among the signal values of the five pixels and outputs the signal value after noise reduction of the pixel located at the center of the five pixels.

When the one-dimensional median filter is provided in the filter section 30C, one is provided for each of the R, G, and B signals input from the switching section 30B, for a total of three.

The A / D converter 20 serves as the A / D conversion means of the present invention, the noise reduction section 30 serves as the noise reduction processing means of the present invention, and the signal processing section 32 serves as the other processing means of the present invention.
Equivalent to each.

Next, the operation of the digital camera 10 configured as above will be described. The user of the digital camera 10 uses the mode switch 60 to select a desired mode (the first mode or the second mode in the present embodiment).
Mode) is set in advance.

First, the system control unit 50 detects the setting state of the mode changeover switch 60, and when the first mode is set, all the R, G, B signals are input to the filter unit 30C. By controlling the switching unit 30B, the second
When the mode is set, the R signal and the B signal are input to the filter unit 30C, and the G signal is switched to bypass the filter unit 30C.
Control B.

Under this condition, in the image pickup device 16, the lens 1
2 and the object image represented by the light incident through the optical LPF 14 is captured, and the R,
Each of the G and B analog image signals is output to the RGB separation unit 22 through the amplifier 18 and the A / D converter 20 in time series and continuously.

In the RGB separation unit 22, the image signal amplified by the amplifier 18 and converted into a digital signal by the A / D converter 20 is separated for each color of R, G, B and interpolated by the noise reduction unit 30. It is sequentially output to the unit 30A.

In the interpolator 30A, the missing portions of the R and B signals are interpolated and sequentially output to the switching unit 30B, and the G signals are sequentially output to the switching unit 30B without being interpolated.

In the switching unit 30B, when the first mode is set by the user, all the R, G, and B signals are sequentially output to the corresponding filters for noise reduction processing provided in the filter unit 30C. When the second mode is set by the user, the R signal and the B signal are sequentially output to the corresponding filter for noise reduction processing provided in the filter unit 30C, and the G signal is output to the filter unit 30C. The signal is sequentially output to the output terminal of the filter unit 30C that bypasses and outputs the G signal.

Then, in the filter section 30C, the switching section 3
The image signals input from 0B are sequentially filtered by the corresponding filters.

Through the above processing, when the first mode is set by the user, the noise reduction section 30 sequentially outputs the R, G, and B signals subjected to the noise reduction processing to the signal processing section 32. When the second mode is set by the user, the R and B signals that have been subjected to the noise reduction processing and the G signal that has not been subjected to the noise reduction processing are sequentially output to the signal processing unit 32.

In the signal processing unit 32, the R, G and B signals input from the noise reduction unit 30 are subjected to predetermined signal processing and output. It should be noted that R, G, and B after various signal processing output from the signal processing unit 32 are processed.
Are used for display on a liquid crystal display (not shown) or an electronic viewfinder that functions as a finder of the digital camera 10, or when a release button (so-called shutter) (not shown) is pressed by the user. It is also used as an image signal for recording.

As described above in detail, in the image signal processing unit 10A and the signal processing method according to the first embodiment,
The analog signal output from the image sensor 16 is converted into a digital signal, noise reduction processing for reducing noise is performed on the digital signal, and the digital signal subjected to the noise reduction processing is other than the noise reduction processing. Since the signal processing of (1) is performed, noise can be effectively reduced from the low luminance portion to the high luminance portion.

Further, in the image signal processing unit 10A and the signal processing method according to the first embodiment, the image pickup device according to the present invention uses the image pickup device according to the present invention in a plurality of colors (in the present embodiment, three colors of R, G and B). ), And picks up an image, and at least one of the plurality of colors (in the present embodiment, 3 of R, G, and B) is used.
Since the noise reduction processing is selectively performed on the color or the two colors R and B), the noise reduction processing can be performed only on the color in which noise is likely to occur, and as a result, the noise is effectively reduced. be able to.

Second Embodiment In the second embodiment, an example of a case in which noise reduction processing is performed on an image signal after each processing of black level correction processing and gain correction processing will be described. . First, the configuration of the digital camera 10 according to the second embodiment will be described with reference to FIG. The same components as those of the digital camera 10 according to the first embodiment shown in FIG. 1 shown in FIG. 1 are assigned the same reference numerals as those of FIG. 1 and their explanations are omitted.

As shown in the figure, in the digital camera 10 according to the second embodiment, the processing for reducing the influence of the dark current generated in the image pickup device 16 between the RGB separation section 22 and the noise reduction section 30. The black level correction unit 24 that performs the black level correction process and the RGB gain correction unit 2 that performs the gain correction process that performs the color balance adjustment.
6 is provided, and instead of the signal processing unit 32, a signal processing unit 32 ′ in which two processes of a black level correction process and a gain correction process are removed is used. It is different from the digital camera 10 according to the first embodiment. The black level correction unit 24 and the RGB gain correction unit 26 correspond to the correction processing means of the present invention.

In the digital camera 10 according to the second embodiment having such a configuration, the R, G, and B signals separated by the RGB separation unit 22 are the same as in the digital camera 10 according to the first embodiment. In contrast, the black level correction unit 2
4, the black level correction process is performed, the RGB gain correction unit 26 performs the gain correction process, and the RGB gain correction unit 26 sequentially outputs the black level correction process to the noise reduction unit 30.

The operation of the parts other than the black level correction part 24 and the RGB gain correction part 26 is the same as that of the digital camera 10 according to the first embodiment, and therefore its explanation is omitted here.

As described in detail above, in the image signal processing unit 10B and the signal processing method according to the second embodiment,
The same effects as the image signal processing unit 10A and the signal processing method according to the first embodiment can be obtained, and both the black level correction processing and the gain correction processing are performed on the digital signal to perform the correction. Since noise reduction processing is performed on the processed digital signal,
Noise reduction processing can be performed on the optimized digital signal, and noise can be reduced more effectively.

In the present embodiment, a case has been described in which both the black level correction processing and the gain correction processing are performed on the digital signal, but the present invention is not limited to this, and black is not limited to this. It goes without saying that the correction processing of only one of the level correction processing and the gain correction processing may be performed.

[Third Embodiment] In the third embodiment, an example of a mode in which noise reduction processing is performed by different filters for an image signal corresponding to an edge of a subject image and an image signal corresponding to another portion will be described. To do. First, the configuration of the digital camera 10 according to the third embodiment will be described with reference to FIG. The same components as those of the digital camera 10 according to the first embodiment shown in FIG. 1 shown in FIG. 1 are assigned the same reference numerals as those of FIG. 1 and their explanations are omitted.

As shown in the figure, in the digital camera 10 according to the third embodiment, the input end is connected to the output end of the RGB separation section 22, and the output end is connected to the noise reduction section 30 'described later. Instead of the point where the edge determination unit 28 is newly added and the noise reduction unit 30,
The one-dimensional acyclic filter shown in FIG.
The difference from the digital camera 10 according to the first embodiment is only that a noise reduction unit 30 ′ including both of the one-dimensional median filter shown in (B) is used. The edge determination unit 28 corresponds to the edge detection means of the present invention.

The noise reduction unit 30 'according to the third embodiment is similar to the noise reduction unit 30 of the first embodiment (see also FIG. 3) in the interpolation unit 30A and the switching unit 3.
0B, and as the filter unit, one of the one-dimensional acyclic filter and the one-dimensional median filter is selectively applied to each of the R, G, and B signals according to the determination result by the edge determination unit 28. It is configured to be able to.

In the digital camera 10 according to the third embodiment having the above-described configuration, the R, G, and B signals separated by the RGB separation unit 22 are the same as in the digital camera 10 according to the first embodiment. Is the noise reduction section 3
0'and output to the edge determination unit 28.

In the edge determination unit 28, the RGB separation unit 22
It is determined whether or not the image corresponding to each of the R, G, and B signals input from corresponds to the edge of the subject image. If it is determined that the image corresponds to the edge, the noise reduction unit 3
For 0 ′, the filter used for the noise reduction processing is set to be a one-dimensional median filter, and when it is determined that the edge does not correspond to the edge, it is considered to correspond to the flat portion and the noise reduction unit 30 ′ is determined. On the other hand, the filter used for the noise reduction processing is set to be a one-dimensional acyclic filter.

As a result, the noise reduction unit 3
In 0 ′, the edge determination unit is applied to the image signal input from the RGB separation unit 22 and to the image signal selected by the switching unit 30B to be subjected to the noise reduction process by the interpolation unit 30A. The filtering using the one-dimensional median filter or one-dimensional acyclic filter set by 28 is performed.

Here, the determination as to whether or not the edge corresponds to the edge of the subject image, which is made by the edge determination unit 28, can be made by using an edge detection filter as shown in FIG. 7 as an example.

That is, for example, one of the R, G, and B signals input from the RGB separation section 22 has a signal value (..., 0, 0, 0, 1, 1, 1) as shown in FIG. 1, ...
.), When the edge detection filter shown in the figure is applied to the signal, the signal value corresponding to the edge (one half in the case of the figure) becomes a value equal to or larger than a predetermined value. Therefore, when the processing result of the edge detection filter for any one of the R, G, and B signals becomes a predetermined value or more, it can be determined that the image corresponding to the signal corresponds to the edge of the subject image. it can.

Since the other operations are the same as those of the digital camera 10 according to the first embodiment, the description thereof is omitted here.

As described in detail above, in the image signal processing unit 10C and the signal processing method according to the third embodiment,
The same effects as the image signal processing unit 10A and the signal processing method according to the first embodiment can be obtained, and a linear filter (a one-dimensional acyclic filter in this embodiment) and a non-linear filter (this embodiment) are provided. In the mode, since the noise reduction processing is performed based on each filter of the one-dimensional median filter), it is possible to perform the noise reduction processing based on a suitable filter according to the state of the processing target. As a result, Noise can be effectively reduced.

Further, in the image signal processing unit 10C and the signal processing method according to the third embodiment, the edge of the image represented by the digital signal is detected, and the digital signal corresponding to the detected edge is subjected to the nonlinear filter. Since the noise reduction process based on the linear filter is performed on the other digital signals, it is effective for the digital signals corresponding to both the edge and non-edge regions of the image. Noise can be reduced.

[Fourth Embodiment] In the fourth embodiment, an example of a mode in which the above-mentioned edge is detected based on a luminance signal will be described.

First, the structure of the digital camera 10 according to the fourth embodiment will be described with reference to FIG. The same components as those of the digital camera 10 according to the first embodiment shown in FIG. 1 shown in FIG. 1 are assigned the same reference numerals as those of FIG. 1 and their explanations are omitted.

As shown in the figure, in the digital camera 10 according to the fourth embodiment, the Y signal generating section 25 having the input terminal connected to the output terminal of the RGB separating section 22, and the Y signal generating section 2
A low-pass filter (hereinafter referred to as "LPF") 27 having an input end connected to the output end of 5, and an edge having an input end connected to the output end of the LPF 27 and an output end connected to the noise reduction unit 30 '. Instead of the point that the determination unit 28 is newly added and the noise reduction unit 30,
The one-dimensional acyclic filter shown in FIG.
The difference from the digital camera 10 according to the first embodiment is only that a noise reduction unit 30 ′ including both of the one-dimensional median filter shown in (B) is used. The edge determination section 28 serves as the edge detection means of the present invention, and the Y signal generation section 25 serves as the luminance signal generation means of the present invention.
Equivalent to each.

The noise reduction section 30 'according to the fourth embodiment is similar to the noise reduction section 30' according to the third embodiment, and the noise reduction section 30 according to the first embodiment (see also FIG. 3). ) Same as the interpolation unit 30
A and a switching unit 30B are provided, and one of a one-dimensional acyclic filter and a one-dimensional median filter is used as a filter unit for each of R, G, and B signals according to the determination result by the edge determination unit 28. It is configured to be selectively applicable.

In the digital camera 10 according to the fourth embodiment having such a configuration, the R, G, and B signals separated by the RGB separation unit 22 are the same as in the digital camera 10 according to the first embodiment. Is the noise reduction section 3
It is output to the 0 ′ and Y signal generation unit 25.

In the Y signal generating section 25, the RGB separating section 22
The Y signal is sequentially generated based on the signal value of each of the R, G, and B signals input from, and is output to the edge determination unit 28 via the LPF 27. The value Y of the Y signal can be calculated by the following equation (2), for example.

Y = 0.3R + 0.59G + 0.11B (2) Here, R represents the R signal value, G represents the G signal value, and B represents the B signal value.

In the edge judging section 28, based on the value Y of the Y signal which has been low-pass filtered by the LPF 27,
A method similar to that of the edge determination unit 28 according to the third embodiment (method using an edge detection filter (see also FIG. 7))
It is determined whether or not the image corresponding to the value Y of the Y signal corresponds to the edge of the subject image. If it is determined that the image corresponds to the edge, the noise reduction unit 30 ′ uses a filter for noise reduction processing. Is set to be a one-dimensional median filter, and when it is determined that it does not correspond to an edge, it is considered to correspond to a flat portion, and the noise reduction unit 30 ′ uses a one-dimensional filter for noise reduction processing. It is set to be a non-recursive filter.

As a result, the noise reduction unit 3
In 0 ′, the edge determination unit is applied to the image signal input from the RGB separation unit 22 and to the image signal selected by the switching unit 30B to be subjected to the noise reduction process by the interpolation unit 30A. The filtering using the one-dimensional median filter or one-dimensional acyclic filter set by 28 is performed.

Since the other operations are the same as those of the digital camera 10 according to the first embodiment, the description thereof is omitted here.

As described above in detail, in the image signal processing unit 10D and the signal processing method according to the fourth embodiment,
The same effects as those of the image signal processing unit 10C and the signal processing method according to the third embodiment can be obtained, and the luminance signal is generated based on the digital signal, and the edge is detected based on the luminance signal. Therefore, the edge detection accuracy can be improved, and as a result, noise can be reduced more effectively.

In the third and fourth embodiments, the noise reduction process based on the nonlinear filter is performed on the image signal corresponding to the edge of the image, and the noise reduction process based on the linear filter is performed on the other image signals. However, the present invention is not limited to this.
For example, the processing result of the nonlinear filter for each pixel with respect to the image signal and the processing result of the linear filter are mixed by increasing the mixing ratio of the processing result of the nonlinear filter as the pixel position of the image signal to be processed approaches an edge. By doing so, it is possible to adopt a mode in which noise reduction processing is performed.

In this case, the noise reduction section can perform the noise reduction process by the method of deriving the filtering result R for each pixel by applying the above-mentioned equation (1).

In this case, the edge of the image represented by the digital signal is detected, and the processing result of the nonlinear filter for each pixel and the processing result of the linear filter are applied to the digital signal to determine the pixel position of the digital signal to be processed. Since the noise reduction processing is performed by increasing the mixing ratio of the processing result by the non-linear filter as it approaches the detected edge and performing the noise reduction processing, an image that can reproduce an image with no discomfort in the boundary region between the edge and the non-edge You can get a signal.

Further, in the fourth embodiment, the value Y of the Y signal is
However, the present invention is not limited to this, and may be calculated by the following equation (3), for example.

Y = (R + G1 + B + G2) / 4 (3) Here, G1 and G2 are signal values of G signals that are diagonally adjacent to each other when the color filter array is the Bayer array, and When the array is a G stripe R / B perfect checkerboard array or a honeycomb array, the signal values are G signals that are vertically adjacent to each other. That is, the Y signal in this case is a so-called YH (Y High) signal, which is a signal of the high frequency component of the luminance signal used to generate a high resolution image signal from the image signal obtained by the image pickup device 16. Also in this case, the same effect as that of the fourth embodiment can be obtained.

Further, in each of the above embodiments, the case where the one-dimensional filter is applied as the filter used for the noise reduction processing has been described, but the present invention is not limited to this, and for example, as shown in FIG. A two-dimensional filter may be applied, or a three-dimensional filter (not shown) may be applied. Also in these cases, the same effect as each of the above-described embodiments can be obtained.

In each of the above embodiments, the case where the present invention is applied to a digital camera has been described. However, the present invention is not limited to this, and for example, an image pickup device such as a flatbed scanner or a film scanner may be used. It goes without saying that it can be applied to all image input devices used.

Further, in each of the above-described embodiments, the case where the noise reduction processing mode which can be set by the user is only the two modes of the first mode and the second mode has been described, but the present invention is not limited to this. However, other modes, for example, a mode in which noise reduction processing is performed only on one of R, G, and B signals, a mode in which noise reduction processing is performed only on R signals and G signals, and G signals and B signals It is also possible to adopt a mode in which a mode for performing the noise reduction processing only for only one can be set. In this case, as compared with each of the above-described embodiments, the image signal to be subjected to the noise reduction processing can be finely specified, so that the noise can be reduced more effectively.

[0129]

According to the signal processing apparatus and the signal processing method of the present invention, the analog signal output from the image pickup device is converted into a digital signal, and noise reduction processing for reducing noise is performed on the digital signal. Since signal processing other than the noise reduction processing is performed on the digital signal on which the noise reduction processing has been performed, it is possible to effectively reduce noise from the low luminance portion to the high luminance portion. Is obtained.

[Brief description of drawings]

FIG. 1 is a block diagram showing a configuration of a digital camera 10 according to a first embodiment.

FIG. 2 is a schematic diagram showing an arrangement example of color filters of R, G, and B provided on the light receiving surface of the light receiving element of the image sensor 16 according to the embodiment.

FIG. 3 is a block diagram showing a configuration of a noise reduction unit 30 according to the embodiment.

FIG. 4 is a block diagram showing a configuration example of a filter provided for performing noise reduction processing in a filter unit 30C in the noise reduction unit 30 according to the exemplary embodiment.

FIG. 5 is a block diagram showing a configuration of a digital camera 10 according to a second embodiment.

FIG. 6 is a block diagram showing a configuration of a digital camera 10 according to a third embodiment.

FIG. 7 is a schematic diagram for explaining an edge detection method.

FIG. 8 is a block diagram showing a configuration of a digital camera 10 according to a fourth embodiment.

FIG. 9 is a schematic diagram showing a configuration example of a two-dimensional filter.

FIG. 10 is a waveform diagram for explaining effects of a linear filter and a non-linear filter.

FIG. 11 is a block diagram for explaining a conventional noise reduction processing technique.

FIG. 12 is a schematic diagram for explaining the problems of the conventional technique.

FIG. 13 is a schematic diagram for explaining the problems of the conventional technique.

[Explanation of symbols]

10 Digital Cameras 10A, 10B, 10C, 10D Image Signal Processor 16 Image Sensor 20 A / D Converter (A / D Converter) 24 Black Level Corrector (Correction Processor) 25 Y Signal Generator (Luminance Signal Generator) ) 26 RGB gain correction unit (correction processing unit) 28 Edge determination unit (edge detection unit) 30, 30 'Noise reduction unit (noise reduction processing unit) 32, 32' Signal processing unit (other processing unit) 50 System control unit

Continued front page    F-term (reference) 5C021 PA31 PA34 PA51 PA53 PA66                       RA02 XA61 XB07 YA01 YA06                 5C065 AA01 BB03 BB22 CC01 DD01                       GG02 GG03 GG15 GG22 GG23                 5C066 AA01 BA01 CA05 EA08 EA14                       EA16 EC05 EC12 GA01 GB01                       JA02 KC02 KD02 KM02

Claims (14)

[Claims] 1. An A / D conversion means for converting an analog signal output from an image sensor into a digital signal, a noise reduction processing means for performing noise reduction processing for reducing noise on the digital signal, and the noise reduction. A signal processing device, comprising: another processing means for performing signal processing other than the noise reduction processing on the digital signal subjected to the noise reduction processing by the processing means. 2. The noise reduction processing means further comprises correction processing means for performing at least one correction processing of a black level correction processing and a gain correction processing on the digital signal converted by the A / D conversion means. The signal processing device according to claim 1, wherein the noise reduction processing is performed on the digital signal that has been corrected by the correction processing means. 3. The signal processing device according to claim 1, wherein the noise reduction processing means performs the noise reduction processing based on at least one of a linear filter and a non-linear filter. 4. An edge detection unit for detecting an edge of an image represented by the digital signal converted by the A / D conversion unit is further provided, and the noise reduction processing unit adds to the edge detected by the edge detection unit. The signal processing device according to claim 3, wherein noise reduction processing based on the non-linear filter is performed on a corresponding digital signal, and noise reduction processing based on the linear filter is performed on another digital signal. 5. The noise reduction processing means further comprises edge detection means for detecting edges of an image represented by the digital signal converted by the A / D conversion means, and the noise reduction processing means has the non-linear filter for each pixel for the digital signal. The processing result by the linear filter and the processing result by the linear filter are mixed by increasing the mixing ratio of the processing result by the non-linear filter as the pixel position of the digital signal to be processed approaches the edge detected by the edge detection means. The signal processing device according to claim 3, wherein the noise reduction process is performed. 6. The apparatus further comprises a luminance signal generation means for generating a luminance signal based on the digital signal converted by the A / D conversion means, and the edge detection means detects the edge based on the luminance signal. The signal processing device according to claim 4 or 5. 7. The image pickup device is configured to pick up an image of a subject by separating it into a plurality of colors, and the noise reduction processing means selectively performs the noise reduction processing on at least one of the plurality of colors. The signal processing device according to any one of claims 1 to 6. 8. An analog signal output from an image sensor is converted into a digital signal, noise reduction processing for reducing noise is performed on the digital signal, and the digital signal subjected to the noise reduction processing is subjected to the noise reduction processing. A signal processing method for performing signal processing other than noise reduction processing. 9. The method according to claim 8, wherein at least one of a black level correction process and a gain correction process is performed on the digital signal, and the noise reduction process is performed on the digital signal on which the correction process is performed. Signal processing method. 10. The signal processing method according to claim 8, wherein the noise reduction processing is performed based on at least one of a linear filter and a non-linear filter. 11. An edge of an image indicated by the digital signal is detected, a noise reduction process based on the nonlinear filter is performed on a digital signal corresponding to the detected edge, and the linear is performed on another digital signal. The signal processing method according to claim 10, wherein noise reduction processing based on a filter is performed. 12. A pixel position of a digital signal to be processed by detecting an edge of an image represented by the digital signal, and processing the digital signal by the non-linear filter for each pixel and the linear filter. 11. The signal processing method according to claim 10, wherein the noise reduction processing is performed by increasing the mixing ratio of the processing result by the non-linear filter and performing mixing as the edge approaches the detected edge. 13. A luminance signal is generated based on the digital signal, and the edge is detected based on the luminance signal.
The signal processing method according to claim 1 or 12. 14. The image pickup device according to claim 8, wherein the image pickup device separates an image of a subject into a plurality of colors and picks up an image, and the noise reduction processing is selectively performed on at least one color of the plurality of colors. The signal processing method according to claim 1.