JP2008252767A - Noise reducer, noise reducing method and electronic apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a noise reducer which decreases a data amount, simplifies a processing, and can obtain a sufficient noise reduction effect in a pixel having a large motion amount also. <P>SOLUTION: The noise reducer 30 has: a first filter and a color separator 37 which act a spacial lowpass filter in a predetermined band upon an RAW image signal after a three-dimensional noise reduction and color-separation at the same time, and obtain a first after filter signal of each color; a second filter and a color separator 38 which act a spacial lowpass filter in a higher band than the first filter and the color separator 37 upon the RAW image signal after the three-dimensional noise reduction and color-separating at the same time, and obtain a second after filter signal of each color; and a weighting adder 40 which weighting-adds the first after filter signal of each color and the second after filter signal of each color for each color according to a weight coefficient calculated in response to the motion amount in a time base direction of each pixel to obtain an output signal of each color. <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a noise reduction device and a noise reduction method for reducing noise of a moving image composed of a plurality of frames, and more particularly to a noise reduction device and a noise reduction method that reduce the amount of data and simplify processing. . The present invention also relates to an electronic apparatus provided with the noise reduction device.

  In recent years, with the development of various digital technologies, imaging devices such as digital cameras and digital videos that take digital images with solid-state imaging devices such as CCD (Charge Coupled Device) and CMOS (Complimentary Metal Oxide Semiconductor) sensors, and digital images Display devices such as liquid crystal displays and plasma televisions for displaying are becoming widespread. Various noise reduction techniques for reducing noise of moving images composed of a plurality of frames in such an imaging apparatus and display apparatus have been proposed.

  Typical examples of this noise reduction technique include a two-dimensional noise reduction process for reducing noise by applying a spatial low-pass filter (LPF), and signals of two frames that are temporally continuous. There is a three-dimensional noise reduction process in which a temporal low-pass filter (LPF) is acted on by adding to reduce noise.

As a noise reduction method using this three-dimensional noise reduction processing, Patent Document 1 discloses that three-dimensional noise reduction processing is performed on RAW image data generated by an image sensor before color separation by pixel interpolation. Thus, a noise reduction method is described in which the amount of data is reduced and the processing is simplified. Note that in the noise reduction method of Patent Document 1, the influence of the three-dimensional noise reduction process is large on a pixel with a small motion amount calculated by the difference between two temporally continuous frames, and the pixel with a large motion amount is used. Are weighted so as to reduce the influence of the three-dimensional noise reduction process.
JP 2005-175864 A

  However, in the noise reduction method of Patent Document 1, although reduction of data amount and simplification of processing are achieved, only three-dimensional noise reduction processing is performed as processing for noise reduction. Since pixels having a large amount of weight are weighted so as to reduce the influence of the three-dimensional noise reduction process, there is a problem that a noise reduction effect is hardly obtained for pixels having a large amount of motion. In order to solve this problem, two-dimensional noise reduction processing may be performed after color separation by pixel interpolation. In this case, in order to perform two-dimensional noise reduction processing, signal values for each color are divided into a plurality of lines. A memory for storage is required, and the increase in size of the apparatus becomes a problem.

  In view of the above problems, the present invention provides a noise reduction apparatus and noise reduction method that reduces the amount of data and simplifies processing, and is a noise that provides a sufficient noise reduction effect even for pixels with a large amount of motion. An object is to provide a reduction device and a noise reduction method. Moreover, an object of this invention is to provide the electronic device provided with this noise reduction apparatus.

  In the present invention, the noise reduction device is a noise reduction device including a three-dimensional noise reduction processing unit that performs a three-dimensional noise reduction process on a RAW image signal input from the outside and obtains a three-dimensional noise-reduced signal. A plurality of noise reduction processing and color separation units that simultaneously perform noise reduction processing and color separation on the three-dimensional noise reduction signal input from the three-dimensional noise reduction processing unit to obtain a filtered signal of each color. And a mixing unit that mixes the post-filter signals of each color input from a plurality of the noise reduction processing and color separation units and obtains an output signal of each color.

  In the present invention, the noise reduction device is a noise reduction device including a three-dimensional noise reduction processing unit that performs a three-dimensional noise reduction process on a RAW image signal input from the outside and obtains a three-dimensional noise-reduced signal. A plurality of filters and a color separation unit that perform a color separation on the three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit and simultaneously perform color separation to obtain a filtered signal of each color. And a mixing unit that mixes the post-filter signals of each color input from the plurality of filters and color separation units to obtain an output signal of each color.

  According to the present invention, in the noise reduction device configured as described above, the plurality of filters and color separation units have a predetermined band with respect to the three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit. The first filter and the color separation unit for performing color separation at the same time as the spatial low-pass filter and obtaining the first post-filter signal of each color, and the three-dimensional noise input from the three-dimensional noise reduction processing unit A second filter and a color separation unit that perform a color separation at the same time as applying a spatial low-pass filter higher than the predetermined band to the post-reduction signal, and obtain a second post-filter signal of each color; The mixing unit is configured to receive the first post-filter signal of each color input from the first filter and the color separation unit, and input from the second filter and the color separation unit. The second post-filter signal for each color is weighted and added by a predetermined weighting factor for each color to obtain the output signal for each color, and the RAW image signal input from the outside and one frame before A motion detection unit that obtains a motion amount in the time axis direction of each pixel by calculating a difference value from the signal after the three-dimensional noise reduction, and the motion in the time axis direction of each pixel input from the motion detection unit It is desirable to have a weight coefficient detection unit that calculates the predetermined weight coefficient according to the amount.

  According to the present invention, in the noise reduction device having the above-described configuration, the plurality of filters and color separation units are spatially applied to the post-three-dimensional noise reduction signal input from the three-dimensional noise reduction processing unit. At the same time as performing the low-pass filter, color separation is performed, and the spatial filter and color separation unit for obtaining the spatial filtered signal of each color, and the three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit, A horizontal low-pass filter is applied simultaneously and color separation is performed, and a horizontal filter and color separation unit for obtaining a horizontal filtered signal for each color, and the three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit On the other hand, the vertical filter and the color separation that obtains the signal after the vertical filter of each color by performing the color separation at the same time as applying the low-pass filter in the vertical direction. And the mixing unit includes the spatial filter post-signal for each color input from the spatial filter and the color separation unit, and the horizontal filter post-signal for each color input from the horizontal filter and the color separation unit. The post-vertical filter signal of each color input from the vertical filter and color separation unit is weighted and added by a predetermined weighting factor for each color to obtain the output signal of each color, which is input from the outside. A motion detection unit that obtains a motion amount in the time axis direction of each pixel by calculating a difference value between the RAW image signal and the three-dimensional noise-reduced signal one frame before, and is input from the motion detection unit. A correlation direction detection unit for calculating a horizontal direction correlation value and a vertical direction correlation value of the motion amount in the time axis direction of each pixel, and a time axis of each pixel input from the motion detection unit The predetermined weighting factor is calculated according to the amount of motion in the direction and the horizontal direction correlation value and the vertical direction correlation value of the amount of motion in the time axis direction of each pixel input from the correlation direction detection unit. And a weighting coefficient detection unit.

  In the present invention, the predetermined weighting factor may be different for each color in the noise reduction device configured as described above.

  In the present invention, the noise reduction method is a noise reduction method including a three-dimensional noise reduction processing step of performing a three-dimensional noise reduction process on a RAW image signal constituting a moving image to obtain a three-dimensional noise-reduced signal. A plurality of noise reduction processing and color separation steps for simultaneously performing noise reduction processing and color separation on the three-dimensional noise reduction signal obtained in the three-dimensional noise reduction processing step to obtain a filtered signal of each color; A mixing step of mixing the post-filter signals of each color obtained in a plurality of the noise reduction processing and color separation steps to obtain an output signal of each color.

  In the present invention, the noise reduction method is a noise reduction method including a three-dimensional noise reduction step of performing a three-dimensional noise reduction process on a RAW image signal constituting a moving image to obtain a three-dimensional noise-reduced signal, A plurality of filters and a color separation step for performing a color separation on the three-dimensional noise-reduced signal obtained in the three-dimensional noise reduction processing step simultaneously with applying a low-pass filter to obtain a filtered signal of each color; And a mixing step of mixing the post-filter signals of the respective colors obtained in the plurality of filters and color separation unit steps to obtain output signals of the respective colors.

  According to the present invention, in the noise reduction method configured as described above, the plurality of filters and color separation steps have a predetermined bandwidth with respect to the three-dimensional noise reduced signal obtained in the three-dimensional noise reduction processing step. The three-dimensional noise obtained in the first filter and the color separation step for performing color separation at the same time as the spatial low-pass filter and obtaining the first filtered signal of each color, and the three-dimensional noise reduction processing step A second filter and a color separation step for applying a spatial low pass filter higher than the predetermined band to the post-reduction signal and simultaneously performing color separation to obtain a second post-filter signal for each color; The mixing step includes the first post-filter signal of each color obtained in the first filter and color separation step, and the second filter. The step of obtaining the output signal of each color by weighting and adding the second post-filter signal of each color obtained in the filter and color separation step by a predetermined weighting factor for each color, the RAW constituting the moving image A motion detection step of obtaining a motion amount in the time axis direction of each pixel by calculating a difference value between the image signal and the three-dimensional noise-reduced signal one frame before, and each pixel obtained in the motion detection step It is desirable to include a weighting coefficient detection step of calculating the predetermined weighting coefficient according to the amount of motion in the time axis direction.

  According to the present invention, in the noise reduction method having the above-described configuration, the plurality of filters and color separation steps are spatially compared to the three-dimensional noise-reduced signal obtained in the three-dimensional noise reduction processing step. At the same time as applying the low-pass filter, color separation is performed, and a spatial filter and a color separation step for obtaining a spatial filtered signal for each color, and the three-dimensional noise-reduced signal obtained in the three-dimensional noise reduction processing step, A horizontal low-pass filter is simultaneously applied to perform color separation, and a horizontal filter and a color separation step for obtaining a horizontal filtered signal for each color; and the three-dimensional noise-reduced signal obtained in the three-dimensional noise reduction processing step. On the other hand, a vertical low-pass filter is applied and color separation is performed at the same time to obtain a vertical filtered signal for each color. A vertical filter and a color separation step, and the mixing step includes a signal after the spatial filter obtained by the spatial filter and the color separation step and a color filter obtained by the horizontal filter and the color separation step. The step of obtaining the output signal of each color by weighting and adding the post-horizontal filter signal and the post-vertical filter signal of each color obtained in the vertical filter and color separation step with a predetermined weighting factor for each color. A motion detection step of obtaining a motion amount in the time axis direction of each pixel by calculating a difference value between the RAW image signal constituting the moving image and the three-dimensional noise-reduced signal one frame before; Correlation direction detection for calculating a horizontal direction correlation value and a vertical direction correlation value of the amount of motion in the time axis direction of each pixel obtained in the detection step. Step, the amount of motion in the time axis direction of each pixel obtained in the motion detection step, the horizontal direction correlation value of the amount of motion in the time axis direction of each pixel obtained in the correlation direction detection step, and It is desirable to have a weighting factor detection step of calculating the predetermined weighting factor according to the vertical direction correlation value.

  In the present invention, the predetermined weighting factor may be different for each color in the noise reduction method configured as described above.

  In the present invention, the electronic device is an electronic device that is provided with a RAW image signal that constitutes a moving image by external input or imaging, and includes a noise reduction device that reduces noise of the moving image. The noise reduction device having the above configuration is characterized in that

  According to the noise reduction device and the noise reduction method of the present invention, a plurality of post-filter signals for each color obtained by applying a low-pass filter to the three-dimensional noise-reduced signal and performing color separation at the same time Since an output signal is obtained by weighted addition for each color by a weighting coefficient, a noise reduction device and a noise reduction method that reduce the amount of data and simplify processing, and further, a noise reduction effect by three-dimensional noise reduction processing A sufficient noise reduction effect can be obtained even for a pixel with a large amount of motion that cannot be sufficiently obtained.

  In addition, according to the noise reduction device and the noise reduction method of the present invention, a spatial low-pass filter of a predetermined band is applied to the three-dimensional noise-reduced signal, and color separation is performed at the same time. A first filtered signal and a second filtered signal of each color obtained by applying a spatial low-pass filter higher than a predetermined band to the signal after three-dimensional noise reduction and simultaneously performing color separation; Is obtained by weighted addition for each color using a weighting coefficient calculated according to the amount of motion in the time axis direction of each pixel. For example, for a pixel with a small amount of motion, a high-frequency space is obtained. The effect of the low-pass filter is strengthened, while the effect of the low-pass spatial low-pass filter is weakened. If the influence of the high-frequency spatial low-pass filter is weakened, it is possible to perform appropriate noise reduction processing according to the amount of motion, so that a sufficient noise reduction effect can be obtained. In addition, image blur can be suppressed.

  In addition, according to the noise reduction device and the noise reduction method of the present invention, a spatial low-pass filter is applied to the three-dimensional noise-reduced signal, and color-separated signals obtained by performing color separation at the same time, A low-pass filter in the horizontal direction obtained by applying a horizontal low-pass filter to the three-dimensional noise-reduced signal and simultaneously performing color separation, and a low-pass in the vertical direction with respect to the three-dimensional noise-reduced signal A vertical filtered signal of each color obtained by performing color separation at the same time as applying a filter, a motion amount in the time axis direction of each pixel, a horizontal correlation value of a motion amount in the time axis direction of each pixel, and Since an output signal is obtained by weighted addition for each color using a weighting coefficient calculated according to the vertical correlation value, for example, the amount of motion is large. For pixels with a large horizontal correlation value of motion amount, the effect of the horizontal low-pass filter is strengthened, while the effect of the spatial low-pass filter and the vertical low-pass filter is weakened to increase the motion amount and the motion amount. For pixels with a large vertical correlation value, the effect of the low-pass filter in the vertical direction is strengthened, while the effect of the spatial low-pass filter and the low-pass filter in the horizontal direction is weakened. If the influence of the spatial low-pass filter is strengthened while the influence of the horizontal low-pass filter and the vertical low-pass filter is weakened, the amount of movement of each pixel in the time axis direction and the movement of each pixel in the time axis direction are reduced. Because it is possible to perform appropriate noise reduction processing according to the strength of the horizontal and vertical correlation of the amount, sufficient noise reduction effect Rukoto can, also it is possible to suppress an image blur.

  In addition, according to the noise reduction device and the noise reduction method of the present invention, the predetermined weighting coefficient is different for each color. For example, the weight of the first filtered signal of the color G is changed to the weight of the first filtered signal of the color R and B. If it is made smaller than the signal weight, the blurring of the color G, which has a high contribution to the luminance signal, can be suppressed.

  Embodiments of the present invention will be described with reference to the drawings. In the following description, an imaging apparatus such as a digital camera or digital video provided with a noise reduction apparatus (hereinafter, included in the “image processing unit”) that performs the noise reduction method of the present invention will be described as an example. As will be described later, a display device that performs digital processing of an image, such as a liquid crystal display or a plasma television, may be used as long as it has a similar noise reduction device.

(Configuration of imaging device)
First, the internal configuration of the imaging apparatus will be described with reference to the drawings. FIG. 1 is a block diagram illustrating an internal configuration of the imaging apparatus.

  The imaging apparatus of FIG. 1 is a solid-state imaging device (image sensor) 1 such as a CCD or CMOS sensor that converts light incident from an object into an electrical signal; and an image signal that is an analog signal output from the image sensor 1 is digitally converted. AFE (Analog FrontEnd) 2 for converting to a signal; a microphone 3 for converting sound input from the outside into an electric signal; various types including color separation processing and noise reduction processing for an image signal to be a digital signal from the AFE 2 An image processing unit 4 that performs image processing; an audio processing unit 5 that converts an audio signal that is an analog signal from the microphone 3 into a digital signal; an image signal from the image processing unit 4 and an audio signal from the audio processing unit 5 A compression processing unit 6 that performs compression coding processing such as MPEG (Moving Picture Experts Group) compression method; and a compression code that is compression-coded by the compression processing unit 6 A driver unit 7 that records the encoded signal in the external memory 20; a decompression processing unit 8 that decompresses and decodes the compressed encoded signal read from the external memory 20 by the driver unit 7; A display 9 for displaying an image based on the image signal; an audio output circuit unit 10 for converting the audio signal from the decompression processing unit 8 into an analog signal; and reproducing and outputting audio based on the audio signal from the audio output circuit unit 10 A speaker 11 that performs; a TG (Timing Generator) 12 that outputs a timing control signal for matching the operation timing of each block; a CPU (Central Processing Unit) 13 that controls the drive operation of the entire imaging apparatus; A memory 14 for storing each program for storing and temporarily storing data when the program is executed; and an operation unit 1 to which an instruction from a user is input When; comprises; a memory 14 and a bus line 16 for exchanging data with each block; CPU 13 and the bus line 17 for exchanging data with each block.

  In the imaging device having such a configuration, the image sensor 1 is a solid-state imaging device in which a plurality of types of color filters having different transmittances are installed on the surface of each pixel, that is, a solid-state imaging device including a single-plate color filter. . This color filter is provided with a different type of color filter for each adjacent pixel. Therefore, the video signal output from the image sensor 1 is a so-called RAW image signal in which each pixel has only single color information. As a single-plate color filter installed in such an image sensor 1, for example, a Bayer filter including an R (Red) filter, a G (Green) filter, and a B (Blue) filter as illustrated in FIG. 2 is configured. The Bayer filter of FIG. 2 is configured such that lines in which G filters and R filters are alternately arranged in a horizontal direction and lines in which B filters and G filters are alternately arranged in a horizontal direction are alternately arranged in a vertical direction. Prepare. That is, lines in which G filters and B filters are alternately arranged in the vertical direction and lines in which R filters and G filters are alternately arranged in the vertical direction are alternately arranged in the horizontal direction. In the image sensor 1 having such a Bayer filter, video signals that are RGB signals are output from pixels having RGB filters.

  In this imaging apparatus, when the operation unit 15 instructs to perform an imaging operation, an image signal that is an analog signal obtained by the photoelectric conversion operation of the image sensor 1 is output to the AFE 2. At this time, the image sensor 1 receives the timing control signal from the TG 12 to perform horizontal scanning and vertical scanning, and output an image signal serving as data for each pixel. In the AFE 2, when an image signal that is an analog signal is converted into a digital signal and input to the image processing unit 4, various image processing such as noise reduction processing and color separation processing is performed. Details of the image processing unit 4 will be described later.

  Then, the image signal subjected to the image processing by the image processing unit 4 is given to the compression processing unit 6. At this time, an audio signal which is an analog signal obtained by inputting the sound into the microphone 3 is converted into a digital signal by the audio processing unit 5 and given to the compression processing unit 6. As a result, the compression processing unit 6 compresses and encodes the image signal and the audio signal, which are digital signals, based on the MPEG compression encoding method, gives the image signal and the audio signal to the driver unit 7, and records them in the external memory 20. At this time, the compressed signal recorded in the external memory 20 is read out by the driver unit 7 and applied to the expansion processing unit 8 to be subjected to expansion processing to obtain an image signal. This image signal is given to the display 9 and a subject image currently being photographed through the image sensor 1 is displayed.

  In the above description, the operation at the time of moving image shooting has been described. However, even when a still image shooting is instructed, an audio signal is not acquired by the microphone 3 and a compressed signal of only an image signal is recorded in the external memory 20. However, the basic operation is the same as that during moving image shooting. In the case of this still image shooting, not only the compressed signal for the still image shot by the operation unit 15 is recorded in the external memory 20, but also the compressed signal for the current image shot by the image sensor 1 is stored in the external memory. 20 is temporarily recorded. As a result, the compression signal for the currently photographed image is decompressed by the decompression processing unit 8 so that the current image photographed by the image sensor 1 is displayed on the display 9 and can be confirmed by the user.

  When performing the imaging operation in this way, the TG 12 gives a timing control signal to the AFE 2, the video processing unit 4, the audio processing unit 5, the compression processing unit 6, and the expansion processing unit 8, and one frame by the image sensor 1. An operation synchronized with each imaging operation is performed. At the time of still image shooting, a timing control signal is given from the TG 12 to the image sensor 1, the AFE 2, the video processing unit 4, and the compression processing unit 6 based on the shutter operation by the operation unit 15. The operation timing of each unit is synchronized.

  When an instruction to reproduce a moving image or image recorded in the external memory 20 is given through the operation unit 15, the compressed signal recorded in the external memory 20 is read out by the driver unit 7 and is decompressed by the decompression processing unit 8. Given to. Then, the decompression processing unit 8 decompresses and decodes the image signal and the audio signal based on the MPEG compression encoding method. Then, the image signal is given to the display 9 to reproduce the image, and the audio signal is given to the speaker 11 via the audio output circuit unit 10 to reproduce the audio. Thereby, the moving image based on the compressed signal recorded in the external memory 20 is reproduced together with the sound. Further, when the compressed signal consists only of the image signal, only the image is reproduced on the display 9.

(First Embodiment of Noise Reduction Device)
Next, a first embodiment of the noise reduction device included in the image processing unit 4 will be described. FIG. 3 is a block diagram showing a schematic configuration of the noise reduction apparatus 30 according to the first embodiment of the present invention. The noise reduction device 30 includes a noise detection unit 31 that receives a signal from the external and frame memory 34 and outputs a signal to the multiplier 35; a signal that is input from the external and frame memory 34, and a feedback coefficient detection unit 33 and a weighting factor A motion detection unit 32 that outputs a signal to the detection unit 39; a feedback coefficient detection unit 33 that outputs a signal to the multiplier 35; and a signal that is input from the subtractor 36; A frame memory 34 that outputs signals to the unit 31 and the motion detection unit 32; a multiplier 35 that receives signals from the noise detection unit 31 and the feedback coefficient detection unit 33 and outputs a signal to the subtractor 36; A signal is input from 35, and a subtractor 36 that outputs signals to the frame memory 34, the first filter and color separation unit 37, and the second filter and color separation unit 38. A first filter and color separation unit 37 that receives a signal from the subtractor 36 and outputs a signal to the weighted addition unit 40; a second filter that receives a signal from the subtractor 36 and outputs a signal to the weighted addition unit 40; And a weight coefficient detection unit 39 that receives a signal from the motion detection unit 32 and outputs a signal to the weighting addition unit 40; a first filter and color separation unit 37, a second filter, A weighting addition unit 40 that receives signals from the color separation unit 38 and the weighting coefficient detection unit 39 and outputs the signals to the subsequent stage. In the following description, the signal of the current frame is input from the outside, but this external means the image processing in the previous stage of the AFE 2 of the imaging device of FIG. 1 or the noise reduction device 30 of the image processing unit 4. It refers to the part to be performed.

  Here, an outline of the operation of the noise reduction device 30 will be described. First, an external current frame signal (Bayer data) is input to the noise detector 31, the motion detector 32, and the subtractor 36. Further, a signal one frame before the current frame from the frame memory 34 is input to the noise detection unit 31 and the motion detection unit 32. Then, the noise reduction processing unit 31 calculates a noise detection value of each pixel from the signal of the current frame input from the outside and the signal of the previous frame input from the frame memory 34, and supplies this to the multiplier 35. Output.

  Next, the motion detection unit 32 calculates the motion amount of each pixel from the signal of the current frame input from the outside and the signal of the previous frame input from the frame memory 34, and this is used to detect the feedback coefficient. To the unit 33 and the weighting coefficient detection unit 39. Then, the feedback coefficient detection unit 33 calculates the feedback coefficient k of each pixel from the motion amount of each pixel input from the motion detection unit 32, and outputs this to the multiplier 35.

  Then, the multiplier 35 multiplies the noise detection value of each pixel input from the noise detection unit 31 by the feedback coefficient k of each pixel input from the feedback coefficient detection unit 33, and obtains each obtained The subtraction noise value of the pixel is output to the subtracter 36. Then, the subtractor 36 subtracts the subtraction noise value of each pixel input from the multiplier 35 from the signal of the current frame input from the outside, and the obtained three-dimensional noise-reduced signal is stored in the frame memory 34, the second signal. The data is output to the first filter and color separation unit 37 and the second filter and color separation unit 38.

  Next, the first filter and color separation unit 37 applies a spatial (two-dimensional) low-pass filter (LPF) to the three-dimensional noise-reduced signal input from the subtractor 36 at the same time. Color separation is performed, and the obtained first filtered signals L1 (R), L1 (G), and L1 (B) of the respective colors R, G, and B are output to the weighted addition unit 40. In addition, the second filter and color separation unit 38 is higher in spatial (two-dimensional) than the first filter and color separation unit 37 with respect to the three-dimensional noise-reduced signal input from the subtractor 36. Color separation is performed at the same time as the low pass filter (LPF) is operated, and the obtained second filtered signals L2 (R), L2 (G), L2 (B) of the respective colors R, G, B are weighted and added. Output to 40. In addition, the weight coefficient detection unit 39 calculates the weight coefficient α of each pixel from the motion amount of each pixel input from the motion detection unit 32, and outputs this to the weight addition unit 40.

  Next, the weighted addition unit 40 receives the first filtered signals L1 (R), L1 (G), and L1 (B) of the R, G, and B colors input from the first filter and color separation unit 37. And the second filtered signals L2 (R), L2 (G), and L2 (B) of the R, G, and B colors input from the second filter and color separation unit 38 from the weight coefficient detection unit 39 Weighted addition is performed for each color by the weighting factor α of each input pixel (that is, O (R) = L1 (R) × α + L2 (R) × (1−α), O (G) = L1 (G) × α + L2 (G) × (1−α), O (B) = L1 (B) × α + L2 (B) × (1−α)), and the obtained output signal O (R for each color of R, G, B ), O (G), and O (B) are output to the subsequent stage.

Next, details of the noise detection unit 31 will be described. The noise detector 31 detects a noise detection value for the difference signal value x of each pixel obtained by subtracting the signal of the previous frame input from the frame memory 34 from the signal of the current frame input from the outside. f (x) is calculated and output to the multiplier 35. Here, as the function f (x), for example, the following equation 1 and those shown in FIG. 4 are used.

  4 and the function f (x) shown in FIG. 4 suppress the noise detection value f (x) when the absolute value of the difference signal value x of each pixel exceeds the threshold th, and the motion f A malfunction that erroneously detects a part as noise can be prevented.

  Next, details of the motion detection unit 32 will be described. FIG. 5 is a block diagram illustrating a schematic configuration of the motion detection unit 32. The motion detection unit 32 receives a signal from the outside and the frame memory 34 and outputs a signal to the absolute value conversion unit 322; and a signal that is input from the subtraction unit 321 and outputs a signal to the filter unit 323 And a filter unit 323 that receives a signal from the absolute value unit 322 and outputs the signal to the feedback coefficient detection unit 33.

  Here, an outline of the operation of the motion detection unit 32 will be described. First, the signal of the current frame from the outside and the signal of the previous frame from the frame memory 34 are input to the subtractor 321. Then, the subtractor 321 calculates the difference between the signal of the current frame input from the outside and the signal of the previous frame input from the frame memory 34, and outputs the obtained difference signal to the absolute value conversion unit 322. To do.

  Next, the absolute value converting unit 322 converts the input difference signal into an absolute value, and outputs the obtained absolute value signal to the filter unit 323. Then, the filter unit 323 outputs the motion amount of each pixel obtained by applying a low pass filter to the input absolute value signal to the feedback coefficient detection unit 33. Here, by applying a low-pass filter by the filter unit 323, the separation accuracy between the noise signal and the signal indicating motion is increased.

Next, details of the feedback coefficient detection unit 33 will be described. The feedback coefficient detection unit 33 calculates the feedback coefficient k = g (y) of each pixel with respect to the motion amount y of each pixel input from the motion detection unit 32, and outputs this to the multiplier 35. Here, as the function g (y), for example, the following equation 2 and those shown in FIG. 6 are used.

  Equation 2 and the function g (y) shown in FIG. 6 indicate that the feedback coefficient k decreases from 1 as a linear function as the amount of motion y increases, and when the amount of motion y exceeds the threshold thA, the feedback coefficient k As a result, the noise reduction effect by the noise detection unit 31 for pixels with a small amount of motion y, that is, the effect of three-dimensional noise reduction processing is increased, and the noise detection unit for pixels with a large amount of motion y 31 reduces the noise reduction effect, that is, the effect of the three-dimensional noise reduction processing. The threshold thA is desirably determined according to the input signal level and the noise level.

  Next, details of the first filter and color separation unit 37 will be described. In the following description, the coordinates of the pixel of interest are (0, 0), the signal value is S (0, 0), and the coordinates of the pixel at the position of the x pixel to the right and the y pixel above the pixel of interest are (x, y), the signal value is represented as S (x, y). That is, the coordinates of the pixel located two pixels to the right and two pixels above the target pixel are represented by (2, 2), and the signal value thereof is represented by S (2, 2). The coordinates of the pixel at the position of 2 pixels are represented as (−2, −2), and the signal value thereof is represented as S (−2, −2).

  The first filter and color separation unit 37 applies a spatial low-pass filter (LPF) to the three-dimensional noise-reduced signal (Bayer data) input from the subtractor 36 and simultaneously performs color separation to obtain an obtained signal. The first filtered signals L1 (R), L1 (G), and L1 (B) of the R, G, and B colors are output to the weighted addition unit 40.

  Specifically, for the pixel of interest for which the signal value of color R is obtained in the Bayer data, the signal value of color G is acquired using the filter shown in FIG. 7A, and FIG. The signal value of color B is acquired using the filter shown, and the signal value obtained in the Bayer data is used as it is for color R.

  That is, for the pixel of interest for which the color R signal value is obtained in the Bayer data, L1 (G) = 1/16 × S (−1,2) + 1/16 × S (1,2) + 1/16 × S (-2,1) + 1/8 × S (0,1) + 1/16 × S (2,1) + 1/8 × S (−1,0) + 1/8 × S (1,0) + 1 / 16 × S (−2, −1) + 1/8 × S (0, −1) + 1/16 × S (2, −1) + 1/16 × S (−1, −2) + 1/16 × S ( The signal value of the color G is acquired by 1 and -2.

  Further, for a target pixel for which a signal value of color R is obtained in Bayer data, L1 (B) = 1/16 × S (−3,3) + 1/16 × S (−1,3) +1/16 × S (1,3) + 1/16 × S (3,3) + 1/16 × S (−3,1) + 1/16 × S (−1,1) + 1/16 × S (1,1) +1 / 16 × S (3,1) + 1/16 × S (−3, −1) + 1/16 × S (−1, −1) + 1/16 × S (1, −1) + 1/16 × S ( 3, −1) + 1/16 × S (−3, −3) + 1/16 × S (−1, −3) + 1/16 × S (1, −3) + 1/16 × S (3, −3 ), The signal value of color B is acquired.

  Among the target pixels whose color G signal value is obtained in the Bayer data, those in which the left and right pixels are pixels whose color R signal value is obtained in the Bayer data are shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 8B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among the target pixels for which the color G signal value is obtained in the Bayer data, L1 (R) = 1 for the pixels in which the left and right pixels have the color R signal value in the Bayer data / 8 × S (−1,2) + 1/8 × S (1,2) + 1/8 × S (−3,0) + 1/8 × S (−1,0) + 1/8 × S (1, 0) + 1/8 × S (3,0) + 1/8 × S (−1, −2) + 1/8 × S (1, −2) is obtained as the signal value of color R.

  Of the target pixels for which the color G signal value is obtained in the Bayer data, L1 (B) = 1 for pixels in which the left and right pixels are for which the color R signal value is obtained in the Bayer data / 8 × S (0,3) + 1/8 × S (−2,1) + 1/8 × S (0,1) + 1/8 × S (2,1) + 1/8 × S (−2, − 1) The signal value of color B is acquired by + 1/8 × S (0, −1) + 1/8 × S (2, −1) + 1/8 × S (0, -3).

  Among the target pixels whose color G signal value is obtained in the Bayer data, those in which the left and right pixels are pixels whose color B signal value is obtained in the Bayer data are shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 9B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among the target pixels for which the color G signal value is obtained in the Bayer data, L1 (R) = 1 for the pixels in which the left and right pixels have the color B signal value in the Bayer data / 8 × S (0,3) + 1/8 × S (−2,1) + 1/8 × S (0,1) + 1/8 × S (2,1) + 1/8 × S (−2, − 1) The signal value of the color R is acquired by + 1/8 × S (0, −1) + 1/8 × S (2, −1) + 1/8 × S (0, -3).

  That is, among the target pixels whose color G signal value is obtained in the Bayer data, L1 (B) = 1 for pixels in which the left and right pixels are pixels whose color B signal value is obtained in the Bayer data / 8 × S (−1,2) + 1/8 × S (1,2) + 1/8 × S (−3,0) + 1/8 × S (−1,0) + 1/8 × S (1, 0) + 1/8 × S (3,0) + 1/8 × S (−1, −2) + 1/8 × S (1, −2) is obtained as the signal value of color B.

  For the target pixel for which the color B signal value is obtained in the Bayer data, the color R signal value is obtained using the filter shown in FIG. 10A, and the filter shown in FIG. Is used to acquire the signal value of the color G, and for the color B, the signal value obtained in the Bayer data is used as it is.

  That is, for a target pixel for which a color B signal value is obtained in Bayer data, L1 (R) = 1/16 × S (−3,3) + 1/16 × S (−1,3) +1/16 × S (1,3) + 1/16 × S (3,3) + 1/16 × S (−3,1) + 1/16 × S (−1,1) + 1/16 × S (1,1) +1 / 16 × S (3,1) + 1/16 × S (−3, −1) + 1/16 × S (−1, −1) + 1/16 × S (1, −1) + 1/16 × S ( 3, −1) + 1/16 × S (−3, −3) + 1/16 × S (−1, −3) + 1/16 × S (1, −3) + 1/16 × S (3, −3 ), The signal value of the color R is acquired.

  Further, for a target pixel for which a color B signal value is obtained in Bayer data, L1 (G) = 1/16 × S (−1,2) + 1/16 × S (1,2) + 1/16 × S (-2,1) + 1/8 × S (0,1) + 1/16 × S (2,1) + 1/8 × S (−1,0) + 1/8 × S (1,0) + 1 / 16 × S (−2, −1) + 1/8 × S (0, −1) + 1/16 × S (2, −1) + 1/16 × S (−1, −2) + 1/16 × S ( The signal value of the color G is acquired by 1 and -2.

  Next, details of the second filter and color separation unit 38 will be described. The second filter and color separation unit 38 is a spatial low-pass that is higher than the first filter and color separation unit 37 for the three-dimensional noise-reduced signal (Bayer data) input from the subtractor 36. At the same time as the filter (LPF) is operated, color separation is performed, and the obtained second filtered signals L2 (R), L2 (G), and L2 (B) of the respective colors R, G, and B are output to the weighted addition unit 40 To do.

  Specifically, for the pixel of interest for which the color R signal value is obtained in the Bayer data, the color G signal value is obtained using the filter shown in FIG. 11A, and FIG. The signal value of color B is acquired using the filter shown, and the signal value obtained in the Bayer data is used as it is for color R.

  That is, for the pixel of interest for which the color R signal value is obtained in the Bayer data, L2 (G) = 1/4 × S (0,1) + 1/4 × S (−1,0) + 1/4 × The signal value of the color G is acquired by S (1, 0) + 1/4 × S (0, −1).

  In addition, for a pixel of interest for which a color R signal value is obtained in Bayer data, L2 (B) = 1/4 × S (−1,1) + 1/4 × S (1,1) + 1/4 × The signal value of the color B is acquired by S (−1, −1) + 1/4 × S (1, −1).

  Of the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are those for which the color R signal value is obtained in the Bayer data, as shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 12B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among pixels of interest for which a color G signal value is obtained in Bayer data, L2 (R) = 1 for pixels in which the left and right pixels are pixels for which a color R signal value is obtained in Bayer data The signal value of the color R is acquired by / 2 × S (−1, 0) + 1/2 × S (1, 0).

  Of the target pixels for which the color G signal value is obtained in the Bayer data, L2 (B) = 1 for pixels in which the left and right pixels are pixels for which the color R signal value is obtained in the Bayer data The signal value of the color B is acquired by / 2 × S (0, 1) + 1/2 × S (0, −1).

  Among the target pixels whose color G signal value is obtained in the Bayer data, those in which the left and right pixels are pixels whose color B signal value is obtained in the Bayer data are shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 13B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among the target pixels for which the color G signal value is obtained in the Bayer data, L2 (R) = 1 for the pixels in which the left and right pixels are obtained for the color B signal value in the Bayer data The signal value of the color R is acquired by / 2 × S (0, 1) + 1/2 × S (0, −1).

  That is, among the target pixels whose color G signal value is obtained in the Bayer data, L2 (B) = 1 for pixels where the left and right pixels are pixels whose color B signal value is obtained in the Bayer data The signal value of the color B is acquired by / 2 × S (−1, 0) + 1/2 × S (1, 0).

  For the target pixel from which the signal value of color B is obtained in the Bayer data, the signal value of color R is acquired using the filter shown in FIG. 14A, and the filter shown in FIG. Is used to acquire the signal value of the color G, and for the color B, the signal value obtained in the Bayer data is used as it is.

  That is, for the pixel of interest for which the color B signal value is obtained in the Bayer data, L2 (R) = 1/4 × S (−1,1) + 1/4 × S (1,1) + 1/4 × The signal value of the color R is acquired by S (−1, −1) + 1/4 × S (1, −1).

  In addition, regarding a target pixel for which a color B signal value is obtained in Bayer data, L2 (G) = 1/4 × S (0,1) + 1/4 × S (−1,0) + 1/4 × The signal value of the color G is acquired by S (1, 0) + 1/4 × S (0, −1).

  Here, the circuit configuration of the second filter and color separation unit 38 will be described. FIG. 15 is a block diagram illustrating a circuit configuration of the second filter and color separation unit 38. In the following, it is assumed that x and y take values of −1, 0, and 1, respectively.

  The second filter and color separation unit 38 generates a filter coefficient a (x, y) and outputs this to the multiplier M (x, y), and stores S (x, y). Then, a memory m (x, y) that outputs this to the multiplier M (x, y) and a (x, y) × S (x, y) are calculated, and this is output to the adder P And an adder P that adds all of a (x, y) × S (x, y) and outputs the result to the weighted addition unit 40.

  Next, an outline of the operation of the second filter and color separation unit 38 will be described. First, the coefficient selection unit A determines whether the pixel of interest is a pixel from which the color R signal value is obtained in the Bayer data from the horizontal address and the vertical address of the pixel of interest, or whether the pixel of interest is the signal value of the color G in the Bayer data. Or whether the pixel of interest is a pixel from which a signal value of color B is obtained in the Bayer data. When the pixel of interest is a pixel for which the signal value of color G is obtained in the Bayer data, whether the pixels on the left and right of the pixel of interest are pixels for which the signal value of color R is obtained in the Bayer data, Alternatively, it is also determined whether the left and right pixels of the target pixel are pixels for which a color B signal value is obtained in the Bayer data.

  Next, the coefficient selection unit A first generates a filter coefficient a (x, y) for acquiring the signal value of the color R according to the determination result of the target pixel, and uses this to generate the multiplier M ( output to x, y). Each memory m (x, y) outputs S (x, y) to the multiplier M (x, y). Then, each multiplier M (x, y) calculates a (x, y) × S (x, y) and outputs this to the adder P. Then, the adder P adds all a (x, y) × S (x, y) input from each multiplier M (x, y), and weights the signal value of the obtained color R. Output to 40.

  Next, the coefficient selection unit A generates a filter coefficient a (x, y) for acquiring the signal value of the color G according to the determination result of the target pixel, and this is used as a multiplier M (x, output to y). Each memory m (x, y) outputs S (x, y) to the multiplier M (x, y). Then, each multiplier M (x, y) calculates a (x, y) × S (x, y) and outputs this to the adder P. Then, the adder P adds all a (x, y) × S (x, y) input from each multiplier M (x, y), and weights the signal value of the obtained color G. Output to 40.

  Next, the coefficient selection unit A generates a filter coefficient a (x, y) for obtaining the signal value of the color B according to the determination result of the target pixel, and this is used as the multiplier M (x, output to y). Each memory m (x, y) outputs S (x, y) to the multiplier M (x, y). Then, each multiplier M (x, y) calculates a (x, y) × S (x, y) and outputs this to the adder P. Then, the adder P adds all a (x, y) × S (x, y) input from each multiplier M (x, y), and weights the obtained signal value of the color B. Output to 40.

  Next, the operation of the second filter and color separation unit 38 will be specifically described by taking as an example the case where the pixel of interest is a pixel from which the signal value of color R is obtained in the Bayer data. In this case, the coefficient selection unit A first sets a (−1, 1) = 0, a (0, 1) = 0, a (1, 1) as filter coefficients for acquiring the signal value of the color R. = 0, a (-1, 0) = 0, a (0, 0) = 1, a (1, 0) = 0, a (-1, -1) = 0, a (0, -1) = 0, a (1, −1) = 0 are generated, and these are output to the corresponding multiplier M (x, y). Each memory m (x, y) outputs S (x, y) to the multiplier M (x, y).

  Then, L2 (R) = 0 × S (−1,1) + 0 × S (0,1) + 0 × S (1,1) + 0 × S is obtained by each multiplier M (x, y) and adder P. (-1, 0) + 1 * S (0,0) + 0 * S (1,0) + 0 * S (-1, -1) + 0 * S (0, -1) + 0 * S (1, -1) = S (0, 0) is calculated and output to the weighted addition unit 40.

  Next, the coefficient selection unit A uses a (-1, 1) = 0, a (0, 1) = 1/4, a (1, 1) = 0, a (-1,0) = 1/4, a (0,0) = 0, a (1,0) = 1/4, a (-1, -1) = 0, a ( 0, −1) = 1/4 and a (1, −1) = 0 are generated (see FIG. 11A), and these are output to the corresponding multiplier M (x, y). Each memory m (x, y) outputs S (x, y) to the multiplier M (x, y).

  Then, L2 (G) = 0 × S (−1,1) + 1/4 × S (0,1) + 0 × S (1,1) +1 is obtained by each multiplier M (x, y) and adder P. / 4 * S (-1,0) + 0 * S (0,0) + 1/4 * S (1,0) + 0 * S (-1, -1) + 1/4 * S (0, -1) +0 X S (1, -1) = 1/4 x S (0, 1) + 1/4 x S (-1, 0) + 1/4 x S (1, 0) + 1/4 x S (0, -1 ) Is calculated and output to the weighted addition unit 40.

  Next, the coefficient selecting unit A uses a (−1, 1) = 1/4, a (0, 1) = 0, a (1, 1) = 1/4, a (-1, 0) = 0, a (0, 0) = 0, a (1, 0) = 0, a (-1, -1) = 1/4, a ( 0, −1) = 0 and a (1, −1) = 1/4 are generated (see FIG. 11B), and these are output to the corresponding multiplier M (x, y). Each memory m (x, y) outputs S (x, y) to the multiplier M (x, y).

  Then, L2 (B) = 1/4 × S (−1,1) + 0 × S (0,1) + 1/4 × S (1,1, 1) by each multiplier M (x, y) and adder P. ) + 0 × S (−1,0) + 0 × S (0,0) + 0 × S (1,0) + 1/4 × S (−1, −1) + 0 × S (0, −1) +1/4 × S (1, −1) = 1/4 × S (−1,1) + 1/4 × S (1,1) + 1/4 × S (−1, −1) + 1/4 × S (1, -1) is calculated and output to the weighted addition unit 40.

  Further, when the pixel of interest is a pixel from which the signal value of color G is obtained in the Bayer data, and the left and right pixels of the pixel of interest are pixels from which the signal value of color R is obtained in the Bayer data, The coefficient selection unit A generates a filter coefficient shown in FIG. 12A as a filter coefficient for acquiring the signal value of the color R, and a ( −1, 1) = 0, a (0, 1) = 0, a (1, 1) = 0, a (−1, 0) = 0, a (0, 0) = 1, a (1, 0 ) = 0, a (-1, -1) = 0, a (0, -1) = 0, a (1, -1) = 0, and filter coefficients for obtaining the color B signal value As shown in FIG. 12, except that the filter coefficient shown in FIG. 12B is generated, the signal value of the color R is obtained in the Bayer data for the target pixel. Is the same as in the pixel that.

  Further, when the pixel of interest is a pixel from which the signal value of color G is obtained in the Bayer data, and the left and right pixels of the pixel of interest are pixels from which the signal value of color B is obtained in the Bayer data, The coefficient selection unit A generates a filter coefficient shown in FIG. 13A as a filter coefficient for acquiring the color R signal value, and a ( −1, 1) = 0, a (0, 1) = 0, a (1, 1) = 0, a (−1, 0) = 0, a (0, 0) = 1, a (1, 0 ) = 0, a (-1, -1) = 0, a (0, -1) = 0, a (1, -1) = 0, and filter coefficients for obtaining the color B signal value As a result, except that the filter coefficient shown in FIG. 13B is generated, the signal value of the color R is obtained in the Bayer data for the target pixel. Is the same as in the pixel that.

  When the pixel of interest is a pixel from which the signal value of color B is obtained in the Bayer data, the coefficient selection unit A uses a filter coefficient for acquiring the signal value of color R as shown in FIG. The filter coefficient shown in FIG. 14B is generated as the filter coefficient for acquiring the color G signal value, and the filter coefficient shown in FIG. 14B is generated as the filter coefficient for acquiring the color B signal value. , A (-1, 1) = 0, a (0, 1) = 0, a (1, 1) = 0, a (-1, 0) = 0, a (0, 0) = 1, a ( 1, 0) = 0, a (-1, -1) = 0, a (0, -1) = 0, a (1, -1) = 0, except that the pixel of interest is a Bayer This is the same as the case where the pixel has a signal value of color R in the data.

  In the above description, the signal values of color R, color G, and color B are acquired in series using a set of multipliers M (x, y) and adder P. The signal values of R, color G, and color B may be acquired in parallel using separate multipliers M (x, y) and adder P. In this case, the memory m (x, y) may be configured to share a set of memories m (x, y).

  Here, for simplification of description, the circuit configuration and operation of the second filter and color separation unit 38 have been described, but the circuit configuration and operation of the first filter and color separation unit 38 are also described. , X and y differ only in that they take values of -3, -2, -1, 0, 1, 2, 3 and the other points are the same.

Next, details of the weighting coefficient detection unit 39 will be described. The weighting coefficient detection unit 39 calculates the weighting coefficient α = h (y) of each pixel with respect to the motion amount y of each pixel input from the motion detection unit 32 and outputs this to the weighting addition unit 40. Here, as the function h (y), for example, the following equation 3 and those shown in FIG. 16 are used.

  Equation (3) and the function h (y) shown in FIG. 16 indicate that the weight coefficient α increases from 0 as a linear function as the amount of motion y increases, and when the amount of motion y exceeds the threshold tha, the weight coefficient α This reduces the influence of the first filter and the color separation unit 37 that causes a low-pass spatial low-pass filter to be applied to a pixel with a small amount of motion y. The second filter that operates the spatial low-pass filter and the color separation unit 38 are strengthened, and the first filter that applies the low-frequency spatial low-pass filter to the pixel having a large amount of motion y While the influence of the color separation unit 37 is strengthened, the influence of the second filter and the color separation unit 38 that cause a high-frequency spatial low-pass filter to act is weakened. The threshold value tha may be set to a value equal to the threshold value thA of the equation 2 and the function g (y) shown in FIG.

  In the noise reduction device 30 having the above-described configuration, the first filter and color separation unit 37 and the second filter and color separation unit 38 operate the spatial low-pass filter (LPF) and simultaneously perform color separation. Yes. Therefore, the spatial low-pass filter (LPF) is also applied to a pixel with a large amount of motion for which the noise reduction effect by the three-dimensional noise reduction processing cannot be sufficiently obtained, so that sufficient noise reduction can be performed. . In addition, since the spatial low-pass filter (LPF) is applied and color separation is performed simultaneously, the data amount is reduced compared to the case where the spatial low-pass filter (LPF) is applied for each color, and processing is performed. Can be simplified.

(Second Embodiment of Noise Reduction Device)
Next, a second embodiment of the noise reduction device included in the image processing unit 4 will be described. FIG. 17 is a block diagram showing a schematic configuration of a noise reduction device 50 according to the second embodiment of the present invention. In the noise reduction device 50, the same parts as those of the noise reduction device 30 according to the first embodiment are denoted by the same reference numerals, and detailed description thereof is omitted. The noise reduction device 50 has a configuration different from that of the noise reduction device 30, and the weighting coefficients α (R), α (G), α of each pixel, which are different for each color, from the amount of movement of each pixel input from the motion detection unit 32. (B) is calculated and is output to the weighted addition unit 52; the weighting coefficient detection unit 51; and the first filtered signal L1 (R1, G, B color input from the first filter and color separation unit 37) R), L1 (G), L1 (B) and the second post-filter signals L2 (R), L2 (G), L2 of the R, G, B colors inputted from the second filter and color separation unit 38 (B) is weighted and added for each color using the weighting factors α (R), α (G), and α (B) of each pixel that is input from the weighting factor detection unit 51 and that is different for each color. Output signals O (R), O (G), and O (B) for R, G, and B colors are output to the subsequent stage. O (R) = L1 (R) × α (R) + L2 (R) × (1−α (R)), O (G) = L1 (G) × α (G) + L2 (G) × ( 1- [alpha] (G)), O (B) = L1 (B) * [alpha] (B) + L2 (B) * (1- [alpha] (B))).

  Next, details of the weighting coefficient detection unit 51 will be described. The weighting coefficient detection unit 51 calculates the equation (3) and h (y) shown in FIG. 16 by changing the threshold value tha for each color with respect to the motion amount y of each pixel input from the motion detection unit 32. Thus, different weighting factors α (R), α (G), and α (B) are calculated for each color, and are output to the weighted addition unit 52.

  Specifically, when the threshold value tha for the color R is thR, the threshold value tha for the color G is thG, and the threshold value tha for the color B is thB, it is desirable that thR = thB <thG. According to this, it is possible to suppress the blurring of the color G that has a high contribution to the luminance signal.

(Third Embodiment of Noise Reduction Device)
Next, a third embodiment of the noise reduction device included in the image processing unit 4 will be described. FIG. 18 is a block diagram showing a schematic configuration of a noise reduction device 60 according to the third embodiment of the present invention. In addition, in the noise reduction apparatus 60, the same code | symbol is attached | subjected to the part same as the noise reduction apparatus 30 which concerns on 1st Embodiment, and the detailed description is abbreviate | omitted. The noise reduction device 60 has a configuration different from that of the noise reduction device 30. The signal is input from the subtractor 36, and the horizontal filter and color separation unit 61 outputs the signal to the weighted addition unit 66; the signal is input from the subtractor 36. A vertical filter and color separation unit 62 that outputs a signal to the weighting addition unit 66; a correlation direction detection unit 63 that receives a signal from the motion detection unit 32 and outputs a signal to the second weighting factor detection unit 65; A signal is input from the detection unit 32, and a signal is input from the correlation direction detection unit 63 and the first weighting factor detection unit 64; a first weighting factor detection unit 64 that outputs a signal to the second weighting factor detection unit 65; A second weight coefficient detection unit 65 that outputs a signal to the weighted addition unit 66; a second filter and color separation unit 38, a horizontal filter and color separation unit 61, a vertical filter and color separation unit 6; , And a second signal from the weight coefficient detecting unit 65 is input, the weighted addition unit 66 for outputting a signal to the subsequent stage; and a. Further, the noise reduction device 60 does not have the first filter and color separation unit 37 that the noise reduction device 30 has.

  Here, an outline of the operation of the noise reduction device 60 will be described. First, an external current frame signal (Bayer data) is input to the noise detector 31, the motion detector 32, and the subtractor 36. Further, a signal one frame before the current frame from the frame memory 34 is input to the noise detection unit 31 and the motion detection unit 32. Then, similarly to the first embodiment, the noise reduction processing unit 31 calculates the noise detection value of each pixel from the signal of the current frame input from the outside and the signal of the previous frame input from the frame memory 34. This is calculated and output to the multiplier 35.

  Next, as in the first embodiment, the motion detection unit 32 determines the amount of motion of each pixel from the signal of the current frame input from the outside and the signal of the previous frame input from the frame memory 34. Is output to the feedback coefficient detector 33, the correlation direction detector 63, and the first weight coefficient detector 64. Then, the feedback coefficient detection unit 33 calculates the feedback coefficient k of each pixel from the motion amount of each pixel input from the motion detection unit 32 and outputs this to the multiplier 35 as in the first embodiment. . Further, the correlation direction detection unit 63 calculates the horizontal direction correlation value dv and the vertical direction correlation value dh of each pixel from the amount of motion of each pixel input from the motion detection unit 32, and uses this to detect the second weight coefficient. To the unit 65. Further, the first weight coefficient detection unit 64 calculates the weight coefficient β of each pixel from the motion amount of each pixel input from the motion detection unit 32, and outputs this to the second weight coefficient detection unit 65.

  Next, as in the first embodiment, the multiplier 35 detects the noise detection value of each pixel input from the noise detection unit 31 and the feedback coefficient k of each pixel input from the feedback coefficient detection unit 33. And the obtained subtraction noise value of each pixel is output to the subtractor 36. Then, as in the first embodiment, the subtractor 36 subtracts the subtraction noise value of each pixel input from the multiplier 35 from the signal of the current frame input from the outside, and the obtained three-dimensional noise. The reduced signal is output to the frame memory 34, the second filter and color separation unit 38, the horizontal filter and color separation unit 61, and the vertical filter and color separation unit 62.

  Next, as in the first embodiment, the second filter and color separation unit 38 applies a spatial low-pass filter (LPF) to the three-dimensional noise-reduced signal input from the subtractor 36. At the same time, the color separation is performed, and the obtained second filtered signals L2 (R), L2 (G), and L2 (B) of the respective colors R, G, and B are output to the weighted addition unit 66. In addition, the horizontal filter and color separation unit 61 applies a horizontal low-pass filter (LPF) to the three-dimensional noise-reduced signal input from the subtractor 36 and simultaneously performs color separation. The horizontal filtered signals LH (R), LH (G), and LH (B) for the G and B colors are output to the weighted addition unit 66. Further, the vertical filter and color separation unit 62 applies a low-pass filter (LPF) in the vertical direction to the three-dimensional noise-reduced signal input from the subtractor 36 and simultaneously performs color separation. The vertical filtered signals LV (R), LV (G), and LV (B) for the G and B colors are output to the weighted addition unit 66. In addition, the second weighting coefficient detection unit 65 receives the horizontal direction correlation value dv and the vertical direction correlation value dh of each pixel input from the correlation direction detection unit 63, and each input from the first weighting coefficient detection unit 64. From the pixel weighting factor β, the horizontal weighting factor γ and the vertical weighting factor δ of each pixel are calculated and output to the weighting addition unit 66.

  Next, the weighted addition unit 66 receives the second filtered signals L2 (R), L2 (G), and L2 (B) of the R, G, and B colors input from the second filter and color separation unit 38. And R, G and B horizontal filtered signals LH (R), LH (G) and LH (B) input from the horizontal filter and color separation unit 61 and a vertical filter and color separation unit 62. The vertical filtered signals LV (R), LV (G), and LV (B) of the R, G, and B colors are used as the horizontal weighting factor γ of each pixel input from the second weighting factor detection unit 65. , And weighting addition for each color by the vertical direction weighting coefficient δ (that is, O (R) = L (R) × (1−γ−δ) + LH (R) × γ + LV (R) × δ, O (G) = L (G) * (1- [gamma]-[delta]) + LH (G) * [gamma] + LV (G) * [delta], O (B) = L (B) * (1- [gamma]-[delta]. + LH (B) is a × γ + LV (B) × δ), obtained R, G, B colors of the output signal O (R), O (G), and outputs O and (B) in the subsequent stage.

  Next, details of the horizontal filter and color separation unit 61 will be described. The horizontal filter and color separation unit 61 obtained the color separation by simultaneously applying the horizontal low-pass filter (LPF) to the three-dimensional noise-reduced signal (Bayer data) input from the subtractor 36. The horizontal filtered signals LH (R), LH (G), and LH (B) for the R, G, and B colors are output to the weighted addition unit 66.

  Specifically, for the target pixel from which the signal value of color R is obtained in the Bayer data, the signal value of color G is acquired using the filter shown in FIG. 19A, and FIG. The signal value of color B is acquired using the filter shown, and the signal value obtained in the Bayer data is used as it is for color R.

  In other words, for a pixel of interest for which a color R signal value is obtained in Bayer data, LH (G) = 1/8 × S (−2,1) + 1/8 × S (0,1) + 1/8 × S (2,1) + 1/8 × S (−1,0) + 1/8 × S (1,0) + 1/8 × S (−2, −1) + 1/8 × S (0, −1) The signal value of the color G is acquired by + 1/8 × S (2, −1).

  Further, for a target pixel for which a color R signal value is obtained in Bayer data, LH (B) = 1/8 × S (−3,1) + 1/8 × S (−1,1) +1/8. × S (1,1) + 1/8 × S (3,1) + 1/8 × S (−3, −1) + 1/8 × S (−1, −1) + 1/8 × S (1, − 1) The signal value of color B is acquired by + 1/8 × S (3, −1).

  Among the target pixels whose color G signal value is obtained in the Bayer data, those in which the left and right pixels are pixels whose color R signal value is obtained in the Bayer data are shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 20B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among the target pixels whose color G signal value is obtained in the Bayer data, LH (R) = 1 for pixels in which the left and right pixels are pixels whose color R signal value is obtained in the Bayer data / 16 × S (−1,2) + 1/16 × S (1,2) + 1/8 × S (−3,0) + 1/4 × S (−1,0) + 1/4 × S (1, 0) + 1/8 × S (3,0) + 1/16 × S (−1, −2) + 1/16 × S (1, −2) is obtained as the signal value of color R.

  Of the target pixels for which the color G signal value is obtained in the Bayer data, LH (B) = 1 for the pixels in which the left and right pixels are for which the color R signal value is obtained in the Bayer data / 8 × S (−2,1) + 1/4 × S (0,1) + 1/8 × S (2,1) + 1/8 × S (−2, −1) + 1/4 × S (0, -1) The signal value of color B is acquired by + 1/8 × S (2, -1).

  Of the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are pixels for which the color B signal value is obtained in the Bayer data, as shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 21B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among the target pixels whose color G signal value is obtained in the Bayer data, LH (R) = 1 for pixels in which the left and right pixels are pixels whose color B signal value is obtained in the Bayer data / 8 × S (−2,1) + 1/4 × S (0,1) + 1/8 × S (2,1) + 1/8 × S (−2, −1) + 1/4 × S (0, -1) The signal value of the color R is acquired by + 1/8 × S (2, -1).

  Of the target pixels for which the color G signal value is obtained in the Bayer data, LH (B) = 1 for pixels in which the left and right pixels are those for which the color B signal value is obtained in the Bayer data / 16 × S (−1,2) + 1/16 × S (1,2) + 1/8 × S (−3,0) + 1/4 × S (−1,0) + 1/4 × S (1, 0) + 1/8 × S (3,0) + 1/16 × S (−1, −2) + 1/16 × S (1, −2) is obtained as the signal value of color B.

  For the target pixel from which the signal value of color B is obtained in the Bayer data, the signal value of color R is obtained using the filter shown in FIG. 22A, and the filter shown in FIG. Is used to acquire the signal value of the color G, and for the color B, the signal value obtained in the Bayer data is used as it is.

  That is, for a target pixel for which a color B signal value is obtained in Bayer data, LH (R) = 1/8 × S (−3, 1) + 1/8 × S (−1, 1) +1/8. × S (1,1) + 1/8 × S (3,1) + 1/8 × S (−3, −1) + 1/8 × S (−1, −1) + 1/8 × S (1, − 1) The signal value of the color R is acquired by + 1/8 × S (3, −1).

  Further, for a target pixel for which a signal value of color B is obtained in Bayer data, LH (G) = 1/8 × S (−2,1) + 1/8 × S (0,1) + 1/8 × S (2,1) + 1/8 × S (−1,0) + 1/8 × S (1,0) + 1/8 × S (−2, −1) + 1/8 × S (0, −1) The signal value of the color G is acquired by + 1/8 × S (2, −1).

  The circuit configuration and operation of the horizontal filter and color separation unit 61 are substantially the same as the circuit configuration and operation of the second filter and color separation unit 38 described above (x and y are −3, − 2), -1, 0, 1, 2, and 3), and the description thereof will be omitted.

  Next, details of the vertical filter and color separation unit 62 will be described. The vertical filter and color separation unit 62 applies a low-pass filter (LPF) in the vertical direction to the three-dimensional noise-reduced signal (Bayer data) input from the subtractor 36, and at the same time performs color separation. The R, G, and B vertical filtered signals LV (R), LV (G), and LV (B) are output to the weighted addition unit 66.

  Specifically, for the target pixel from which the signal value of color R is obtained in the Bayer data, the signal value of color G is acquired using the filter shown in FIG. The signal value of color B is acquired using the filter shown, and the signal value obtained in the Bayer data is used as it is for color R.

  That is, for a target pixel for which a color R signal value is obtained in Bayer data, LV (G) = 1/8 × S (−1,2) + 1/8 × S (1,2) + 1/8 × S (0,1) + 1/8 × S (−1,0) + 1/8 × S (1,0) + 1/8 × S (0, −1) + 1/8 × S (−1, −2) The signal value of the color G is acquired by + 1/8 × S (1, -2).

  In addition, with respect to the pixel of interest for which the color R signal value is obtained in the Bayer data, LV (B) = 1/8 × S (−1,3) + 1/8 × S (1,3) + 1/8 × S (-1,1) + 1/8 * S (1,1) + 1/8 * S (-1, -1) + 1/8 * S (1, -1) + 1/8 * S (-1,- 3) The signal value of color B is acquired by + 1/8 × S (1, -3).

  Among the target pixels whose color G signal value is obtained in the Bayer data, those in which the left and right pixels are pixels whose color R signal value is obtained in the Bayer data are shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 24B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are pixels for which the color R signal value is obtained in the Bayer data, LV (R) = 1. / 8 × S (−1,2) + 1/8 × S (1,2) + 1/4 × S (−1,0) + 1/4 × S (1,0) + 1/8 × S (−1, -2) The signal value of color R is acquired by + 1/8 × S (1, -2).

  Of the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are those for which the color R signal value is obtained in the Bayer data, and LV (B) = 1. / 8 × S (0,3) + 1/16 × S (−2,1) + 1/4 × S (0,1) + 1/16 × S (2,1) + 1/16 × S (−2, − 1) The signal value of color B is acquired by + 1/4 × S (0, −1) + 1/16 × S (2, −1) + 1/8 × S (0, -3).

  Among the target pixels whose color G signal value is obtained in the Bayer data, those in which the left and right pixels are pixels whose color B signal value is obtained in the Bayer data are shown in FIG. The signal value of color R is obtained using the filter shown, the signal value of color B is obtained using the filter shown in FIG. 25B, and the signal value obtained in the Bayer data is obtained for color G. It will be used as it is.

  That is, among the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are pixels for which the color B signal value is obtained in the Bayer data, LV (R) = 1. / 8 × S (0,3) + 1/16 × S (−2,1) + 1/4 × S (0,1) + 1/16 × S (2,1) + 1/16 × S (−2, − 1) The signal value of the color R is acquired by + 1/4 × S (0, −1) + 1/16 × S (2, −1) + 1/8 × S (0, -3).

  Of the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are pixels for which the color B signal value is obtained in the Bayer data, and LV (B) = 1. / 8 × S (−1,2) + 1/8 × S (1,2) + 1/4 × S (−1,0) + 1/4 × S (1,0) + 1/8 × S (−1, -2) The signal value of color B is acquired by + 1/8 * S (1, -2).

  For the pixel of interest for which the signal value of color B is obtained in the Bayer data, the signal value of color R is obtained using the filter shown in FIG. 26A, and the filter shown in FIG. Is used to acquire the signal value of the color G, and for the color B, the signal value obtained in the Bayer data is used as it is.

  That is, for a target pixel for which a color B signal value is obtained in Bayer data, LV (R) = 1/8 × S (−1,3) + 1/8 × S (1,3) + 1/8 × S (-1,1) + 1/8 * S (1,1) + 1/8 * S (-1, -1) + 1/8 * S (1, -1) + 1/8 * S (-1,- 3) The signal value of color R is acquired by + 1/8 × S (1, -3).

  For the target pixel for which the color B signal value is obtained in the Bayer data, LV (G) = 1/8 × S (−1,2) + 1/8 × S (1,2) + 1/8 × S (0,1) + 1/8 × S (−1,0) + 1/8 × S (1,0) + 1/8 × S (0, −1) + 1/8 × S (−1, −2) The signal value of the color G is acquired by + 1/8 × S (1, -2).

  Note that the circuit configuration and operation of the vertical filter and color separation unit 62 are almost the same as the circuit configuration and operation of the second filter and color separation unit 38 described above (x and y are −3, − 2), -1, 0, 1, 2, and 3), and the description thereof will be omitted.

  Next, details of the correlation direction detection unit 63 will be described. In the following, the coordinates of the pixel of interest are (0, 0), the amount of movement is V (0, 0), and the coordinates of the pixel at the position of the x pixel to the right and the y pixel above the pixel of interest are y) The amount of movement is represented as V (x, y). That is, the coordinates of the pixel located two pixels to the right and two pixels above the target pixel are represented by (2, 2), and the amount of movement thereof is represented by V (2, 2). The coordinates of the pixel at the position of 2 pixels are represented as (−2, −2), and the amount of movement is represented as V (−2, −2).

  The correlation direction detection unit 63 calculates the horizontal direction correlation value dv of each pixel by applying a vertical gradient detection filter to the movement amount of each pixel input from the motion detection unit 32, and also detects motion. The vertical direction correlation value dh of each pixel is calculated by applying a horizontal direction gradient detection filter to the movement amount of each pixel input from the unit 32, and this is output to the second weight coefficient detection unit 65. Is.

  Specifically, for the amount of movement of each pixel, the horizontal correlation value dv is calculated using the filter shown in FIG. 27A, and the vertical correlation value dh is calculated using the filter shown in FIG. Is calculated. That is, dv = V (−1,1) + V (0,1) + V (1,1) −V (−1, −1) −V (0, −1) −V ( 1, −1), the horizontal correlation value dv is calculated, and dh = V (−1,1) + V (−1,0) + V (−1, −1) −V (1,1) −V (1 , 0) −V (1, −1), the vertical correlation value dh is calculated.

  The circuit configuration and operation of the correlation direction detection unit 63 are almost the same as the circuit configuration and operation of the second filter and color separation unit 38 described above (each memory m (x, y) is a signal). The motion amount V (x, y) is stored and output instead of the value S (x, y), and the coefficient selection unit A does not generate a filter coefficient for acquiring the signal value of each color. The only difference is that filter coefficients for calculating the direction correlation value dv and the vertical direction correlation value dh are generated), and the description thereof will be omitted.

  Next, the details of the first weight coefficient detection unit 64 will be described. The first weight coefficient detection unit 64 calculates the equation (3) and h (y) shown in FIG. 16 with respect to the motion amount y of each pixel input from the motion detection unit 32, thereby obtaining each pixel's motion amount y. The weighting factor β is calculated and output to the second weighting factor detector 65.

Next, the details of the second weight coefficient detection unit 65 will be described. The second weight coefficient detection unit 65 receives the horizontal direction correlation value dv and the vertical direction correlation value dh of each pixel input from the correlation direction detection unit 63, and each pixel input from the first weight coefficient detection unit 64. With respect to the weighting coefficient β, the horizontal weighting coefficient γ and the vertical weighting coefficient δ of each pixel are calculated using the following formula 4, and output to the weighting addition unit 66.

  At this time, the horizontal weighting factor γ increases as the weighting factor β and the horizontal correlation value dv increase. The weight coefficient β increases as the amount of motion increases, and the horizontal direction correlation value dv increases when the amount of motion has a strong correlation in the horizontal direction, so that the amount of motion is large and the amount of motion is horizontal. When there is a strong correlation in the direction, the influence of the horizontal filter and color separation unit 61 suitable for this case is strengthened.

  Further, the vertical weighting factor δ increases as the weighting factor β and the vertical correlation value dh increase. The weight coefficient β increases as the amount of motion increases, and the vertical correlation value dh increases when the amount of motion has a strong correlation in the vertical direction, so that the amount of motion is large and the amount of motion is vertical. When there is a strong correlation in the direction, the influence of the vertical filter and the color separation unit 62 suitable for this case is strengthened.

(Fourth Embodiment of Noise Reduction Device)
Next, a fourth embodiment of the noise reduction device included in the image processing unit 4 will be described. FIG. 28 is a block diagram showing a schematic configuration of a noise reduction apparatus 70 according to the fourth embodiment of the present invention. Note that, in the noise reduction device 70, the same parts as those of the noise reduction device 30 according to the first embodiment and the noise reduction device 60 according to the third embodiment are denoted by the same reference numerals, and detailed description thereof will be given. Omitted. The noise reduction device 70 is configured differently from the noise reduction device 30 and the noise reduction device 60, and receives a signal from the correlation direction detection unit 63 and the first weight coefficient detection unit 64 and outputs a signal to the weighting addition unit 72. A second weight coefficient detection unit 71; a first filter and color separation unit 37, a second filter and color separation unit 38, a horizontal filter and color separation unit 61, a vertical filter and color separation unit 62, and a second weight A weighted addition unit 72 that receives a signal from the coefficient detection unit 71 and outputs the signal to a subsequent stage.

  Here, an outline of the operation of the noise reduction device 70 will be described. First, an external current frame signal (Bayer data) is input to the noise detector 31, the motion detector 32, and the subtractor 36. Further, a signal one frame before the current frame from the frame memory 34 is input to the noise detection unit 31 and the motion detection unit 32. Then, as in the first embodiment, the noise reduction processing unit 31 calculates the noise detection value of each pixel from the signal of the current frame input from the outside and the signal of the previous frame input from the frame memory 34. This is calculated and output to the multiplier 35.

  Next, as in the first embodiment, the motion detection unit 32 determines the amount of motion of each pixel from the signal of the current frame input from the outside and the signal of the previous frame input from the frame memory 34. Is output to the feedback coefficient detector 33, the correlation direction detector 63, and the first weight coefficient detector 64. Then, the feedback coefficient detection unit 33 calculates the feedback coefficient k of each pixel from the motion amount of each pixel input from the motion detection unit 32 and outputs this to the multiplier 35 as in the first embodiment. . Similarly to the third embodiment, the correlation direction detection unit 63 calculates the horizontal direction correlation value dv and the vertical direction correlation value dh of each pixel from the amount of movement of each pixel input from the motion detection unit 32. This is output to the second weighting coefficient detector 71. Also, the first weight coefficient detection unit 64 calculates the weight coefficient β of each pixel from the amount of motion of each pixel input from the motion detection unit 32, as in the third embodiment. The data is output to the weight coefficient detection unit 71.

  Next, as in the first embodiment, the multiplier 35 detects the noise detection value of each pixel input from the noise detection unit 31 and the feedback coefficient k of each pixel input from the feedback coefficient detection unit 33. And the obtained subtraction noise value of each pixel is output to the subtractor 36. Then, as in the first embodiment, the subtractor 36 subtracts the subtraction noise value of each pixel input from the multiplier 35 from the signal of the current frame input from the outside, and the obtained three-dimensional noise. The reduced signal is output to the frame memory 34, the first filter and color separation unit 37, the second filter and color separation unit 38, the horizontal filter and color separation unit 61, and the vertical filter and color separation unit 62.

  Next, the first filter and color separation unit 37 applies a spatial low-pass filter (LPF) to the three-dimensional noise-reduced signal input from the subtractor 36 as in the first embodiment. At the same time, the color separation is performed, and the obtained first filtered signals L1 (R), L1 (G), and L1 (B) of the respective colors R, G, and B are output to the weighted adder 72. In addition, the second filter and color separation unit 38 is more effective than the first filter and color separation unit 37 with respect to the three-dimensional noise-reduced signal input from the subtractor 36, as in the first embodiment. A high-pass spatial low-pass filter (LPF) is applied and color separation is performed at the same time, and the obtained second filtered signals L2 (R), L2 (G), and L2 (B) for the respective colors R, G, and B are obtained. Is output to the weighted addition unit 72.

  Similarly to the third embodiment, the horizontal filter and color separation unit 61 applies a horizontal low-pass filter (LPF) to the three-dimensional noise-reduced signal input from the subtractor 36, and at the same time performs color Separation is performed, and the obtained horizontal filtered signals LH (R), LH (G), and LH (B) for the respective colors R, G, and B are output to the weighted adder 72. Similarly to the third embodiment, the vertical filter and color separation unit 62 applies a low-pass filter (LPF) in the vertical direction to the three-dimensional noise-reduced signal input from the subtractor 36 and at the same time performs color Separation is performed, and the obtained vertical filtered signals LV (R), LV (G), and LV (B) for the respective colors R, G, and B are output to the weighted adder 72. Further, the second weighting coefficient detection unit 71 receives the horizontal direction correlation value dv and the vertical direction correlation value dh of each pixel input from the correlation direction detection unit 63, and each input from the first weighting coefficient detection unit 64. From the pixel weighting factor β, a first weighting factor ε, a horizontal weighting factor γ, and a vertical weighting factor δ for each pixel are calculated and output to the weighting addition unit 72.

  Next, the weighted adder 72 receives the first filtered signals L1 (R), L1 (G), and L1 (B) for the R, G, and B colors input from the first filter and the color separator 37. The second filtered signals L2 (R), L2 (G), and L2 (B) of R, G, and B colors input from the second filter and color separation unit 38, and the horizontal filter and color separation unit 61 The R, G, B horizontal filtered signals LH (R), LH (G), LH (B) inputted by the color filter and the vertical color of the R, G, B colors inputted by the vertical filter and color separation unit 62 are inputted. The filtered signals LV (R), LV (G), and LV (B) are input to the first weighting coefficient ε of each pixel input from the second weighting coefficient detection unit 71, the horizontal weighting coefficient γ, and the vertical Weighting is added for each color by the direction weighting coefficient δ (that is, O (R) = L1 (R) × ε + LH (R) × γ + LV (R) × δ + L2 (R) × (1−γ−δ−ε), O (G) = L1 (G) × ε + LH (G) × γ + LV (G) × δ + L2 (G) × (1-γ-δ-ε), O (B) = L1 (B) × ε + LH (B) × γ + LV (B) × δ + L2 (B) × (1-γ-δ-ε)) Output signals O (R), O (G), and O (B) for R, G, and B colors are output to the subsequent stage.

  Next, details of the second weight coefficient detection unit 71 will be described. If the sum of the horizontal direction correlation value dv and the vertical direction correlation value dh is smaller than the threshold Thξ for each pixel, the second weighting coefficient detection unit 71 sets the first weighting coefficient ε = β and the horizontal direction weighting coefficient γ = If the sum of the horizontal direction correlation value dv and the vertical direction correlation value dh is equal to or greater than the threshold Thξ, the first weighting coefficient ε = 0 and the horizontal direction weighting coefficient is calculated. γ and the vertical weighting coefficient δ are calculated using Equation 4 and output to the weighting addition unit 72.

  At this time, if the sum of the horizontal correlation value dv and the vertical correlation value dh is smaller than the threshold Thξ, that is, the sum of the horizontal correlation strength of the motion amount and the vertical correlation strength of the motion amount. Is small, the effects of the first filter and color separation unit 37 and the second filter and color separation unit 38 are enhanced, while the horizontal filter and color separation unit 61 are suitable. The influence of the vertical filter and color separation unit 62 is eliminated. If the sum of the horizontal correlation value dv and the vertical correlation value dh is larger than the threshold Thξ, that is, the sum of the horizontal correlation strength of the motion amount and the vertical correlation strength of the motion amount is If it is large, the effects of the horizontal filter and color separation unit 61 and the vertical filter and color separation unit 62 are strengthened as appropriate in this case, while the influence of the first filter and color separation unit 37 is eliminated. The influence of the second filter and color separation unit 38 is weakened.

  In the above description, the noise reduction method according to the present invention has been described by taking the image pickup apparatus having the configuration shown in FIG. 1 as an example. However, the present invention is not limited to the image pickup apparatus. The noise reduction method according to the present invention can also be used in a display device that performs processing. FIG. 29 shows a display device including a noise reduction device (hereinafter, included in “image processing unit”) that performs the noise reduction method of the present invention.

  The display device shown in FIG. 29 includes an image processing unit 4, a display 9, a TG 12, a CPU 13, a memory 14, an operation unit 15, and bus lines 16 and 17, similar to the imaging device shown in FIG. A tuner unit 21 that selects a broadcast signal received externally, a demodulator unit 22 that demodulates a broadcast signal selected by the tuner unit 21, and an interface to which a compressed signal that is a digital signal input from the outside is input. 23.

  29, when receiving a broadcast signal, the tuner unit 21 selects a broadcast signal of a desired channel, and then the demodulation unit 22 demodulates the broadcast signal to obtain an image signal.

  When the display of the image is instructed by the operation unit 15, the image signal obtained by the demodulation unit 22 is given to the image processing unit 4, and various image processing including the above-described noise reduction processing and color separation processing is performed. After that, the obtained display signal is given to the display 9, and an image is displayed. Although omitted in FIG. 29, it is also possible to adopt a configuration that enables audio output by including the audio output circuit unit 10, the speaker 11, and the like, as in the imaging apparatus shown in FIG.

  In addition, this invention is not limited to said embodiment, A various change is possible in the range which does not deviate from the main point of this invention. For example, in the third and fourth embodiments described above, the first weighting factor ε, the horizontal weighting factor γ, and the vertical weighting factor δ of each pixel are common to the colors R, G, and B. However, it may be different for each color as in the second embodiment. In the above embodiment, the RAW image signal is Bayer data, but is not limited to this. In the above embodiment, the color filter of the image sensor 1 is a primary color filter, but may be a complementary color filter.

  INDUSTRIAL APPLICABILITY The present invention is effective as a noise reduction method and a noise reduction device that reduce noise in a moving image composed of a plurality of frames. The present invention is also effective as an electronic device equipped with this noise reduction device.

The block diagram which shows the internal structure of an imaging device. FIG. 3 is an explanatory diagram showing the arrangement of filters installed in the image sensor 1. FIG. 2 is a block diagram showing a schematic configuration of a noise reduction device 30. The graph which shows function f (x). FIG. 3 is a block diagram showing a schematic configuration of a motion detection unit 32. The graph which shows the function g (y). FIG. 6 is an explanatory diagram showing a filter used for a pixel of interest for which a color R signal value is obtained in Bayer data in the first filter and color separation unit 37. In the first filter and color separation unit 37, among the target pixels whose color G signal value is obtained in the Bayer data, the left and right pixels are pixels whose color R signal value is obtained in the Bayer data Explanatory drawing which shows the filter used for a pixel. In the first filter and color separation unit 37, among the pixels of interest whose color G signal value is obtained in the Bayer data, the left and right pixels are pixels whose color B signal value is obtained in the Bayer data Explanatory drawing which shows the filter used for a pixel. FIG. 6 is an explanatory diagram showing a filter used for a pixel of interest for which a color B signal value is obtained in Bayer data in the first filter and color separation unit 37. FIG. 10 is an explanatory diagram showing a filter used for a pixel of interest for which a color R signal value is obtained in Bayer data in the second filter and color separation unit. In the second filter and color separation unit 38, among the pixels of interest whose color G signal value is obtained in the Bayer data, the left and right pixels are pixels whose color R signal value is obtained in the Bayer data Explanatory drawing which shows the filter used for a pixel. In the second filter and color separation unit 38, among the pixels of interest whose color G signal value is obtained in the Bayer data, the left and right pixels are pixels whose color B signal value is obtained in the Bayer data Explanatory drawing which shows the filter used for a pixel. FIG. 7 is an explanatory diagram showing a filter used for a pixel of interest for which a color B signal value is obtained in Bayer data in the second filter and color separation unit. The block diagram which shows the circuit structure of the 2nd filter and the color separation part 38. FIG. The graph which shows the function h (y). FIG. 2 is a block diagram showing a schematic configuration of a noise reduction device 50. FIG. 2 is a block diagram showing a schematic configuration of a noise reduction device 60. FIG. 6 is an explanatory diagram showing a filter used for a pixel of interest for which a color R signal value is obtained in Bayer data in the horizontal filter and color separation unit 61. In the horizontal filter and color separation unit 61, among the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are the target pixels for which the color R signal value is obtained in the Bayer data. Explanatory drawing which shows the filter used. In the horizontal filter and color separation unit 61, among the target pixels for which the color G signal value is obtained in the Bayer data, the left and right pixels are the target pixels for which the color B signal value is obtained in the Bayer data. Explanatory drawing which shows the filter used. FIG. 6 is an explanatory diagram showing a filter used for a pixel of interest for which a color B signal value is obtained in Bayer data in the horizontal filter and color separation unit 61. 4 is an explanatory diagram showing a filter used for a pixel of interest for which a color R signal value is obtained in Bayer data in the vertical filter and color separation unit 62. FIG. In the vertical filter and color separation unit 62, among the target pixels whose color G signal value is obtained in the Bayer data, the left and right pixels are the target pixels whose color R signal value is obtained in the Bayer data. Explanatory drawing which shows the filter used. In the vertical filter and color separation unit 62, among the target pixels whose color G signal value is obtained in the Bayer data, the left and right pixels are the target pixels whose color B signal value is obtained in the Bayer data. Explanatory drawing which shows the filter used. FIG. 7 is an explanatory diagram showing a filter used for a pixel of interest for which a color B signal value is obtained in Bayer data in the vertical filter and color separation unit 62. Explanatory drawing which shows a vertical direction gradient detection filter and a horizontal direction gradient detection filter. FIG. 2 is a block diagram showing a schematic configuration of a noise reduction device 70. The block diagram which shows the internal structure of a display apparatus.

Explanation of symbols

30, 50, 60, 70 Noise reduction device 32 Motion detection unit 37 First filter and color separation unit (filter and color separation unit)
38 Second filter and color separation unit (filter and color separation unit, spatial filter and color separation unit)
39, 51 Weight coefficient detection unit 40, 52, 66, 72 Weighting addition unit 61 Horizontal filter and color separation unit (filter and color separation unit)
62 Vertical filter and color separation unit (filter and color separation unit)
63 Correlation direction detector 64 First weight coefficient detector 65, 71 Second weight coefficient detector

Claims (7)

A noise reduction device having a three-dimensional noise reduction processing unit that performs a three-dimensional noise reduction process on a RAW image signal input from the outside and obtains a signal after three-dimensional noise reduction,
A plurality of noise reduction processing and color separation units that simultaneously perform noise reduction processing and color separation on the three-dimensional noise reduction signal input from the three-dimensional noise reduction processing unit to obtain a filtered signal of each color;
A mixing unit that mixes the post-filter signals of each color input from a plurality of the noise reduction processing and color separation units, and obtains an output signal of each color;
A noise reduction device comprising: A noise reduction device having a three-dimensional noise reduction processing unit that performs a three-dimensional noise reduction process on a RAW image signal input from the outside and obtains a signal after three-dimensional noise reduction,
A plurality of filters and a color separation unit that perform a color separation on the three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit and simultaneously perform color separation to obtain a filtered signal of each color;
A mixing unit that mixes the post-filter signals of each color input from the plurality of filters and color separation units, and obtains an output signal of each color;
A noise reduction device comprising: The plurality of filters and color separation units are
A spatial low-pass filter of a predetermined band is applied to the post-three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit, and color separation is performed simultaneously to obtain a first post-filter signal for each color. A first filter and a color separation unit;
A spatial low-pass filter having a frequency higher than the predetermined band is applied to the post-three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit, and color separation is performed simultaneously. A second filter and color separation unit for obtaining a filtered signal;
Composed by
The mixing unit includes the first post-filter signal of each color input from the first filter and the color separation unit, and the second post-filter signal of each color input from the second filter and the color separation unit. Are weighted and added by a predetermined weighting factor for each color to obtain the output signal of each color,
A motion detection unit that obtains a motion amount in the time axis direction of each pixel by calculating a difference value between the RAW image signal input from the outside and the three-dimensional noise-reduced signal one frame before;
A weighting factor detection unit that calculates the predetermined weighting factor according to the amount of motion in the time axis direction of each pixel input from the motion detection unit;
The noise reduction device according to claim 2, comprising: The plurality of filters and color separation units are
A spatial filter and a color separation for obtaining a post-spatial signal of each color by applying a spatial low-pass filter to the post-three-dimensional noise reduction signal input from the three-dimensional noise reduction processing unit and simultaneously performing color separation. And
A horizontal filter and a color separation that obtains a signal after horizontal filtering for each color by applying a low-pass filter in the horizontal direction and simultaneously performing color separation on the signal after three-dimensional noise reduction input from the three-dimensional noise reduction processing unit And
A vertical filter and a color separation that obtains a vertical filtered signal for each color by applying a low-pass filter in the vertical direction and simultaneously performing color separation on the post-three-dimensional noise-reduced signal input from the three-dimensional noise reduction processing unit And
Composed by
The mixing unit includes the spatial filtered signal for each color input from the spatial filter and the color separating unit, the horizontal filtered signal for each color input from the horizontal filter and the color separating unit, the vertical filter and the color. The vertical filtered signal of each color input from the separation unit is weighted and added by a predetermined weighting factor for each color to obtain the output signal of each color,
A motion detection unit that obtains a motion amount in the time axis direction of each pixel by calculating a difference value between the RAW image signal input from the outside and the three-dimensional noise-reduced signal one frame before;
A correlation direction detection unit that calculates a horizontal direction correlation value and a vertical direction correlation value of the motion amount in the time axis direction of each pixel input from the motion detection unit;
The horizontal direction correlation value and the vertical direction correlation of the motion amount in the time axis direction of each pixel input from the motion detection unit and the motion amount in the time axis direction of each pixel input from the correlation direction detection unit. A weighting factor detector that calculates the predetermined weighting factor according to a value;
The noise reduction device according to claim 2, comprising:   The noise reduction apparatus according to claim 3 or 4, wherein the predetermined weight coefficient is different for each color. A noise reduction method including a three-dimensional noise reduction processing step of performing a three-dimensional noise reduction process on a RAW image signal constituting a moving image to obtain a signal after three-dimensional noise reduction,
A plurality of noise reduction processing and color separation steps for simultaneously performing noise reduction processing and color separation on the three-dimensional noise reduction signal obtained in the three-dimensional noise reduction processing step to obtain a filtered signal for each color;
A mixing step of mixing the post-filter signals of each color obtained in a plurality of the noise reduction processing and the color separation step to obtain an output signal of each color;
A noise reduction method characterized by comprising: A RAW image signal constituting a moving image is provided by external input or imaging, and an electronic device including a noise reduction device that reduces noise of the moving image,
The electronic apparatus according to claim 1, wherein the noise reduction device is the noise reduction device according to claim 1.

Images