JP4901625B2 - Noise reduction device and noise reduction method - Google Patents

Description

  The present invention relates to a technique for reducing noise in moving images.

  As a method for reducing noise in moving images, a method of temporally smoothing a moving image using a recursive filter or a non-recursive filter, or a method of spatially smoothing an image for each frame is used. . When temporal smoothing processing is performed on a moving image, an afterimage occurs in a portion where there is motion in the moving image. Therefore, in order to suppress the occurrence of afterimages, it is preferable to apply spatial smoothing to the image for each frame.

JP-A-6-169920 JP-A-9-322020

  However, blurring usually occurs in an image subjected to spatial smoothing. Therefore, when noise reduction processing by spatial smoothing is performed, the image quality of a moving image after noise reduction may be reduced due to the occurrence of blurring.

  The present invention has been made in order to solve the above-described conventional problems, and an object of the present invention is to provide a technique for suppressing a decrease in image quality due to spatial smoothing in a noise reduction process for moving images.

  SUMMARY An advantage of some aspects of the invention is to solve at least a part of the problems described above, and the invention can be implemented as the following forms or application examples.

[Application Example 1]
A noise reduction device for reducing noise of a moving image, wherein a motion detection unit detects a moving region having motion in the moving image and a static region other than the moving region, and an input constituting the moving image A spatial smoothing processing unit that performs a spatial smoothing process on the frame image to generate a spatial smoothed frame image, and the spatial smoothing processing unit performs the spatial smoothing for each pixel constituting the input frame image. The degree of the spatial smoothing process in the pixels belonging to the static area is higher than the degree of the spatial smoothing process in the pixels belonging to the moving area. Noise reduction device set to

  According to this application example, the spatial smoothing processing unit that performs spatial smoothing processing on the input frame image is configured to be able to set the degree of spatial smoothing processing for each pixel constituting the input frame image. Then, the degree of the spatial smoothing process in the static area pixel is set to be higher than the degree of the spatial smoothing process in the moving area pixel. For this reason, it is possible to further suppress the blur effect due to the smoothing process for the moving area where the influence of noise is small, and to further reduce the noise for the static area where the influence of noise is large. Therefore, it is possible to suppress the deterioration of the image quality due to the blur and to reduce the influence of noise satisfactorily, so that the quality of the entire moving image can be improved.

[Application Example 2]
The noise reduction device according to Application Example 1, wherein the spatial smoothing processing unit includes a spatial smoothing filter that performs a spatial smoothing process on the input frame image, and a smooth frame image that has passed through the spatial smoothing filter. A non-smooth frame image that does not pass through the spatial smoothing filter, and a mixing processing unit that mixes the smoothing frame image and the non-smooth frame image for each pixel. A noise reduction device that can be set.

  According to this application example, the degree of spatial smoothing is changed by setting a mixing ratio between a smooth frame image that has passed through the spatial smoothing filter and a non-smooth frame image that has not passed through the spatial smoothing filter. Can do. Therefore, the setting of the degree of spatial smoothing becomes easier.

  Note that the present invention can be realized in various modes. For example, noise reduction device and noise reduction method, video camera to which the noise reduction device and method are applied, video camera control device and control method, noise reduction device and method thereof, video camera, video camera control device and control method, Can be realized in the form of a computer program for realizing the above, a recording medium storing the computer program, a data signal including the computer program and embodied in a carrier wave, and the like.

Next, embodiments of the present invention will be described in the following order based on examples.
A. First embodiment:
B. Second embodiment:
C. Third embodiment:
D. Variations:

A. First embodiment:
FIG. 1 is a schematic configuration diagram showing a schematic configuration of a video camera 10 as an embodiment of the present invention. The video camera 10 includes an imaging lens 100, an image sensor 200, a pre-processing unit 300, a noise reduction unit 400, a post-processing unit 500, and a video signal generation unit 600.

  The imaging lens 100 forms an image of the subject on the imaging surface 202 by condensing light from the subject on the imaging surface 202 of the image sensor 200. Note that the imaging lens 100 normally includes a plurality of single lenses and a diaphragm mechanism, but is illustrated as one single lens in FIG.

  The image sensor 200 includes a color filter 210 and an image sensor 220 provided with a plurality of sensor elements on the imaging surface 202 side. In the color filter 210, a plurality of RGB primary color filters are formed corresponding to the plurality of sensor elements provided in the image sensor 220. The RGB primary color filters of the color filter 210 are arranged in a checkered pattern (referred to as a “Bayer array”). The image of the subject formed on the imaging surface 202 is decomposed into three color lights of RGB by the primary color filter. Then, the separated color light is incident on a sensor element provided in the image sensor 220.

  Each sensor element of the image sensor 220 generates and accumulates charges according to the amount of light incident on the sensor element. An electrical signal (image signal) representing the amount of charge accumulated in each sensor element is amplified by an amplifier (not shown). The amplified image signal is converted into digital data by an A-D converter (not shown) to generate image data BD1. In the video camera 10, image data BD1 for one screen (frame) is periodically generated.

  As described above, the color light separated by the Bayer array color filter 210 is incident on each sensor element. Therefore, the image data BD1 generated by the image sensor 220 is data (Bayer data) in which pixels corresponding to individual sensor elements have color component values of only one of RGB. As the image sensor 220, for example, a CMOS image sensor in which an amplifier and an A-D converter are incorporated, or a CCD to which an amplifier and an A-D converter (also referred to as “analog front end”) are attached. An image sensor can be used.

  The image data BD1 generated by the image sensor 220 is supplied to the preprocessing unit 300. The pre-processing unit 300 generates image data BD2 by performing processing such as clamping processing and white balance adjustment processing on the image data BD1 supplied from the imaging element 220. In the processing in the preprocessing unit 300, processing is separately performed on each of the RGB color component values. Therefore, the image data BD2 generated by the preprocessing unit 300 is also Bayer data.

  The image data BD2 generated by the preprocessing unit 300 is supplied to the noise reduction unit 400. The noise reduction unit 400 generates image data BD3 by performing noise reduction processing on the image data BD2 supplied from the preprocessing unit 300. Also in the noise reduction unit 400, noise reduction processing is separately performed on each of the RGB color component values. Therefore, the image data BD3 generated by the noise reduction unit 400 is also Bayer data. A specific configuration and function of the noise reduction unit 400 will be described later.

  The image data BD3 generated by the noise reduction unit 400 is supplied to the post-processing unit 500. The post-processing unit 500 includes an interpolation processing unit 510 for performing interpolation processing. The interpolation processing unit 510 generates missing color component values for each pixel of the image data BD3. Specifically, for each pixel of the image data BD3, the color component value missing from that pixel is interpolated and generated from the color component values of surrounding pixels. Thereby, the image data subjected to the interpolation processing becomes image data in which each pixel has three color component values of RGB. The post-processing unit 500 performs processing such as color adjustment processing (contrast adjustment processing and color correction processing) on the image data on which the interpolation processing has been performed to generate image data IMG. Note that the content of the color adjustment processing does not affect the present invention, and therefore the description thereof is omitted here.

  The image data IMG generated by the post-processing unit 500 is supplied to the video signal generation unit 600. The video signal generation unit 600 generates a video signal VSG of a predetermined format that can be received by the monitor device from the image data IMG supplied from the post-processing unit 500. When the generated video signal VSG is supplied to a monitor device connected to the video camera 10, an image of the subject is displayed on the monitor device.

  FIG. 2 is a block diagram illustrating a functional configuration of the noise reduction unit 400 in the first embodiment. The noise reduction unit 400 includes a frame accumulation unit 410, an image mixing unit 420, and a spatial smoothing filter 430.

  The frame storage unit 410 includes a motion detection unit 412 and a frame memory 414. The frame memory 414 has two storage areas each storing an image (frame image) for one frame of the image data BD2. The frame image F1 input to the noise reduction unit 400 is alternately written into two storage areas of the frame memory 414 for each frame. The immediately previous frame image F0 stored in the frame memory 414 is read from the storage area in which the frame image F1 is not written out of the two storage areas.

  Note that two storage areas for storing frame images are provided in the frame memory 414, and reading and writing are alternately performed on different areas, so that the image data BD3 generated by the noise reduction unit 400 can be generated per second. The number of frames (frame rate) can be different from the frame rate of the image data BD2 supplied to the noise reduction unit 400. When it is not necessary to convert the frame rate, the frame memory 414 only needs to store one frame image.

  For each pixel of the image data BD2, the motion detection unit 412 is a pixel in an image region (moving region) corresponding to a portion where there is motion in the subject, or an image region (static) corresponding to a portion where there is no motion in the subject. It is determined whether the pixel is in the area. Then, a motion detection result MD representing a determination result as to whether each pixel is a pixel belonging to the moving region (moving region pixel) or a pixel belonging to the static region (static region pixel) is supplied to the image mixing unit 420. To do.

  Specifically, the motion detection unit 412 determines the color component value (hereinafter also referred to as “pixel value”) in the frame image F1 written in the frame memory 414 for the pixel to be determined (target pixel) and the frame memory. A difference Δ between the color component values in the frame image F0 read from 414 is calculated. The difference Δ can be calculated by, for example, the following equation (1) using the pixel values p1 and p0 of the target pixels of the two frame images F1 and F0.

Δ = | p1-p0 | (1)

  Here, the right side of Equation (1) represents the absolute value of the difference obtained by subtracting the pixel value p0 of the frame image F0 from the pixel value p1 of the frame image F1.

  Since the pixel value of the moving region pixel usually varies with time in accordance with the movement in the subject, the difference Δ between the pixel values between the two frame images F1 and F0 increases. On the other hand, since the pixel value of the static region pixel usually has a small temporal variation, the difference Δ between the pixel values between the two frame images F1 and F0 is small. Therefore, the motion detection unit 412 compares the difference Δ calculated by the above equation (1) with a preset threshold value Δc, and when the difference Δ is larger than the threshold value Δc (Δ> Δc), the target pixel is a moving region. Judged to be a pixel. On the other hand, when the difference Δ is equal to or less than the threshold value Δc (Δ ≦ Δc), it is determined that the target pixel is a static region pixel. Note that the threshold value Δc can be determined based on the obtained fluctuation width by experimentally obtaining the fluctuation width of the pixel value of the moving area pixel and the fluctuation width of the pixel value of the static area pixel. For example, when the pixel value is represented by an 8-bit numerical value (0 to 255), the threshold value Δc is set to 150.

  The frame image F0 read from the frame memory 414 is supplied to the image mixing unit 420 directly or via the spatial smoothing filter 430. The spatial smoothing filter 430 performs a spatial smoothing process on the frame image F0 to generate a frame image F0 '. The smoothing of the frame image F0 can be performed, for example, by applying a spatial smoothing filter such as a Gaussian filter or a median filter to the frame image F0. By performing smoothing, the high frequency component of the frame image F0 at the spatial frequency is reduced, and the low frequency component of the spatial frequency component of the smoothed frame image F0 'is relatively high. Therefore, the spatial smoothing filter 430 is also called a “spatial low-pass filter (spatial LPF)”.

  The image mixing unit 420 includes a mixing ratio determination unit 422 and a mixing processing unit 424. The mixing ratio determining unit 422 determines the mixing ratio A (0 ≦ A ≦ 1) of the two images mixed in the mixing processing unit 424 based on the motion detection result MD supplied from the motion detecting unit 412. A method for determining the mixing ratio A based on the motion detection result MD will be described later.

  Based on the mixing ratio A determined by the mixing ratio determination section 422, the mixing processing section 424 uses the frame image F0 directly supplied from the frame memory 414 and the frame image F0 ′ smoothed by the spatial smoothing filter 430. To mix each pixel. Specifically, the pixel value q of the output image data BD3 is calculated from the pixel values p and p 'of the two frame images F0 and F0' based on the following equation (2).

q = A * p + (1-A) * p '(2)

  As apparent from the above equation (2), as the mixing ratio A approaches 0, the proportion of the pixel value p ′ of the smoothed frame image F0 ′ in the pixel value q of the image data BD3 increases. . On the other hand, as the mixing ratio A approaches 1, the ratio of the pixel value p of the unsmoothed frame image F0 in the pixel value q of the image data BD3 increases. Thus, by changing the mixing ratio A, the degree of smoothing in the output image data BD3 can be changed.

Based on the motion detection result MD supplied from the motion detection unit 412, the mixing ratio determination unit 422 performs mixing so as to reduce the degree of smoothing for moving region pixels and increase the degree of smoothing for static region pixels. The ratio A is determined. For example, the mixing ratio determination unit 422 determines the mixing ratio A as follows.
(A) For moving area pixels, the mixing ratio A = 27/32 (about 0.84)
(B) For static area pixels, the mixing ratio A = 16/32 (0.5)

  Thus, the mixture ratio determination unit 422 sets the mixture ratio A to a large value for the moving region pixels. On the other hand, the mixing ratio A is set to a small value for the static region pixels. Therefore, in the image data BD3 output from the noise reduction unit 400, the degree of spatial smoothing is low in the moving area, and the degree of spatial smoothing is high in the static area. As is clear from the above description, the spatial smoothing processing unit configured by the spatial smoothing filter 430 and the image mixing unit 420 can set the degree of spatial smoothing for each pixel.

  Generally, blurring occurs in an image of a subject by performing spatial smoothing. Therefore, when the influence of noise on the image quality is small, it is preferable to reduce the degree of spatial smoothing. Usually, noise is not noticeable in an image of a moving part in a subject. For this reason, by reducing the degree of spatial smoothing with respect to the moving area, it is possible to suppress deterioration in the image quality of the moving area due to blurring. On the other hand, noise is conspicuous in an image of a portion that does not move in the subject. Therefore, by increasing the degree of spatial smoothing with respect to the static area, it is possible to suppress the deterioration of the image quality of the static area due to noise.

  As described above, according to the first embodiment, the degree of spatial smoothing is reduced for a moving area where noise is not conspicuous, and blurring of the image of the moving area is suppressed. On the other hand, the noise in the static region is further reduced by increasing the degree of spatial smoothing for the static region where noise is conspicuous. As a result, it is possible to reduce the influence of noise on the entire image and to suppress the deterioration of the image quality due to the blurring of the moving part.

  In the first embodiment, in order to set the degree of spatial smoothing for each pixel, the mixing ratio A of the two frame images F0 and F0 ′ is determined for each pixel. However, the degree of spatial smoothing is set. Can also be done by other methods. For example, the smoothing parameter in the spatial smoothing filter 430 can be set for each pixel based on the motion detection result MD from the motion detection unit 412. In this case, the image mixing unit 420 can be omitted. However, since the configuration of the spatial smoothing filter 430 becomes simpler, as in the first embodiment, the non-smooth frame image F0 not subjected to spatial smoothing and the smooth frame image subjected to spatial smoothing. It is preferable to set the degree of spatial smoothing by setting the mixing ratio A with F0 ′.

B. Second embodiment:
FIG. 3 is a block diagram illustrating a functional configuration of the noise reduction unit 400a according to the second embodiment. The second embodiment is different from the first embodiment in that the configuration of the noise reduction section 400a is different from the noise reduction section 400 (FIG. 2) of the first embodiment. The other points are the same as in the first embodiment.

  The noise reduction unit 400a according to the second embodiment includes an image superimposing unit 440 and a frame storage unit 410a. The frame storage unit 410a is different from the frame storage unit 410 (FIG. 2) of the first embodiment in that the frame memory 414 is replaced with a superimposed frame memory 416. The other points are the same as the frame storage unit 410 of the first embodiment.

  The superimposed frame memory 416 is changed to the point that the frame image to be written is changed to the frame image G1 supplied from the image superimposing unit 440 and the frame image G0 immediately before being read in accordance with the change of the written frame image. This is different from the frame memory 414 (FIG. 2) of the first embodiment. However, since the configuration and operation of the superimposed frame memory 416 are the same as those of the frame memory 414 of the first embodiment, description thereof is omitted here.

  The image superimposing unit 440 includes a superimposition ratio determining unit 442 and a superimposition processing unit 444. The superposition ratio determination unit 442 determines a superposition ratio B (0 ≦ B ≦ 1) used for processing in the superposition processing unit 444 based on the motion detection result MD supplied from the motion detection unit 412. A method for determining the superposition ratio B based on the motion detection result MD will be described later.

  The superimposition processing unit 444 performs pixel processing on the frame image F1 supplied to the noise reduction unit 400a and the frame image G0 read from the superimposition frame memory 416 based on the superposition ratio B determined by the superposition ratio determination unit 442. Each frame is mixed to generate a frame image G1. Specifically, the pixel value s of the frame image G1 is calculated based on the following formula (3) from the pixel values p and r of the two frame images F1 and G0.

s = (1−B) × p + B × r (3)

  The generated frame image G1 is supplied to the superimposed frame memory 416 and written to the superimposed frame memory 416. Then, the immediately preceding frame image G0 stored in the superimposed frame memory 416 is read and mixed with the frame image F1 in the image superimposing unit 440.

  4A and 4B, the pixel value of the frame image F1 sequentially input to the noise reduction unit 400a is reflected on the pixel value of the frame image G0 stored in the superimposed frame memory 416. It is explanatory drawing which shows the time change of a ratio (frame ratio). FIG. 4A shows the temporal change of the frame ratio when the superposition ratio B is 16/32 (= 0.5), and FIG. 4B shows the superposition ratio B of 5/32 (≈0). .16) shows the temporal change of the frame ratio.

  Immediately after the video camera 10 is started (first frame), the frame image (# 1) input to the noise reduction unit 400a is stored in the superimposed frame memory 416 as it is. Then, in the next second frame, the frame image (# 1) input in the first frame and the frame image (# 2) input in the second frame are mixed according to the superposition ratio B, and the mixed image Is stored in the superimposed frame memory 416. Therefore, in the example of FIG. 4A, the frame ratios of the two frame images (# 1, # 2) are both 50%. On the other hand, in the example of FIG. 4B, the frame ratio of the frame image (# 2) input in the second frame is about 84%, the frame ratio of the frame image input in the first frame is about 16%, The influence of the frame image (# 1) is lower than that in the case of FIG.

  Similarly, in the next third frame, in the example of FIG. 4A, the frame ratios of the two frame images (# 1, # 2) are both 25%, and the frame image input in the third frame ( The frame ratio of # 3) is 50%. On the other hand, in the example of FIG. 4B, the frame ratio of the frame image (# 1) input in the first frame is reduced to about 2%.

  Thus, in the example of FIG. 4A in which the superposition ratio B is higher, the pixel value of the previous frame image is reflected for a longer period. Therefore, since the frame image G0 stored in the superimposed frame memory 416 (FIG. 3) is averaged over a longer time, the degree of noise reduction of the frame image G0 output from the noise reduction unit 400a increases. On the other hand, in the moving region pixels, since the pixel value of the previous frame image is reflected for a longer period, an afterimage is generated in the moving region of the frame image G0 output from the noise reduction unit 400a. On the other hand, in the example of FIG. 4B in which the superposition ratio B is lower, the period in which the pixel value of the previous frame image is reflected is shortened, so that the degree of noise reduction of the frame image G0 is reduced. It is possible to suppress the occurrence of an afterimage in the frame image G0.

  4A and 4B, the frame image F1 input to the noise reduction unit 400a (FIG. 3) and the previous frame image G0 stored in the superimposed frame memory 416 are mixed. Thus, according to the process of generating the superimposed frame image G1 and storing the generated frame image G1 in the superimposed frame memory 416, the pixel values are temporally smoothed. Thus, a filter that performs temporal smoothing is also called a “cyclic filter” or an “IIR filter”. Therefore, it can be said that the superimposition processing unit 444 and the superimposition frame memory 416 together constitute a recursive filter. The superimposition ratio B can also be referred to as a parameter that specifies the degree of temporal smoothing in the recursive filter.

Based on the motion detection result MD supplied from the motion detection unit 412, the superimposition ratio determination unit 442 performs superimposition so as to suppress the occurrence of afterimages for moving regions and to reduce noise for static regions where no afterimages occur. The ratio B is determined. For example, the superimposition ratio determination unit determines the superimposition ratio B as follows.
(A) For moving area pixels, B = 5/32 (about 0.16)
(B) For static region pixels, B = 16/32 (0.5)

  As described above, the superimposition ratio determining unit 442 sets the superimposition ratio B to a small value for the moving region pixels. On the other hand, the superposition ratio B is set to a large value for the static region pixels. Therefore, in the image data BD3 output from the noise reduction unit 400a, the occurrence of afterimages is suppressed in the moving area, and the noise is satisfactorily reduced in the static area.

  Note that the second embodiment is more preferable than the first embodiment in that noise is reduced by temporal smoothing of the frame image, and blurring due to the spatial smoothing process can be suppressed. On the other hand, the first embodiment is more preferable than the second embodiment in that the generation of afterimages due to temporal smoothing of the frame image can be suppressed.

C. Third embodiment:
FIG. 5 is a block diagram illustrating a functional configuration of the noise reduction unit 400b according to the third embodiment. The third embodiment is different from the second embodiment in that the configuration of the noise reduction section 400b is different from the noise reduction section 400a (FIG. 3) of the second embodiment. The other points are the same as in the second embodiment.

  In the noise reduction unit 400b of the third embodiment, a timing adjustment unit 450, a spatial smoothing filter 430, and an image mixing unit 420b are added, and a non-overlapping frame memory 418 is added to the frame storage unit 410b. This is different from the noise reduction unit 400a of the second embodiment shown in FIG. The other points are the same as the noise reduction unit 400a of the second embodiment.

  The spatial smoothing filter 430 and the mixing processing unit 424 of the image mixing unit 420b are respectively the same as the spatial smoothing filter 430 and the mixing processing unit 424 provided in the noise reduction unit 400 of the first embodiment. The mixing ratio determining unit 422b of the image mixing unit 420b is different from the mixing ratio determining unit 422 of the first embodiment in that the method of determining the mixing ratio A (described later) is different.

  In the noise reduction unit 400b of the third embodiment, the input frame image F1 is supplied to both the image superimposition unit 440 and the timing adjustment unit 450. The frame image F1 supplied to the image superimposing unit 440 is subjected to time smoothing as in the second embodiment, thereby reducing noise. The frame image G0 output from the superimposed frame memory 416 is directly supplied to the mixing processing unit 424.

  On the other hand, the timing adjustment unit 450 delays the frame image F1 by a processing time required for the mixing processing of the two frame images F1 and G0 by the superimposition processing unit 444, and generates a delayed frame image F1 '. The delayed frame image F <b> 1 ′ is stored in the non-superimposed frame memory 418. The previous frame image F0 stored in the non-superimposed frame memory 418 is supplied to the spatial smoothing filter 430 in synchronization with the timing at which the superimposed frame memory 416 outputs the frame image G0. The frame image F 0 ′ spatially smoothed by the spatial smoothing filter 430 is supplied to the mixing processing unit 424.

  The mixing processing unit 424 mixes the temporally smoothed frame image G0 supplied from the superimposed frame memory 416 and the frame image F0 'spatially smoothed by the spatial smoothing filter.

The mixing ratio determining unit 422b, for a moving region pixel having a low superposition ratio B, has a higher pixel value of the spatially smoothed frame image F0 ′ than a pixel value of the temporally smoothed frame image G0. The mixing ratio A is determined so as to reflect the ratio. On the other hand, for the static region pixels having a low superposition ratio, the pixel value of the temporally smoothed frame image G0 is reflected at a higher ratio than the pixel value of the spatially smoothed frame image F0 ′. The mixing ratio A is determined. Specifically, the mixing ratio determination unit 422b determines the mixing ratio A as follows.
(A) For moving area pixels, the mixing ratio A = 16/32 (0.5)
(B) For static area pixels, the mixing ratio A = 27/32 (about 0.84)

  As described above, by setting the superposition ratio B to a smaller value for the moving area pixels, the occurrence of afterimages is suppressed in the moving area of the image data BD3 output by the noise reduction unit 400b. Moreover, since the pixel value of the spatially smoothed frame image F0 ′ is more strongly reflected by setting the mixing ratio A to a smaller value for the moving region pixels, the noise of the moving region pixels Is reduced by spatial smoothing.

  On the other hand, for the static region pixels, the mixing ratio A is set to a larger value, so that the temporally smoothed frame image G0 is displayed in the static region of the image data BD3 output by the noise reduction unit 400b. Pixel values are reflected more strongly. Therefore, since the degree of spatial smoothing is reduced in the static area, blurring of the image in the static area is suppressed. Then, by setting the superposition ratio B to a larger value with respect to the static region pixels, noise in the static region with less afterimage problems can be satisfactorily reduced by temporal smoothing.

  As described above, according to the third embodiment, by setting both the mixing ratio A and the superposition ratio B to smaller values for the moving region pixels, it is possible to suppress the occurrence of afterimages and reduce the noise satisfactorily. Is done. On the other hand, it is possible to suppress the occurrence of blur due to spatial smoothing, in which both the mixing ratio A and the superposition ratio B are set to larger values for the static region pixels. Therefore, according to the third embodiment, it is possible to suppress the generation of afterimages and reduce the noise of the entire image.

  In the third embodiment, the superimposition ratio determination unit 442 determines the superposition ratio B based on the motion detection result MD supplied from the motion detection unit 412, but the superposition ratio determination unit 442 may be omitted. Is possible. Even in this case, by setting the mixing ratio A in the mixing processing unit 424 to a smaller value and more strongly reflecting the pixel value of the spatially smoothed frame image F0 ′, the occurrence of afterimages can be suppressed. Can do. However, it is more preferable to determine the superimposition ratio B based on the motion detection result MD in that the generation of afterimages can be more effectively suppressed by reducing the superposition ratio B.

  The third embodiment is less than the first and second embodiments in that the noise reduction unit 400b can reduce the noise of the entire image by reducing the noise in either the static region or the moving region. preferable. On the other hand, the first and second embodiments are preferable to the third embodiment in that the configuration of the noise reduction units 400 (FIG. 2) and 400a (FIG. 3) is simpler.

D. Variations:
In addition, this invention is not restricted to the said Example and embodiment, It can implement in a various aspect in the range which does not deviate from the summary, For example, the following deformation | transformation is also possible.

D1. Modification 1:
In each of the above embodiments, the noise reduction unit 400 (FIG. 1) is arranged before the post-processing unit 500 having the interpolation processing unit 510, and the noise reduction processing is performed on the Bayer data before the interpolation processing. Noise reduction processing may be performed on the data after interpolation processing. However, the noise reduction processing can be performed on the Bayer data in that the amount of calculation processing required for the noise reduction processing can be reduced by performing the noise reduction processing for each color component value of any one of RGB. More preferred.

D2. Modification 2:
In the second to third embodiments, the temporal smoothing process is performed by a recursive filter. In general, a temporally smoothed frame image can be generated using a plurality of frame images. For example, an arbitrary temporal smoothing method can be used. The temporal smoothing process can also be performed using, for example, an acyclic filter. However, it is preferable to perform the temporal smoothing process using a cyclic filter in that the capacity of the frame memory for storing the frame image can be further reduced.

D3. Modification 3:
In each of the above embodiments, the motion detection unit is configured so that each pixel is a moving region and a static region based on the difference between the pixel value of the frame image stored in the frame memory and the pixel value of the frame image input to the frame memory. However, it is also possible to determine whether each pixel belongs to a moving area or a static area by another method. For example, it is possible to divide a frame image into a plurality of blocks and detect the change for each block. It is also possible to detect the position and motion vector of a moving object from a plurality of frame images and determine whether each pixel belongs to a moving area or a static area based on the position and motion vector of the moving object. is there.

D4. Modification 4:
In each of the above embodiments, the present invention is applied to a video camera. However, the present invention can be applied to any apparatus as long as it is an apparatus that inputs and outputs moving images. The present invention can also be applied to, for example, a moving picture recording apparatus and a moving picture reproducing apparatus such as a video recorder and a video disc, and a moving picture display apparatus such as a television and a projector.

1 is a schematic configuration diagram showing a schematic configuration of a video camera 10 as one embodiment of the present invention. The block diagram which shows the functional structure of the noise reduction part 400 in 1st Example. The block diagram which shows the functional structure of the noise reduction part 400a in 2nd Example. Explanatory drawing which shows the time change of a frame ratio. The block diagram which shows the functional structure of the noise reduction part 400b in 3rd Example.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 ... Video camera 100 ... Imaging lens 200 ... Image sensor 202 ... Imaging surface 210 ... Color filter 220 ... Imaging element 300 ... Pre-processing part 400, 400a, 400b ... Noise reduction part 410, 410a, 410b ... Frame storage part 412 ... Motion Detection unit 414 ... Frame memory 416 ... Superimposition frame memory 418 ... Non-superimposition frame memory 420, 420b ... Image mixing unit 422, 422b ... Mixing ratio determination unit 424 ... Mixing processing unit 430 ... Spatial smoothing filter 440 ... Image superposition unit 442 ... Superimposition ratio determination unit 444 ... Superimposition processing unit 450 ... Timing adjustment unit 500 ... Post-processing unit 510 ... Interpolation processing unit 600 ... Video signal generation unit

Claims (3)

A noise reduction device that reduces noise in a moving image,
A motion detection unit that detects a moving region having movement in the moving image and a static region other than the moving region;
A spatial smoothing processing unit that generates a spatially smoothed frame image by applying a spatial smoothing process to an input frame image constituting the moving image;
With
The spatial smoothing processing unit is configured to be able to set the degree of the spatial smoothing process for each pixel constituting the input frame image,
The degree of the spatial smoothing process in the pixels belonging to the static area is set to be higher than the degree of the spatial smoothing process in the pixels belonging to the moving area.
Noise reduction device. The noise reduction device according to claim 1,
The spatial smoothing processing unit
A spatial smoothing filter that performs a spatial smoothing process on the input frame image;
A mixing processing unit that mixes a smooth frame image that has passed through the spatial smoothing filter and a non-smooth frame image that has not passed through the spatial smoothing filter;
With
The mixing processing unit is configured to be able to set a mixing ratio of the smooth frame image and the non-smooth frame image for each pixel.
Noise reduction device. A noise reduction method for reducing noise in a moving image,
(A) detecting a moving area having movement in the moving image and a static area other than the moving area;
(B) generating a spatially smoothed frame image by applying a spatial smoothing process to the input frame image according to the degree of the spatial smoothing process set for each pixel constituting the input frame image;
(C) setting the degree of the spatial smoothing process in the pixels belonging to the static area to be higher than the degree of the spatial smoothing process in the pixels belonging to the moving area;
Comprising
Noise reduction method.

Images