JP2014158083A - Video processing device, video processing method, broadcast reception device, video imaging device, video storage device, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a video processing device, video processing method, broadcast reception device, video imaging device, video storage device, and program capable of reducing noise while avoiding reduction in video quality.SOLUTION: A video processing device 1 includes: a movement detection unit 10 which takes one of a plurality of frames as an attention frame, uses at least either of frames preceding and succeeding the attention frame as a movement detection reference frame, and detects a movement vector MV in a pixel unit; and a noise reduction unit 20 for generating a noise reduction frame FS3 by using the movement vector MV, attention frame, and noise reduction reference frame. The noise reduction unit 20 generates movement vector boundary pixel information, analyzes a pixel value change rate between the attention frame and noise reduction reference frame, generates analysis result information on the pixel value change rate, and generates a noise reduction frame by using the movement vector boundary pixel information and analysis result information on the pixel value change rate.

Description

  The present invention relates to an apparatus, a method, and a program that can reduce noise included in a video signal.

  In general, noise is added to a video signal in a process of photographing a subject to generate a video signal, a process of transmitting the video signal, and a process of displaying a video on a display device based on the received video signal. This noise includes dark noise and coding noise. Dark noise is noise that appears when a small fluctuation of a video signal is amplified by amplification processing for displaying a dark subject as a bright video. The coding noise is noise generated in the video signal by the coding process for transmitting the video signal.

  As a method of reducing such noise, a method of performing a smoothing process in a spatial direction on a video signal using an averaging filter or a Gaussian filter is known. However, when the smoothing process in the spatial direction is performed, there is a problem that the sharpness of the image is lowered and the image is blurred as the degree of noise reduction is increased. In addition, a method of performing a smoothing process in the spatial direction while preserving the edge of an image using a median filter is also known, but in this case, another problem that the shape of the edge of the image changes occurs. .

  In order to solve the above problem, a three-dimensional noise reduction method for performing noise reduction processing based on a comparison result of a plurality of temporally adjacent video frames (hereinafter also referred to as “frames” or “one-frame video data”). Has been proposed. In this method, for example, a current frame (frame of interest) that is a target frame for noise reduction, a frame temporally preceding the current frame (previous frame), and a frame temporally subsequent to the current frame (back frame) ) Is calculated and the pixel value of the pixel having the same coordinate is calculated, and the pixel value of the current frame is replaced with the calculated pixel value (average value). The three-dimensional noise reduction method does not generate image blur as in the case of the smoothing process in the spatial direction, but refers to the pixel values of the pixels at the same coordinates in a plurality of frames at different times, so Is present, there is a problem that blurring such as afterimage occurs.

  Blurring such as an afterimage can be reduced by using a motion compensation method that refers to the pixel value of the pixel at the same position of the same object instead of referring to the pixel value of the pixel of the same coordinate (for example, Patent Documents). 1). However, in the three-dimensional noise reduction method using the motion compensation method, the position of the pixel to be referred to is determined according to the motion of the object (the motion vector between frames). Detection accuracy greatly affects noise reduction performance. For example, when motion is detected by mistake, not only noise cannot be reduced, but also the video quality may be lowered. In particular, if there are areas that are hidden in the foreground object over time, or areas that appear behind the foreground object over time (i.e. areas that were hidden), There is no pixel to be referenced in a frame that temporally precedes the frame of interest, and the coordinate position of the pixel to be referenced cannot be appropriately determined according to the movement of the object.

JP 2008-294601 A

  As described above, in the conventional three-dimensional noise reduction method, when a motion vector between frames is erroneously detected, not only noise can be appropriately reduced but also video quality may be lowered.

  Therefore, the present invention has been made to solve the above-described problems of the prior art, and an object of the present invention is to provide a video processing apparatus and video processing capable of appropriately reducing noise while avoiding deterioration in video quality. A method, a broadcast receiving device, a video photographing device, a video storage device, and a program are provided.

  The video processing apparatus according to one aspect of the present invention uses one of a plurality of frames constituting a moving image as a target frame, and at least one of a temporally preceding frame and a temporally subsequent frame of the target frame. As a motion detection reference frame, estimating a motion between the frame of interest and the motion detection reference frame, and detecting a motion vector in units of pixels constituting the frame of interest, and the motion Using the reference vector for noise reduction that is at least one of the motion vector, the frame of interest, the frame temporally preceding the frame of interest, and the frame temporally following the frame of interest detected by the detection unit, A noise reduction unit that generates a noise reduction frame with reduced noise, the noise reduction unit Pixel value change between the motion vector boundary detection means for detecting the boundary pixel of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixel, and the reference frame for noise reduction A pixel value change rate analyzing means for analyzing a rate and generating analysis result information of the pixel value change rate, and generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate And correction means for performing the above.

In the video processing method according to another aspect of the present invention, one of a plurality of frames constituting a moving image is set as a frame of interest, and at least of a frame temporally preceding and temporally following the frame of interest. Using one of them as a reference frame for motion detection, estimating a motion between the frame of interest and the reference frame for motion detection, and detecting a motion vector in units of pixels constituting the frame of interest; The target frame using the noise reduction reference frame that is at least one of the motion vector, the target frame, the temporally preceding frame, and the temporally following frame detected in the motion detection step. A noise reduction step for generating a noise reduction frame with reduced noise The noise reduction step includes a motion vector boundary detection step of detecting a boundary pixel of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixel; the frame of interest; and the motion detection reference frame; A pixel value change rate analysis step for analyzing the pixel value change rate between the two, generating the pixel value change rate analysis result information, and using the motion vector boundary pixel information and the pixel value change rate analysis result information And a noise reduction frame generation step for generating the noise reduction frame.

A broadcast receiving apparatus according to another aspect of the present invention includes a broadcast receiving unit that receives a broadcast wave and decodes video information included in the broadcast wave, and a series of a plurality of pieces that configure a moving image from the broadcast receiving unit. And using one of the plurality of frames as a target frame, using at least one of a temporally preceding frame and a temporally following frame as the motion detection reference frame, Estimating a motion between the frame of interest and the reference frame for motion detection, detecting a motion vector in units of pixels constituting the frame of interest, the motion vector detected by the motion detector, Noise reduction that is at least one of a frame of interest and a frame temporally preceding and temporally following the frame of interest Using the reference frames, and a noise reduction unit which generates a noise reduction frame noise of the focus frame is reduced, the noise reduction unit,
Pixel value change between the motion vector boundary detection means for detecting the boundary pixel of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixel, and the frame of interest and the reference frame for motion detection A pixel value change rate analyzing means for analyzing a rate and generating analysis result information of the pixel value change rate, and generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate And correction means for performing the above.

  An image capturing apparatus according to another aspect of the present invention includes an imaging unit that captures a video and generates a series of frame data constituting a moving image, and a series of frames that constitute a moving image from the imaging unit. Receiving one of the plurality of frames as a target frame, using at least one of a temporally preceding frame and a temporally following frame as the motion detection reference frame, A motion detection unit that estimates a motion between the motion detection reference frame and detects a motion vector in units of pixels constituting the target frame; the motion vector detected by the motion detection unit; the target frame; And at least one of a temporally preceding frame and a temporally following frame of the frame of interest. A noise reduction unit that generates a noise reduction frame in which noise of the frame of interest is reduced using a reference frame, wherein the noise reduction unit detects a boundary pixel of the motion vector, and detects the detected boundary Motion vector boundary detection means for generating motion vector boundary pixel information indicating a pixel; and a pixel value change rate between the frame of interest and the motion detection reference frame; and analysis result information of the pixel value change rate It has a pixel value change rate analyzing means to be generated, and a correcting means for generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate.

  A video storage device according to another aspect of the present invention includes a storage unit that stores a series of frames constituting a moving image, and a frame of interest as one of the series of frames stored in the storage unit. And using at least one of a temporally preceding frame and a temporally following frame as the motion detection reference frame to estimate a motion between the focus frame and the motion detection reference frame. A motion detection unit that detects a motion vector in units of pixels constituting the target frame, the motion vector detected by the motion detection unit, the target frame, a frame temporally preceding the target frame, and a temporal The noise of the frame of interest is reduced by using a noise reduction reference frame that is at least one of the following frames. A noise reduction unit that generates a noise reduction frame, wherein the noise reduction unit detects a boundary pixel of the motion vector and generates motion vector boundary pixel information indicating the detected boundary pixel. Boundary detection means, pixel value change rate analysis means for analyzing a pixel value change rate between the frame of interest and the motion detection reference frame, and generating analysis result information of the pixel value change rate; and the motion vector And correction means for generating the noise reduction frame using boundary pixel information and analysis result information of the pixel value change rate.

  A program according to another aspect of the present invention is a program that causes a computer to execute a process, and the process uses one of a plurality of frames constituting a moving image as a target frame, and is temporally more than the target frame. Pixels constituting the frame of interest by estimating the motion between the frame of interest and the reference frame for motion detection using at least one of the frame preceding the frame and the frame following temporally as the frame for motion detection At least one of a motion detection process for detecting a unit motion vector, the motion vector detected in the motion detection process, the frame of interest, a frame temporally preceding the frame of interest, and a frame temporally following Using a noise reduction reference frame, the noise of the frame of interest is reduced. Noise reduction processing for generating a noise reduction frame, wherein the noise reduction processing detects a boundary pixel of the motion vector and generates motion vector boundary pixel information indicating the detected boundary pixel. A pixel value change rate analysis process for analyzing a pixel value change rate between the frame of interest and the motion detection reference frame, and generating analysis result information of the pixel value change rate, and the motion vector boundary pixel information And noise reduction frame generation processing for generating the noise reduction frame using the analysis result information of the pixel value change rate.

  According to the present invention, not only the motion vector but also the motion vector boundary pixel information indicating the boundary pixel of the motion vector and the analysis result information of the pixel value change rate between the target frame and the motion detection reference frame are used to reduce noise. Since the frame is generated, it is possible to appropriately reduce noise while avoiding deterioration in video quality.

It is a functional block diagram which shows roughly the structure of the video processing apparatus (namely, apparatus which can implement the video processing method which concerns on Embodiment 1) which concerns on Embodiment 1 of this invention. It is a functional block diagram which shows roughly the structure of the noise reduction part of the video processing apparatus shown by FIG. It is a functional block diagram which shows roughly the structure of the frame blend rate calculation means of the noise reduction part shown by FIG. FIG. 3 is a functional block diagram schematically showing a configuration of a motion error detection region correction coefficient calculation unit of the noise reduction unit shown in FIG. 2. FIG. 6 is a diagram schematically showing a frame and a coordinate position (upper side in FIG. 5) referred to in noise reduction processing in the video processing apparatus according to Embodiment 1, and a frame subjected to noise reduction processing (lower side in FIG. 5). . (A)-(i) is a graph which shows the analysis pattern as a specific example of the analysis result information of the pixel value change rate by the pixel value change rate analysis means of the noise reduction part shown by FIG. FIG. 3 is a flowchart schematically showing pixel value change pattern classification processing executed by a pixel value change rate analysis unit of the noise reduction unit shown in FIG. 2. FIG. (A)-(c) is a figure which shows an example of an occlusion. It is a figure which shows an example of the occlusion area | region based on the occlusion determination result output from the occlusion determination means of the frame blend rate calculation means shown by FIG. FIG. 5 is a diagram illustrating an example of a reference frame indefinite region detected by a reference frame indefinite region detecting unit of the motion error detection region correction coefficient calculating unit shown in FIG. 4. FIG. 5 is a diagram illustrating an example of a motion vector concentration region detected by a motion vector boundary concentration region detection unit of a motion error detection region correction coefficient calculation unit illustrated in FIG. 4. It is a functional block diagram which shows roughly the structure of the video processing apparatus (namely, apparatus which can implement the video processing method which concerns on Embodiment 2) concerning Embodiment 2 of this invention. 6 is a diagram illustrating frames used for noise reduction processing in the video processing apparatus according to Embodiment 1. FIG. FIG. 10 is a diagram illustrating frames used for noise reduction processing in the video processing apparatus according to the second embodiment. It is a functional block diagram which shows roughly the structure of the video processing apparatus (namely, apparatus which can implement the video processing method which concerns on the modification of Embodiment 2) which concerns on the modification of Embodiment 2 of this invention. . FIG. 10 is a diagram illustrating frames used for noise reduction processing in a video processing apparatus according to a modification of the second embodiment. It is a functional block diagram which shows roughly the structure of the broadcast receiver which concerns on Embodiment 3 of this invention. FIG. 11 is a functional block diagram schematically showing a configuration of a broadcast receiving device according to a modification of the third embodiment. It is a functional block diagram which shows roughly the structure of the video imaging device which concerns on Embodiment 4 of this invention. FIG. 10 is a functional block diagram schematically showing a configuration of a video imaging apparatus according to a modification of the fourth embodiment. It is a functional block diagram which shows schematically the structure of the video storage apparatus which concerns on Embodiment 5 of this invention. FIG. 10 is a functional block diagram schematically showing a configuration of a video storage device according to a modification of the fifth embodiment.

<< 1 >> Embodiment 1
<< 1-1 >> Configuration of Embodiment 1 FIG. 1 shows a configuration of a video processing apparatus 1 according to Embodiment 1 of the present invention (that is, an apparatus that can implement the video processing method according to Embodiment 1). FIG. As shown in FIG. 1, the video processing device 1 according to the first embodiment includes a motion detection unit 10 and a noise reduction unit 20. As shown in FIG. 1, the video processing device 1 may include a frame buffer 30 and a frame buffer 40. The frame buffer 30 is a storage unit for storing a series of a plurality of frames FS1 constituting a moving image. The frame buffer 40 is a storage unit for storing a series of a plurality of frames constituting a moving image. For example, the frame buffer 40 is a series of a plurality of frames (noises) that are video data with reduced noise output from the noise reduction unit 20. Reduction frame) FS3 can be stored. The video processing device 1 is a device that uses a three-dimensional noise reduction method using a motion compensation method, and even when the motion detection unit 10 detects motion by mistake, it is appropriately selected so as not to deteriorate the video quality. This is a device that can reduce noise.

  The video processing method according to the first embodiment is a method that can be executed by the video processing device 1 according to the first embodiment. In the video processing method according to the first embodiment, one of a plurality of frames constituting a moving image is set as a frame of interest, and at least one of a temporally preceding frame and a temporally subsequent frame of the frame of interest is used. A motion detection step (motion detection unit 10) that uses the motion detection reference frame as a motion detection reference frame to estimate motion between the target frame and the motion detection reference frame and detect a motion vector in units of pixels constituting the target frame. And a reference for noise reduction that is at least one of the motion vector detected in the motion detection step, the frame of interest, a frame temporally preceding the frame of interest, and a frame temporally following the frame of interest. Noise reduction frame in which the noise of the frame of interest is reduced using a frame And a (steps performed by the noise reduction unit 20) noise reduction step of generating. Here, the noise reduction step is performed by a motion vector boundary detection step (detected by a motion vector boundary detection unit 202 described later) that detects a boundary pixel of a motion vector and generates motion vector boundary pixel information indicating the detected boundary pixel. And a pixel value change rate analysis step (a pixel value to be described later) for analyzing the pixel value change rate between the frame of interest and the motion detection reference frame and generating analysis result information of the pixel value change rate A step executed by the change rate analysis unit 201) and a noise reduction frame generation step (frame blend described later) for generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate. Rate calculation means 204, noise reduction image generation means 205, motion error detection area correction coefficient Detecting means 203 includes the step) and executed by such motion erroneous detection area correcting unit 206.

  The video processing apparatus 1 may be a computer that operates according to a program. This program is a program for causing a computer to execute processing. In the processing, one of a plurality of frames constituting a moving image is set as a frame of interest, and a frame temporally preceding the frame of interest and temporal Is used as a motion detection reference frame to estimate a motion between the frame of interest and the motion detection reference frame, and detect a motion vector in pixel units constituting the frame of interest A noise detection reference frame that is at least one of a motion detection process, the motion vector detected in the motion detection process, the frame of interest, a frame temporally preceding the frame of interest, and a frame temporally following the frame of interest. Using a noise reduction frame in which the noise of the frame of interest is reduced. Noise reduction processing to be performed, wherein the noise reduction processing detects a boundary pixel of the motion vector and generates motion vector boundary pixel information indicating the detected boundary pixel; and A pixel value change rate analysis process for analyzing a pixel value change rate between the frame and the motion detection reference frame, and generating analysis result information of the pixel value change rate, the motion vector boundary pixel information, and the pixel value And noise reduction frame generation processing for generating the noise reduction frame using the analysis result information of the change rate.

  The program can be recorded on a computer-readable recording medium. In this case, a computer-readable recording medium on which the program is recorded also forms a part of the present invention.

The motion detection unit 10 shown in FIG. 1 receives a series of a plurality of frames (that is, a frame sequence composed of a plurality of frames arranged temporally) FS1 constituting a moving image, and the series of the plurality of frames FS1. One of the frames is a target frame, and at least one of a temporally preceding frame and a temporally subsequent frame is used as a reference frame (in the first embodiment, both frames are used for error detection. The motion vector MV is detected in pixel units (that is, for each pixel in the frame) by estimating the motion between the frame of interest and the frames before and after the frame of interest. For example, the motion detection unit 10 receives a series of a plurality of frames FS1 constituting a moving image from the frame buffer 30, and a first frame F _n−1 which is a frame adjacent to each other in the received series of frames FS1. Motion vector between the second frame F _n and the second frame F _n and a motion vector between the second frame F _n and the third frame F _{n + 1} adjacent to each other. Note that n is a positive integer, and the subscripts n of F, n−1, n, n + 1,... Are integers indicating the order of frame arrangement. Also, the frames used for motion vector detection are not limited to the three consecutive frames.

  The noise reduction unit 20 includes a reference frame (a reference frame for noise reduction) that is at least one of a motion vector detected by the motion detection unit 10, a frame of interest, a frame temporally preceding the frame of interest, and a temporally subsequent frame. ) Is used to generate a noise reduction frame in which the noise of the frame of interest is reduced. The noise reduction unit 20 detects a motion vector boundary pixel, generates motion vector boundary pixel information indicating the detected boundary pixel, motion vector boundary detection means (motion vector boundary detection means 202 described later), a frame of interest A pixel value change rate analyzing means (pixel value change rate analyzing means 201 described later) for analyzing a pixel value change rate between the reference frame for motion detection and a motion detection reference frame, and generating analysis result information of the pixel value change rate; Noise reduction frame generation means for generating the noise reduction frame FS3 using the vector boundary pixel information and the analysis result information of the pixel value change rate (frame blend rate calculation means 204, noise reduction image generation means 205, motion error described later) Detection area correction coefficient calculation means 203, motion error detection area correction means 206). Note that the frame used for generating the noise reduction frame FS3 is not limited to the three consecutive frames.

FIG. 2 is a functional block diagram schematically showing the configuration of the noise reduction unit 20 shown in FIG. As shown in FIG. 2, the noise reduction unit 20 analyzes the change rate of the pixel value between frames, and generates pixel value change rate analysis means 201 that generates analysis result information AN of the pixel value change rate between frames. A motion vector boundary detection unit 202 that analyzes the distribution of the motion vector MV, detects a boundary of the motion vector MV, and generates motion vector boundary pixel information BO indicating the boundary pixel of the motion vector MV; It has a motion erroneous detection region correction coefficient calculation means 203 for detecting a region where it can be determined that the detected motion is erroneous and calculating a correction coefficient ER for correcting the image based on the detection result. In addition, the noise reduction unit 20 includes a frame blend rate calculation unit 204 that calculates a blend weight coefficient (frame blend rate) BW that is a frame synthesis coefficient used when calculating a weighted average of frames for noise reduction. Due to the detected motion error in the noise-reduced image generation means 205 that performs noise reduction processing using the blend weight coefficient BW and the region in which the motion detection performed by the motion detection unit 10 can be determined to be erroneous. And a motion erroneous detection region correction means 206 for correcting the disturbance of the noise reduced image. In FIG. 2, reference numeral 207 denotes a series of a plurality of frames (that is, a frame sequence including the first, second, and third frames F _n−1 , F _n , and F _{n + 1} ) FS1 that is a moving image. 208 denotes an input terminal to which a pixel-based motion vector MV is input, and 209 denotes an output terminal from which a noise reduction frame FS3, which is image data with reduced noise, is output.

The pixel value change rate analysis unit 201 includes a first frame, a second frame, a third frame F _n−1 , F _n , F _{n + 1} in a series of a plurality of frames FS 1 input via the input terminal 207, and an input terminal. A pixel-by-pixel motion vector MV input via 208, and based on the received first, second, and third frames F _n−1 , F _n , F _{n + 1} and the motion vector MV, pixels between frames The change in value is analyzed for each pixel, and analysis result information AN of the pixel value change rate between frames is generated.

  The motion vector boundary detection unit 202 receives the motion vector MV in units of pixels input via the input terminal 208, and the difference between the motion vector of the target pixel to be detected and the motion vector of the adjacent pixel adjacent to the target pixel. A target pixel having a value equal to or greater than a predetermined value is extracted as a motion vector boundary pixel, and motion vector boundary pixel information BO and motion vector MV, which are information related to the extracted boundary pixel, are output.

  The motion error detection region correction coefficient calculation unit 203 receives the analysis result information AN of the pixel value change rate between frames from the pixel value change rate analysis unit 201, and receives the motion vector MV and motion vector boundary pixel information from the motion vector boundary detection unit 202. A pixel that has received the BO, determines the presence or absence of erroneous detection of the motion vector in each pixel based on the received pixel value change rate analysis result information AN, the motion vector MV, and the motion vector boundary pixel information BO. The correction coefficient ER is calculated.

The frame blend rate calculation unit 204 receives the analysis result information AN of the pixel value change rate between frames from the pixel value change rate analysis unit 201, and receives the motion vector MV and the motion vector boundary pixel information BO from the motion vector boundary detection unit 202. Based on the received pixel value change rate analysis result information AN, motion vector MV, and motion vector boundary pixel information BO, the first, second, and third frames F _n−1 , F _n , and F _{n + 1} are weighted. A blend weight coefficient BW used for averaging is calculated.

The noise reduction image generation unit 205 receives the first, second, and third frames F _n−1 , F _n , and F _{n + 1} via the input terminal 207, receives the blend weight coefficient BW from the frame blend rate calculation unit 204, Based on the received blend weight coefficient BW and the first, second, and third frames F _n−1 , F _n , and F _{n + 1} , the noise-reduced image of the frame of interest (correction by the correction coefficient ER considering the motion error detection region) The previous noise reduction frame) FS2 is generated.

The motion error detection area correction unit 206 receives the first, second, and third frames F _n−1 , F _n , and F _{n + 1} via the input terminal 207, and the correction coefficient ER from the motion error detection area correction coefficient calculation unit 203. And the noise reduction frame FS2 is received from the noise reduction image generation means 205, and is based on the received first, second, and third frames F _n−1 , F _n , F _{n + 1} , the correction coefficient ER, and the noise reduction image FS2. Thus, the motion erroneous detection region of the noise reduced image FS2 is corrected, and a corrected image, that is, a series of a plurality of frames (noise reduced frames) FS3 constituting the noise-reduced video data is generated.

FIG. 3 is a functional block diagram schematically showing the configuration of the frame blend rate calculation means 204 shown in FIG. As shown in FIG. 3, the frame blend rate calculation means 204 reduces the noise of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} from the analysis result information AN of the pixel value change rate. Reference frame priority calculation means 2041 for calculating a priority PR according to the contribution rate of the image, occlusion determination means 2042 for determining occlusion from the motion vector MV and the motion vector boundary pixel information BO, and generating an occlusion determination result OC; Blend weight coefficient calculation means 2043 for calculating the blend weight coefficient rate BW. In FIG. 3, 2044 indicates an input terminal to which the analysis result information AN of the pixel value change rate is input, 2045 indicates an input terminal to which the motion vector MV and the motion vector boundary information BO are input, and 2046 indicates , The output terminal from which the blend weight coefficient BW is output.

The reference frame priority calculation means 2041 receives the analysis result information AN of the pixel value change rate via the input terminal 2044, and each noise of the first, second, and third frames F _n−1 , F _n , and F _{n + 1.} And the presence / absence of occlusion that hides the background of the foreground object, and the priority PR optimum for the noise reduction processing of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} is calculated.

  The occlusion determination unit 2042 receives the motion vector MV and the motion vector boundary pixel information BO via the input terminal 2045, determines the presence / absence of occlusion for each pixel, and generates an occlusion determination result OC indicating the presence / absence of occlusion for each pixel. .

The blend weight coefficient calculating unit 2043 receives the priority PR of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} from the reference frame priority calculating unit 2041, and for each pixel from the occlusion determining unit 2042. The blend weight coefficient used for the weighted average of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} based on the received priority PR and the occlusion determination result OC BW is calculated.

  FIG. 4 is a functional block diagram schematically showing the configuration of the motion error detection region correction coefficient calculation unit 203 shown in FIG. As shown in FIG. 4, the motion error detection region correction coefficient calculation unit 203 includes a pixel value sudden change region detection unit 2031 that determines whether correction is necessary based on the analysis result information AN of the pixel value change rate, and a motion vector. Based on the boundary pixel information BO, a reference frame indefinite area detecting unit 2032 that generates a reference frame indefinite area information BR indicating a detection result by detecting a reference frame indefinite area in which the reference frame becomes indefinite when determining a reference frame. And have. The motion error detection area correction coefficient calculation means 203 includes a motion vector boundary concentration area detection means 2033 for detecting a motion vector boundary concentration area where motion vector boundaries are concentrated, and a correction coefficient for correcting a motion vector error detection area. Correction coefficient calculation means 2034 for calculating ER. In FIG. 4, 2035 indicates an input terminal to which the analysis result information AN of the pixel value change rate is input, 2036 indicates an input terminal to which the motion vector MV and the motion vector boundary pixel information BO are input, Reference numeral 2037 denotes an output terminal from which a correction coefficient ER for correcting a region where motion is erroneously detected is output.

  The pixel value sudden change area detection unit 2031 receives the analysis result information AN of the pixel value change rate via the input terminal 2035, and the pixel value is affected by the erroneous detection of motion based on the received analysis result information AN of the pixel value change rate. A region where the value changes rapidly is detected, and correction target region information SR indicating the detected region is generated.

  The reference frame indefinite region detection means 2032 receives the motion vector MV and the motion vector boundary pixel information BO via the input terminal 2036, and avoids occlusion based on the received motion vector MV and the motion vector boundary pixel information BO. An area where the reference frame for determining the reference destination is indefinite is detected, and reference frame indefinite area information RR indicating an area where the reliability of the motion vector MV is low and noise reduction processing is inappropriate is generated.

  The motion vector boundary concentration region detection unit 2033 receives the motion vector MV and the motion vector boundary pixel information BO via the input terminal 2036, and detects motion erroneously based on the received motion vector MV and motion vector boundary pixel information BO. Motion vector boundary concentration region information BR indicating a motion vector boundary concentration region estimated to be caused is generated.

  The correction coefficient calculation unit 2034 receives the correction target region information SR in which the pixel value is rapidly changing from the pixel value sudden change region detection unit 2031, receives the reference frame indefinite region information RR from the reference frame indefinite region detection unit 2032, and moves The motion vector boundary concentration region information BR is received from the vector boundary concentration region detection means 2033, and the image disturbance on the noise reduced image (FS2 in FIG. 2) in these regions is corrected based on the received information SR, RR, BR. A correction coefficient ER is calculated for this purpose.

<< 1-2 >> Operation of Embodiment 1 << 1-2-1 >> Pixel Value Change Rate Analysis Unit 201
Details of processing contents of each functional block will be described below with reference to the drawings. The pixel value change rate analysis unit 201 receives the first, second, and third frames F _n−1 , F _n , and F _{n + 1} , and motion-compensated first, second, and third frames F _n for each pixel. Changes in pixel values between ₋₁ , F _n , and F _{n + 1} are analyzed.

FIG. 5 shows first, second, and third frames F _n−1 , F _n , F _{n + 1} and coordinate positions (upper side in FIG. 5) that are referred to in noise reduction processing in the video processing apparatus 1 according to the first embodiment. And it is a figure which shows roughly the noise reduction frame _Mn (lower side of FIG. 5) by which the noise reduction process was carried out. As shown in FIG. 5, in the first embodiment, the video processing device 1 sets the second frame F _n as a frame of interest that is a target for noise reduction, and precedes the second frame F _{n in} terms of time. A first frame F _n−1 that is a frame (previous frame) and a third frame F _{n + 1} that is a temporally subsequent frame (rear frame) are used as reference frames. However, the set of frames used by the video processing apparatus 1 is not limited to the example of FIG. For example, the motion detection unit 10 selects two frames from the first, second, and third frames F _n−1 , F _n , and F _{n + 1} , and detects motion using the selected two frames. The noise reduction unit 20 may be configured to perform noise reduction processing using three frames of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} .

FIG. 5 shows how the circular foreground object 110 moves on the stationary background 111 from the lower left to the upper right in the image. In the example of FIG. 5, since the pixel of interest P1 _n on the background (region other than the foreground object 110) 111 in the second frame F _n is on the stationary background 111, the motion vector output from the motion detection unit 10 A pixel on the first frame F _n−1 having the same coordinates as the pixel P1 _n on the second frame F _n also on the first frame F _n−1 and the third frame F _{n + 1} based on the “0 vector” that is MV. Reference may be made to pixel P1 _{n + 1} on P1 _n-1 and third frame F _{n + 1} . On the other hand, the pixel P2 _n on the second frame _{F n} is, because there on the foreground object 110 to be moved, in the first frame _{F n-1} and the third on the frame _{F n + 1,} the pixels on the second frame _{F n} P2 A pixel having a coordinate different from the coordinate of _n is referred to. That is, the pixel P2 _n on the second frame _{F n} is, according to the motion vector output from the motion detection unit 10, the first frame _{F n-1} on the pixel _{P2 n-1} and the third frame _{F n + 1} on the pixel P2 Refer to _{n + 1} .

  6A to 6I are graphs showing analysis patterns PA1, PA2,..., PA9 as specific examples of the analysis result information AN of the pixel value change rate by the pixel value change rate analyzing unit 201 shown in FIG. It is. The pixel value change rate analysis unit 201 changes pixel values over three frames, which are motion-compensated based on the motion vector MV output from the motion detection unit 10, in nine patterns shown in FIGS. 6 (a) to 6 (i). It is classified into one of PA1, PA2,.

  FIGS. 6A to 6C show patterns in which the absolute value of the pixel value difference is relatively small. FIG. 6A shows a pattern in which the absolute value of the difference between the pixel values is equal to or less than a first threshold TH1, which is determined in advance. In this case, the change in the pixel value is classified into a minute change pattern PA1. The FIG. 6B shows that the absolute value of the difference between the pixel values is larger than the predetermined first threshold value TH1, but less than or equal to the second threshold value TH2 (however, TH1 <TH2), and over time. A pattern in which the absolute value of the difference between the pixel values increases is shown. In this case, the change in the pixel value is classified into a monotone increasing pattern PA2. FIG. 6C shows that the absolute value of the pixel value difference is larger than the predetermined first threshold value TH1 and less than or equal to the second threshold value TH2, and the absolute value of the pixel value difference decreases with time. In this case, the change in the pixel value is classified into a monotone decreasing pattern PA3.

FIG 6 (d) (f) is greater than the absolute value of the second threshold TH2 of the difference between the pixel values, the pixel of the pixel and the third frame _{F n + 1} on the rear differential DifN (second frame _{F n,} which will be described later The absolute value of the difference between the pixel values between the first frame F _n−1 and the third frame F _{n + 1} ), the forward difference DifP A pattern in which (the absolute value of the difference in pixel value between the pixel on the first frame F _n−1 and the pixel on the second frame F _n ) is equal to or less than a predetermined third threshold value TH3 is shown. FIG. 6D shows that the absolute value of the difference between the pixel values of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} is larger than the second threshold value TH2, and the rear end difference DifN is , DifN ≦ TH3. In this case, the change in pixel value is classified into the forward mismatch pattern PA4. In FIG. 6E, the absolute value of the difference between the pixel values of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} is larger than the second threshold TH2, and the difference DifM between the both ends is A pattern where DifM ≦ TH3 is shown. In this case, the change in pixel value is classified into the center mismatch pattern PA5. FIG. 6F shows that the absolute value of the difference between the pixel values of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} is larger than the second threshold value TH2, and the forward difference DifP is A pattern in which DifP ≦ TH3 is shown. In this case, a change in pixel value is classified into a backward mismatch pattern PA6.

FIGS. 6G to 6I show patterns in which the absolute value of the difference between the pixel values of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} is large. FIG. 6H shows a pattern that is not classified into any of the patterns PA1,..., PA6 of FIGS. 6A to 6F and monotonously increases. Are classified into a monotonous large increase pattern PA8. FIG. 6 (i) shows a pattern that is not classified into any of the patterns PA1,..., PA6 in FIGS. 6 (a) to 6 (f) and monotonously decreases. In this case, the pixel value changes. Are classified into the monotonous large decrease pattern PA9. FIG. 6 (g) shows patterns that are not classified into any of the patterns PA1,..., PA6, PA8, PA9 in FIGS. 6 (a) to (f), (h), (i). The change in pixel value is classified into an irregular pattern PA7. However, the set of analysis result patterns in FIGS. 6A to 6I is an example, and other patterns may be adopted.

FIG. 7 is a flowchart schematically showing a pixel value change pattern classification process executed by the pixel value change rate analysis unit 201 shown in FIG. As shown in FIG. 7, the pixel value change rate analysis unit 201 performs pattern classification for each pixel of the frame of interest in the images of the first, second, and third frames F _n−1 , F _n , and F _{n + 1.} I do.

  First, the pixel value change rate analysis unit 201 sets the coordinates of the initial value of the target pixel (x, y) to (0, 0) (step ST1).

Next, the pixel value change rate analyzing unit 201 calculates the absolute value of the difference between the pixel values between the first, second, and third frames F _n−1 , F _n , and F _{n + 1} , for example, Equations (1) and ( 2) and (3).

DifN (x, y), DifM (x, y), and DifP (x, y) are the backward difference DifN, which is the absolute value of the pixel value difference between the second and third frames F _n and F _{n + 1} , respectively. The absolute value of the difference between both ends DifM, which is the absolute value of the difference between the pixel values between the first and third frames F _n−1 and F _{n + 1} , and the difference between the pixel values between the first and second frames F _n−1 and F _n. A certain forward difference DifP is shown. Further, I _n−1 (x, y), In (x, y), and I _{n + 1} (x, y) are coordinates in the first, second, and third frames F _n−1 , F _n , and F _{n + 1, respectively} . The pixel value of the pixel (x, y) is shown. Further, v _x (x, y) and v _y (x, y) indicate the x and y components of the motion vector at the coordinates (x, y) in the second frame F _n , respectively. The pixel value may be a luminance value of a pixel at coordinates (x, y), or may be a sum of calculation results for each of RGB values.

The pixel value change rate analysis unit 201 obtains the largest value among the absolute values of the pixel value differences between the frames obtained by the equations (1), (2), and (3) as the equation (4). It is determined as the indicated value maxDif (x, y).

  The pixel value change rate analysis unit 201 determines that the pixel value is changed if maxDif (x, y) is equal to or less than a predetermined first threshold value TH1 (maxDif (x, y) ≦ TH1) (YES in step ST2). The change is classified into a minute change pattern PA1 (FIG. 6A) (step ST3).

  In the pixel value change rate analysis unit 201, maxDif (x, y) is larger than the first threshold value TH1 (minDif (x, y)> TH1) (NO in step ST2), and maxDif (x, y) is If the pixel value is less than or equal to a predetermined second threshold TH2 (maxDif (x, y) ≦ TH2) (YES in step ST4) and the pixel value increases with time (YES in step ST5), the pixel The change in value is classified into a monotone increasing pattern PA2 (FIG. 6B) (step ST6).

  The pixel value change rate analysis unit 201 is greater than the first threshold value TH1 (minDif (x, y)> TH1), maxDif (x, y) is equal to or less than the second threshold value TH2, and time elapses. At the same time, when the pixel value decreases (YES in step ST7), the change in the pixel value is classified into a monotonous decrease pattern PA3 (FIG. 6C) (step ST8).

  The pixel value change rate analyzing means 201 is a case where the change in the pixel value is other than the minute change pattern PA1, the monotone increase pattern PA2, and the monotone decrease pattern PA3, and the front end difference DifN (x, y) shown in Expression (1) is calculated in advance. If it is equal to or less than the determined third threshold TH3 (YES in step ST9), the change in pixel value is classified into the forward mismatch pattern PA4 (step ST10), and the value DifM (x, y shown in equation (2) ) Is less than or equal to the third threshold value TH3 (YES in step ST11), the change in pixel value is classified into the center mismatch pattern PA5 (FIG. 6 (e)) (step ST12), as shown in equation (3). When the value DifP (x, y) is equal to or smaller than the third threshold value TH3 (YES in step ST13), the change in the pixel value is classified into the backward mismatch pattern PA6 (FIG. 6 (f)). Flop ST14).

  The pixel value change rate analysis means 201 is a monotonically large change in pixel value when the pixel value changes other than the patterns PA1,..., PA6 and the pixel value increases with time (YES in step ST15). If the pixel value decreases with the passage of time (step ST17: YES), it is classified into the increase pattern PA8 (FIG. 6 (h)) (step ST17: YES). i)) (step ST18), and when the pixel value change is other than the patterns PA1,..., PA6, PA8, PA9 (NO in step ST17), the pixel value change is changed to the irregular pattern PA9 (FIG. 6). (I)) (step ST19).

  Next, the pixel value change rate analysis unit 201 returns the process to step ST1 if the target pixel is not the final pixel in the target frame (one frame), and performs processing if the target pixel is the final pixel in one frame. Is finished (step ST20). When the process returns to step ST1, the pixel value change rate analysis unit 201 changes the coordinates of the initial value of the target pixel (x, y) to unprocessed coordinates, and executes the processes after step ST2.

ただし、動きベクトルの差を式（５）とは異なる式で算出してもよい。 << 1-2-2 >> Motion vector boundary detection means 202
The motion vector boundary detection unit 202 classifies pixels whose motion vector difference between adjacent pixels is equal to or greater than a predetermined threshold as a motion vector boundary. The difference between the motion vectors of adjacent pixels is defined as in Expression (5) based on a Laplacian filter, for example.

However, the motion vector difference may be calculated by an expression different from Expression (5).

<< 1-2-3 >> Frame Blend Rate Calculation Means 204
The frame blend ratio calculation unit 204 calculates a blend weight coefficient BW used when the noise reduction image generation unit 205 calculates a weighted average of the first, second, and third frames F _n−1 , F _n , and F _{n + 1.} To do. The reference frame priority calculation unit 2041 receives the pixel value change rate analysis result information AN output from the pixel value change rate analysis unit 201, and receives first blend weight coefficients (K _N−1 , K _N , K _{N + 1} ). Is calculated. When the pixel value change rate analysis result information AN output from the pixel value change rate analyzing unit 201 is any one of the minute change pattern PA1, the monotone increase pattern PA2, and the monotone decrease pattern PA3, the first blend weight coefficient ( K _N−1 , K _N , K _{N + 1} ) are set as shown in Expression (6). When the analysis result information AN of the pixel value change rate output from the pixel value change rate analyzing unit 201 is the forward mismatch pattern PA4, the first blend weight coefficients (K _N−1 , K _N , K _{N + 1} ) are expressed by the formula When the pixel value change rate analysis result information AN is the center mismatch pattern PA5, the first blend weight coefficients (K _N−1 , K _N , K _{N + 1} ) are expressed by the formula (8). ), And when the pixel value change rate analysis result information AN is the backward mismatch pattern PA6, the first blend weight coefficients (K _N−1 , K _N , K _{N + 1} ) are expressed by the equation (9). In other cases, the first blend weight coefficients (K _N−1 , K _N , K _{N + 1} ) are set as in Expression (10).

For PA1, PA2 and PA3:
(K _N−1 , K _N , K _{N + 1} ) = (1/3, 1/3, 1/3) Equation (6)
For PA4:
(K _N−1 , K _N , K _{N + 1} ) = (1−α, α / 2, α / 2) Equation (7)
For PA5:
(K _N−1 , K _N , K _{N + 1} ) = (α / 2, 1−α, α / 2) Equation (8)
For PA6:
(K _N−1 , K _N , K _{N + 1} ) = (α / 2, α / 2, 1−α) Equation (9)
For PA7, PA8, PA9:
(K _N−1 , K _N , K _{N + 1} ) = ((1−α) / 2, α, (1−α) / 2) Equation (10)
Here, α is a predetermined value of 2/3 or more and 1 or less.

When the pixel value change rate analysis result information AN output from the pixel value change rate analysis unit 201 is any one of the minute change pattern PA1, the monotone increase pattern PA2, and the monotone decrease pattern PA3, the first, second, and second 3 frame _{F n-1,} F _n, the pixel values of the corresponding coordinates of _{F n + 1,} since both are close values, first, second, third frame _{F n-1, F n,} F n + 1 for all These frames are effective for noise reduction.

On the other hand, when the pixel value change rate analysis result information AN output from the pixel value change rate analyzing unit 201 is any one of the front mismatch pattern PA4, the center mismatch pattern PA5, and the rear mismatch pattern PA6, the first frame Since F _n−1 , the second frame F _n and the third frame F _{n + 1} show values that are significantly different from other frames due to the influence of noise or the like, the analysis result of the pixel value change rate during the noise reduction processing As compared with the case where the information AN is any one of the minute change pattern PA1, the monotone increase pattern PA2, and the monotone decrease pattern PA3, noise can be more appropriately reduced by reducing the influence of this frame.

When the analysis result information AN of the pixel value change rate output from the pixel value change rate analysis unit 201 is the patterns PA7, PA8, and PA9, the first, second, and third frames F _n−1 , F _n , and F _{n + 1.} Can be estimated to be inappropriate for noise reduction by taking a weighted average. Therefore, by increasing the weight of the second frame F _n is a target frame noise reduction, to reduce the influence of the reference frame (first and third frames), avoiding the risk of noise from being emphasized doing.

  In the first embodiment, the frame blend rate calculation unit 204 does not distinguish between the minute change pattern PA1, the monotone increase pattern PA2, and the monotone decrease pattern PA3, and the irregular pattern PA7, the monotone large increase pattern. Since the PA8 and the monotonic large decrease pattern PA9 are not distinguished, a simple method that does not distinguish between them may be employed in the classification stage in the pixel value change rate analysis unit 201.

  Further, in the frame blend rate calculation means 204, the value of α in the case of the minute change pattern PA1, the monotone increase pattern PA2, the monotone decrease pattern PA3, the irregular pattern PA7, the monotone large increase pattern PA8, and the monotone large decrease pattern PA9. Detailed processing such as changing the value of α in each case or adjusting individual coefficients may be employed.

FIGS. 8A to 8C are diagrams illustrating an example of occlusion. The occlusion determination means 2042 detects a region where the foreground object hides the background region, and performs noise reduction by selecting an appropriate frame from the first, second, and third frames F _n−1 , F _n , and F _{n + 1.} A second blend weight coefficient is calculated. FIGS. 8A to 8C show the rectangular foreground object 120 facing the right on the stationary background 121 in the first, second, and third frames F _n−1 , F _n , and F _{n + 1} . It shows how it moves. The region 122 on the left side of the foreground object 120 in the second frame F _n is hidden by the foreground object 120 in the first frame F _n−1 . Similarly, the right region 123 of the foreground object 120 is hidden by the foreground object 120 in the third frame F _{n + 1} . That is, it is not appropriate to refer to the first frame F _n−1 when reducing the noise in the region 122, and it is not appropriate to refer to the third frame F _{n + 1} when reducing the noise in the region 123. . The occlusion determination unit 2042 detects a region hidden by the first temporally preceding first frame F _n−1 and the temporally succeeding third frame F _{n + 1} and calculates a blend weight coefficient BW.

The area hidden by the preceding and following frames is distributed around the boundary between the foreground object 120 and the background 121. The boundary can be detected by applying a differential filter such as a Sobel filter to the image. However, when the differential filter is applied, a pattern in the same object is also detected as a boundary. Therefore, in the first embodiment, a motion vector boundary is used. The motion vector boundary pixel information BO is received from the motion vector boundary detection means 202, and the periphery of the motion vector boundary pixel is set as an occlusion region. Specifically, an occlusion area is set based on the difference between the x and y components of a motion vector obtained from pixels of a certain distance in the vertical and horizontal directions for all motion vector boundary pixels in the frame. That is, a motion vector in which a rectangular area having the absolute value of DistX (x, y) represented by Expression (11) as the horizontal width and the absolute value of DistY (x, y) as the vertical width is located at the coordinates (x, y) is shown. This is an occlusion area set around the boundary pixel.
DistX (x, y) = v _x (x + χ, y) −v _x (x−χ, y) Equation (11)
DistY (x, y) = v _y (x, y + χ) −v _y (x, y−χ) Equation (12)
Here, χ is a predetermined distance value for determining a reference pixel in the vertical and horizontal directions, and v _x (x, y) and v _y (x, y) are coordinates in the second frame F _n ( x, y) represents the x, y component of the motion vector.

FIG. 9 is a diagram illustrating an example of the occlusion areas 132 and 133 based on the occlusion determination result OC output from the occlusion determination unit 2042 of the frame blend ratio calculation unit 204 illustrated in FIG. FIG. 9 shows an example in which occlusion areas 132 and 133 (shaded portions) are set around 131 (dark hatched portions). Here, for the sake of simplicity, only the horizontal direction will be described. On the left side of the boundary pixel 131 of the motion vector, a “1” motion is detected rightward in the image, and a leftward “2” (rightward “−2”) motion is detected on the right side of the image. Explain the case. In this case,
DistX (x, y) = (− 2) −1 = −3
It becomes. In this case, since the horizontal width of the occlusion areas 132 and 133 is “3”, the occlusion areas 132 and 133 having the width “3” are set on both sides of the boundary pixel 131 of the motion vector.

As shown in FIGS. 8A to 8C, the occlusion determination unit 2042 determines whether the occlusion area is hidden by the first frame F _n−1 that temporally precedes like the area 122. In this way, it is determined from the boundary pixels 125 and 126 of the motion vector from which the occlusion area is generated whether it is hidden in the third frame F _{n + 1} that temporally follows. That is, the absolute value of DistX (x, y) is compared with the absolute value of DistY (x, y). If the larger sign is positive, the first frame F _n that precedes in time as in the region 122. _If it is hidden by ₋₁ and the larger sign is negative, it is hidden in the third frame F _{n + 1} that temporally follows as in the region 123. Focusing on the boundary pixel 125 of the motion vector, the left side of the boundary is a stationary background, so the movement is “0”, and the right side is moving rightward, and thus has a positive movement in the horizontal direction. DistX (x, y) is positive. On the other hand, paying attention to the boundary pixel 126 of the motion vector, the right side of the boundary is a stationary background, so the motion is “0”, and the left side is moving rightward, and thus has a positive motion in the horizontal direction. DistX (x, y) is negative.

The second blend weight coefficient (C _N−1 , C _N , C _{N + 1} ) determines whether or not the occlusion area is present and whether the occlusion area is hidden in the frame before or after in time. To be determined. The second blend weight coefficients (C _N−1 , C _N , C _{N + 1} ) of the pixels other than the occlusion area are calculated by the equation (13), and the pixels of the area hidden in the temporally subsequent frame in the occlusion area The second blend weight coefficient (C _N−1 , CN, C _{N + 1} ) is calculated by the equation (14), and the second blend weight coefficient ((2) of the pixel in the region hidden in the temporally preceding frame in the occlusion region ( C _N−1 , C _N , C _{N + 1} ) are calculated by the equation (15).
(C _N−1 , C _N , C _{N + 1} ) = (1/3, 1/3, 1/3) Equation (13)
(C _N−1 , C _N , C _{N + 1} ) = (β / 2, β / 2, 1−β) Equation (14)
(C _N−1 , C _N , C _{N + 1} ) = (1−β, β / 2, β / 2) Equation (15)
Here, β is a predetermined value of 2/3 or more and 1 or less.

  In this way, by focusing on the boundary pixels of the motion vector and setting the periphery thereof as an occlusion area, it is possible to reduce noise by selecting an appropriate frame in consideration of hiding by the foreground object.

The blend weight coefficient calculation means 2043 uses the first blend weight coefficient calculated by the reference frame priority calculation means 2041 and the second blend weight coefficient calculated by the occlusion determination means 2042, and the noise reduction image generation means 205 uses the first blend weight coefficient. Blend weight coefficients (B _N−1 , B _N , B _{N + 1} ) for calculating the weighted average of the first, second, and third frames F _n−1 , F _n , and F _{n + 1} are calculated. The calculation formula is formula (16) or formulas (17) and (18).

ここで、γは、０以上１以下の所定の値、δは、ブレンド重み係数の正規化のための値である。

Here, γ is a predetermined value between 0 and 1, and δ is a value for normalizing the blend weight coefficient.

<< 1-2-4 >> Noise Reduction Image Generating Unit 205
The noise reduction image generation unit 205 uses the blend weight coefficients (B _N−1 , B _N , B _{N + 1} ) calculated by the frame blend rate calculation unit 204 to reduce noise from data input through the input terminal 207. Generate an image. For example, the noise reduced image generation unit 205 generates a noise reduced image by calculating each pixel value NR (x, y) of the noise reduced image according to Expression (19).

<< 1-2-5 >> Motion Error Detection Area Correction Coefficient Calculation Means 203
The motion error detection region correction coefficient calculation unit 203 receives the analysis result information AN of the pixel value change rate from the pixel value change rate analysis unit 201, and receives the motion vector MV and the motion vector boundary pixel information BO from the motion vector boundary detection unit 202. In order to detect a pixel having an error in motion detected by the motion detector 10 based on the analysis result information AN of the pixel value change rate, the motion vector MV, and the motion vector boundary pixel information BO, and to correct the pixel The correction coefficient ER is calculated.

The pixel value sudden change region detection unit 2031 receives the analysis result information AN of the pixel value change rate via the input terminal 2035, and detects a pixel whose pixel value is changing rapidly due to an erroneous motion vector. The pixel value has a sudden change in the analysis result information AN of the pixel value change rate, that is, the front mismatch pattern PA4, the center mismatch pattern PA5, the rear mismatch pattern PA6, the monotonically large increase pattern PA8, the monotonously large decrease pattern PA9, This is the case of the irregular pattern PA7. However, the forward mismatch pattern PA4, the center mismatch pattern PA5, and the backward mismatch pattern PA6 can be presumed that the pixel value has changed suddenly due to occlusion, and the frame blend rate calculation means 204 calculates a coefficient corresponding to the occlusion. Since noise reduction is performed, these three patterns are not subjected to correction using the correction coefficient ER. On the other hand, in the case of the monotonous large increase pattern PA8, the monotonous large decrease pattern PA9, and the irregular pattern PA7, the values are large over the three frames of the first, second, and third frames F _n−1 , F _n , and F _{n + 1.} Since it has changed and the motion vector is incorrect, it is presumed that a value of a different object has been referred to. Therefore, the pixel value sudden change region detection means 2031 outputs the pixels classified into the monotone large increase pattern PA8, the monotone large decrease pattern PA9, and the irregular pattern PA7 as correction target pixels.

FIG. 10 is a diagram illustrating an example of the reference frame indefinite region 134 detected by the reference frame indefinite region detecting unit 2032 of the motion error detection region correction coefficient calculating unit 203 illustrated in FIG. The reference frame indefinite region detection unit 2032 receives the motion vector MV and the motion vector boundary pixel information BO via the input terminal 2036, and outputs a region where the reference region cannot be determined as a correction target. From the motion vector MV and the motion vector boundary pixel information BO, the occlusion determination means 2042 sets the periphery of the motion vector boundary pixel 131 as the occlusion regions 132 and 133 as shown in FIG. It was determined whether the first frame F _n−1 or the third frame F _{n + 1} was hidden by the foreground object. On the other hand, as shown in FIG. 10, the reference frame indefinite area detection unit 2032 has a motion vector boundary pixel 131 (dark hatched portion) and a motion vector boundary pixel 141 (dark hatched portion) close to each other. Then, a region 134 in which the peripheral regions 132 and 133 of the motion vector boundary pixel 131 and the peripheral regions 142 and 143 of the motion vector boundary pixel 141 overlap is generated. Since the image data is overwritten in the occlusion area generated later, the occlusion determination means 2042 evaluates the boundary pixel 131 of the motion vector first, and evaluates the boundary pixel 141 of the motion vector later. Is determined to be the occlusion region 142 around the boundary pixel 141 of the motion vector. However, it may be appropriate to treat the area 134 as belonging to the occlusion area 133. Further, since it is rare that the motion vector boundaries become closer as the occlusion regions overlap each other, it can be estimated that the motion estimation unit 10 has erroneously detected a motion vector. Therefore, the reference frame indefinite area detecting unit 2032 outputs information (reference frame indefinite area information) RR that is a correction target for the area 134 where the occlusion areas overlap.

  The motion vector boundary concentration region detection unit 2033 receives the motion vector MV and the motion vector boundary pixel information BO via the input terminal 2036, and sets information on the region where the motion vector boundary is locally concentrated as a correction target region (motion Vector concentrated area information) BR is output. Since it is rare that motion vectors of various magnitude directions are mixed in a narrow region and the motion vector boundaries are concentrated, the motion vector boundaries are locally localized in the motion estimation unit 10 due to erroneous detection of motion vectors. You can guess that it is concentrated.

  FIG. 11 is a diagram illustrating an example of a motion vector concentration region detected by the motion vector boundary concentration region detection unit 2033 of the motion error detection region correction coefficient calculation unit 203 illustrated in FIG. The motion vector boundary concentration area detection unit 2033 performs block processing to determine whether or not the boundary pixels 151 (dark hatched portions) of the motion vector are concentrated. A predetermined threshold value is set, and the number of motion vector boundary pixels 151 in the pixels in the block (a thick square area of 16 pixels in 4 rows and 4 columns in FIG. 11) exceeds a predetermined threshold value. The block is output as a pixel to be corrected, and the pixels in the block are output as correction targets. FIG. 11 shows processing in a four-pixel square block. In the block 154, four of the 16 pixels in the block are the boundary pixels 151 of the motion vector. On the other hand, in the block 155, 7 out of 16 pixels are motion vector boundary pixels 151. When the threshold value is “5”, the block 154 does not correspond to a motion vector boundary pixel concentrated block, but the block 155 corresponds to a motion vector boundary pixel concentrated block. For this reason, the motion vector boundary concentration region detection unit 2033 outputs information (motion vector concentration region information) BR indicating that 16 pixels belonging to the block 155 are correction target pixels.

  The correction coefficient calculation unit 2034 receives information on the correction target pixel from each of the pixel value sudden change region detection unit 2031, the reference frame indefinite region detection unit 2032, and the motion vector boundary concentration region detection unit 2033. A pixel correction coefficient is calculated. The correction coefficient Conp (x, y) is one of the output results of the pixel value sudden change region detection unit 2031, the reference frame indefinite region detection unit 2032, and the motion vector boundary concentration region detection unit 2033, as shown in Expression (20). If it is a correction target, “1” is set, and if both are not correction targets, “0” is set.

ここで、Ｓ_１（ｘ，ｙ），Ｓ_２（ｘ，ｙ），Ｓ_３（ｘ，ｙ）は、座標（ｘ，ｙ）について、画素値急変領域検出手段２０３１、参照フレーム不定領域検出手段２０３２、動きベクトル境界集中領域検出手段２０３３の出力結果が補正対象なら「真」、補正対象外なら「偽」を出力する関数である。

Here, S ₁ (x, y), S ₂ (x, y), and S ₃ (x, y) are pixel value sudden change area detection means 2031 and reference frame indefinite area detection means for coordinates (x, y). 2032, a function that outputs “true” if the output result of the motion vector boundary concentration area detection unit 2033 is a correction target, and “false” if it is not a correction target.

  The correction coefficient may be calculated based on an expression other than Expression (20). For example, as shown in Expression (21), the correction coefficient is calculated by weighting the output results of the pixel value sudden change region detection unit 2031, the reference frame indefinite region detection unit 2032, and the motion vector boundary concentration region detection unit 2033. May be.

ここで、Ｐ_１（ｘ，ｙ），Ｐ_２（ｘ，ｙ），Ｐ_３（ｘ，ｙ）は、座標（ｘ，ｙ）について、画素値急変領域検出手段２０３１、参照フレーム不定領域検出手段２０３２、動きベクトル境界集中領域検出手段２０３３の出力結果が補正対象なら「１」、補正対象外なら「０」を出力する関数であり、Ｃｏｌ_１，Ｃｏｌ_２は、予め決められたブレンド重み係数である。

Here, P ₁ (x, y), P ₂ (x, y), and P ₃ (x, y) are pixel value sudden change area detection means 2031 and reference frame indefinite area detection means for coordinates (x, y). 2032 is a function that outputs “1” if the output result of the motion vector boundary concentration area detection unit 2033 is a correction target and “0” if it is not a correction target. Col ₁ and Col ₂ are blend weight coefficients determined in advance. is there.

Further, as shown in the equations (22) and (23), even if the correction coefficient is changed according to the difference in the pixel values between the first, second, and third frames F _n−1 , F _n , and F _{n + 1.} Good.

ここで、ＣｏｎｐＩ（ｘ，ｙ）は、座標（ｘ，ｙ）に対する画素値の差に応じて変更した補正係数であり、ｍａｘＤｉｆ（ｘ，ｙ）は、式（４）に示されるとおりであり、ＬＩＭは、補正上限の閾値である。

Here, CompI (x, y) is a correction coefficient changed according to the difference in pixel value with respect to coordinates (x, y), and maxDif (x, y) is as shown in Expression (4). , LIM is a correction upper limit threshold.

<< 1-2-6 >> Motion Error Detection Area Correction Unit 206
As shown in FIG. 2, the motion error detection region correction unit 206 receives the first, second, and third frames F _n−1 , F _n , and F _{n + 1} via the input terminal 207, and noise reduced image generation unit The noise-reduced image FS3 is received from 205, the correction coefficient ER is received from the motion error detection area correction coefficient calculation means 203, and the disturbance of the noise-reduced image due to motion error detection is corrected using the received information FS2 and ER. The correction is performed according to the correction coefficient ER, for example, using equations (24) and (25).

ここで、ＣＬＰＦは、０から１の範囲内において予め決められた係数である。また、ＬＰＦ（ｘ，ｙ）は、平滑化のために算出した座標（ｘ，ｙ）を中心とする３画素四方（すなわち、３行３列の９画素）の正方領域の画素値の平均値である。ただし、ＬＰＦ（ｘ、ｙ）の算出に、より広い領域又はより狭い領域における平均値又は重みを付けした平均値を採用してもよい。

Here, CLPF is a coefficient determined in the range of 0 to 1. LPF (x, y) is an average value of pixel values in a square area of 3 pixels square (ie, 9 pixels in 3 rows and 3 columns) centered on coordinates (x, y) calculated for smoothing. It is. However, an average value or a weighted average value in a wider area or a narrower area may be employed for calculating LPF (x, y).

<< 1-3 >> Effects of the First Embodiment According to the video processing device 1 and the video processing method according to the first embodiment described above, processing is performed according to the blend rate according to the concealment or the change in the pixel value. In addition, by detecting a region having a motion vector error and correcting this region, it is possible to reduce noise while suppressing deterioration in video quality due to erroneous detection of the motion vector.

<< 2 >> Embodiment 2
FIG. 12 is a functional block diagram schematically showing the configuration of the video processing apparatus 2 (that is, the apparatus capable of performing the video processing method according to the second embodiment) 2 according to the second embodiment of the present invention. 12, the same reference numerals are given to the same or corresponding components as those shown in FIG. The video processing device 2 according to the second embodiment includes a frame buffer 30 that stores a plurality of frames constituting a moving image, and a first temporally adjacent to each other among the plurality of frames constituting the moving image. The motion detector 10a receives the second and third frames, detects the motion between these frames, and outputs the motion vector MV, and the pixel-based motion vector MV and the first motion vector MV output from the motion detector 10a. receives the second, third frame, and the noise reduction unit 20a for generating a noise by reducing the noise reduction frame FS3a of the second frame F _n is a frame of interest, the frame buffer 40 for storing the noise reduction frame as image data Have The image processing device 2 and the image processing method according to the second embodiment are different from the image processing device 1 and the image processing method according to the first embodiment in terms of a frame that is referred to when generating the noise reduction frame FS3a. In other respects, the video processing device 2 and the video processing method according to the second embodiment are the same as the video processing device 1 and the video processing method according to the first embodiment.

FIG. 13 is a diagram illustrating frames used for noise reduction processing in the video processing device 1 according to the first embodiment, and is a diagram for explaining the second embodiment in comparison with the first embodiment. As shown in FIG. 13, in the first embodiment, the motion detection unit 10 and the noise reduction unit 20 include first and second temporally adjacent to each other as a series of frames constituting a moving image. The third frames F _n−1 , F _n , and F _{n + 1} are input. In the first embodiment, as shown in FIG. 13, the noise reduction frame M _n corresponding to the second frame F _n in terms of time is represented by the first, second, and third frames F that are temporally adjacent. It generates based on _n-1 , _Fn , and _{Fn + 1} .

FIG. 14 is a diagram illustrating frames used for noise reduction processing in the video processing device 2 according to the second embodiment. As shown in FIG. 14, in the second embodiment, the motion detection unit 10a and the noise reduction unit 20a have three frames that are temporally adjacent to each other as a series of frames constituting a moving image. Among them, the noise reduction unit 20a generates the first frame in the past, and substitutes the noise reduction frame M _n−1 stored in the frame buffer 40. In the video processing device 2 according to the second embodiment, as illustrated in FIG. 14, the noise reduction frame M _n corresponding to the second frame F _n in terms of time is already generated and stored in the frame buffer 40. Generated based on the noise reduction frame M _n−1 and the second and third frames F _n and F _{n + 1} .

FIG. 15 schematically shows a configuration of a video processing apparatus 3 according to a modification of the second embodiment of the present invention (that is, an apparatus capable of performing the video processing method according to the modification of the second embodiment). It is a functional block diagram. In FIG. 15, the same or corresponding components as those shown in FIG. The video processing device 3 according to the modification of the second embodiment includes a frame buffer 30 that stores a plurality of frames constituting a moving image and a temporally adjacent one of the plurality of frames constituting the moving image. Motion detection unit 10b that receives the first, second, and third frames to be detected, detects a motion between these frames, and outputs the motion vector MV, and a motion vector MV in pixel units output from the motion detection unit 10b and first, second, receiving a third frame, the frame store and a noise reduction unit 20b for generating a second frame F _n noise by reducing the noise reduction frame FS3a of a frame of interest, the noise reduction frame as image data And a buffer 40. The image processing device 3 and the image processing method according to the modification of the second embodiment are the same as the image processing device 2 and the image processing method according to the second embodiment in terms of the frame that is referred to when the noise reduction frame FS3b is generated. Is different.

FIG. 16 is a diagram illustrating frames used for noise reduction processing in the video processing device 3 according to the modification of the second embodiment. As shown in FIG. 16, in the modification of the second embodiment, the motion detection unit 10b and the noise reduction unit 20b are temporally adjacent inputs as a series of frames constituting a moving image. Two of the three frames are generated by the noise reduction unit 20b in the past, and the noise reduction frames M _n−1 and M _n stored in the frame buffer 40 are used instead. Then, as shown in FIG. 16, in the modification of the second embodiment, temporal noise reduction frame corresponding to the second frame F _n, the noise reduction already stored in the frame buffer 40 generated frame M _{It is} generated from _n−1 , M _n and the third frame F _{n + 1} .

As described above, according to the video processing device 2 and the video processing method according to the second embodiment, and the video processing device 3 and the video processing method according to the modification of the second embodiment, the noise reduction unit 20a or 20b uses, as a reference frame, at least one noise reduction frame that is generated in the past and temporally precedes the frame of interest. Specifically, the noise reduction unit 20a or 20b uses the noise reduction frame M _n−1 generated in the past or the noise reduction frames M _n−1 and M _n generated in the past to generate the frame of interest. Since the noise reduction process is newly performed to generate a noise reduction frame, a higher noise reduction effect can be obtained.

<< 3 >> Embodiment 3
FIG. 17 is a functional block diagram schematically showing the configuration of the broadcast receiving device 4 according to Embodiment 3 of the present invention. In FIG. 17, the same or corresponding components as those shown in FIG. As shown in FIG. 17, the broadcast receiving device (or image receiving device) 4 according to Embodiment 3 receives a broadcast wave, demultiplexes video and audio, and outputs a decoded video frame. 51, a frame buffer 30 for storing a plurality of frames constituting a moving image based on a broadcast wave received by the broadcast receiving unit 51, and a temporally adjacent one of a plurality of frames constituting the moving image A motion detection unit 10 that receives the first, second, and third frames F _n−1 , F _n , and F _{n + 1} to detect the motion between these frames and outputs them as a motion vector MV; pixel unit output from the motion vector MV and the first, second, third frame _{F n-1,} F _n, receives the _{F n + 1,} to reduce the noise of the second frame _{F n} is a frame of interest With a noise reduction unit 20 for generating a noise reduction frame FS3, a frame buffer 40 for storing the noise reduction frame as image data, and a video display unit 52 for displaying an image based on the noise reduction frame stored in the frame buffer 40. The broadcast receiving device 4 according to the third embodiment is different from the video processing device 1 according to the first embodiment in that a broadcast receiving unit 51 and a video display unit 52 are provided. For other configurations, the third embodiment is the same as the first embodiment.

  FIG. 18 is a functional block diagram schematically showing a configuration of broadcast receiving apparatus 5 according to a modification of the third embodiment of the present invention. In FIG. 18, the same or corresponding components as those shown in FIG. As shown in FIG. 18, the broadcast receiving apparatus 5 according to the modification of the third embodiment replaces the motion detection unit 10 and the noise reduction unit 20 with a motion detection unit 10a (or 10b) and a noise reduction unit 20a ( Or, the point provided with 20b) is different from the broadcast receiving apparatus 4 according to the third embodiment. Regarding other configurations, the modification of the third embodiment is the same as the third embodiment (FIG. 17). The motion detection unit 10a (or 10b) and the noise reduction unit 20a (or 20b) are the same as those described with reference to FIG. 12 (or FIG. 15).

  As described above, according to the broadcast receiving device 4 according to the third embodiment and the broadcast receiving device 5 according to the modification of the third embodiment, the noise of the video based on the received broadcast wave is reduced. The quality of the video displayed on the video display unit 52 can be improved.

Further, according to the broadcast receiving device 5 according to the modification of the third embodiment, the noise reduction frame M _n−1 generated in the past or the noise reduction frames M _n−1 and M _n generated in the past are received. Since a new noise reduction frame is generated using this, a higher noise reduction effect can be obtained.

<< 4 >> Embodiment 4
FIG. 19 is a functional block diagram schematically showing the configuration of the video imaging apparatus 6 according to Embodiment 4 of the present invention. In FIG. 19, the same or corresponding components as those shown in FIG. As shown in FIG. 19, the video imaging device 6 according to the fourth embodiment generates a series of a plurality of moving images by generating an electrical signal according to the intensity of light incident through an optical system such as a lens. An imaging unit 61 that generates an image frame, a frame buffer 30 that stores a series of frames generated by the imaging unit 61, and first temporally adjacent ones of a series of frames that form a moving image. , The second and third frames F _n−1 , F _n , F _{n + 1} are received, the motion between these frames is detected and output as a motion vector MV, and output from the motion detector 10 motion vector MV and the first pixel unit and, second, third frame _{F n-1,} F _n, receives the _{F n + 1,} the noise low to reduce noise of the second frame _{F n} is a frame of interest A noise reduction unit 20 that generates a frame FS3, a frame buffer 40 that stores the noise reduction frame as image data, and a video that records the noise reduction frame stored in the frame buffer 40 on a recording medium of a hard disk, an optical disk, or a semiconductor storage device And a recording unit 62. The video imaging device 6 according to the fourth embodiment is different from the video processing device 1 according to the first embodiment in that an imaging unit 61 and a video recording unit 62 are provided. For other configurations, the fourth embodiment is the same as the first embodiment.

  FIG. 20 is a functional block diagram schematically showing the configuration of the video imaging apparatus 7 according to a modification of the fourth embodiment of the present invention. 20, the same reference numerals are given to the same or corresponding components as those shown in FIG. As illustrated in FIG. 20, the video imaging apparatus 7 according to the modification of the fourth embodiment replaces the motion detection unit 10 and the noise reduction unit 20 with a motion detection unit 10a (or 10b) and a noise reduction unit 20a ( Alternatively, the fourth embodiment is different from the video photographing apparatus 6 in that it includes 20b). Regarding other configurations, the modification of the fourth embodiment is the same as the fourth embodiment (FIG. 19). The difference from the broadcast receiving apparatus 4 according to the third embodiment is that the motion detection unit 10a (or 10b) and the noise reduction unit 20a (or 20b) are provided. Regarding other configurations, the modification of the third embodiment is the same as the third embodiment (FIG. 17). The motion detection unit 10a (or 10b) and the noise reduction unit 20a (or 20b) are the same as those described with reference to FIG. 12 (or FIG. 15). The motion detection unit 10a (or 10b) and the noise reduction unit 20a (or 20b) are the same as those described with reference to FIG. 12 (or FIG. 15).

  As described above, according to the video imaging device 6 according to the fourth embodiment and the video imaging device 7 according to the modification of the fourth embodiment, the noise of the video including noise generated in the imaging unit 61 is reduced. The quality of the video recorded in the video recording unit 62 can be improved.

Moreover, according to the video imaging device 7 according to the modification of the fourth embodiment, the noise reduction frame M _n−1 generated in the past or the noise reduction frames M _n−1 and M _n generated in the past are used. Since the noise reduction frame is generated by using it, a higher noise reduction effect can be obtained.

<< 5 >> Embodiment 5
FIG. 21 is a functional block diagram schematically showing the configuration of the video storage device 8 according to Embodiment 5 of the present invention. In FIG. 21, the same or corresponding elements as those shown in FIG. As shown in FIG. 21, the video storage apparatus 8 according to the fifth embodiment includes the first, second, and third frames F _n that are temporally adjacent to each other in a series of frames constituting a moving image. _-1 , F _n , F _{n + 1} are received, the motion between these frames is detected and output as a motion vector MV, the pixel-based motion vector MV output from the motion detector 10 and the first A noise reduction unit 20 that receives the first, second, and third frames F _n−1 , F _n , and F _{n + 1} and reduces the noise of the second frame F _n that is the frame of interest to generate the noise reduction frame FS3; A frame buffer 40 that stores reduced frames as image data, and a noise reduction frame stored in the frame buffer 40 are stored in a hard disk, an optical disk, or a semiconductor memory device. In the video storage device 8 according to the fifth embodiment having the video recording unit 62 for recording on a medium, a series of a plurality of frames stored in the video recording unit 62 is supplied to the motion detection unit 10 and the noise reduction unit 20. Is done. The video storage device 8 according to the fifth embodiment is related to the first embodiment in that a series of a plurality of frames stored in the video recording unit 62 are supplied to the motion detection unit 10 and the noise reduction unit 20. Different from the video processing apparatus 1. For other configurations, the fifth embodiment is the same as the first embodiment.

  FIG. 22 is a functional block diagram schematically showing the configuration of the video storage device 9 according to a modification of the fifth embodiment of the present invention. 22, the same reference numerals are given to the same or corresponding components as those shown in FIG. As illustrated in FIG. 22, the video storage device 9 according to the modification of the fifth embodiment replaces the motion detection unit 10 and the noise reduction unit 20 with a motion detection unit 10a (or 10b) and a noise reduction unit 20a ( Alternatively, the fifth embodiment is different from the video storage device 8 in that it includes 20b). Regarding other configurations, the modification of the fifth embodiment is the same as the fifth embodiment (FIG. 21). The motion detection unit 10a (or 10b) and the noise reduction unit 20a (or 20b) are the same as those described with reference to FIG. 12 (or FIG. 15). The motion detection unit 10a (or 10b) and the noise reduction unit 20a (or 20b) are the same as those described with reference to FIG. 12 (or FIG. 15).

  As described above, according to the video storage device 8 according to the fifth embodiment and the video shooting device 9 according to the modification of the fifth embodiment, the noise of the video frames stored without performing the noise reduction process Can be recorded in the video recording unit 62 as a new noise reduction frame.

Further, according to the video storage device 9 according to the modification of the fifth embodiment, the noise reduction frame M _n−1 generated in the past or the noise reduction frames M _n−1 and M _n generated in the past are used. Since the noise reduction frame is generated by using it, a higher noise reduction effect can be obtained.

  The present invention relates to an apparatus for performing processing for reducing noise included in a video signal, such as a television receiver, a broadcast receiver, a video recording / reproducing apparatus, a personal computer, and a communication terminal such as a mobile phone and a smartphone. The present invention can be applied to a method for performing processing for reducing noise contained in a video signal and a program for causing a device to execute processing for reducing noise contained in a video signal.

1, 2, 3 Video processing device, 4, 5 Broadcast receiving device, 6, 7 Video shooting device, 8, 9 Video storage device, 10, 10a, 10b Motion detection unit, 20, 20a, 20b Noise reduction unit, 30, 40 frame buffer, 51 broadcast receiving unit, 52 video display unit, 61 imaging unit, 62 video recording unit, 201 pixel value change rate analysis unit, 202 motion vector boundary detection unit, 203 motion error detection area correction coefficient calculation unit, 204 frame Blend rate calculation means, 205 noise reduction image generation means, 206 motion error detection area correction means, 2031 pixel value sudden change area detection means, 2032 reference frame indefinite area detection means, 2033 motion vector boundary concentration area detection means, 2034 correction coefficient calculation means 2041 reference frame priority calculation means, 2042 occlusion determination means, 2043 Blend weight coefficient calculation means, AN pixel value change rate analysis result information, BO motion vector boundary pixel information, BR motion vector concentration area information, BW blend weight coefficient, ER correction coefficient, F _n−1 , F _n , F _{n + 1} 1st, 2nd, 3rd frame, FS frame, FS2 noise reduction image, FS3, FS3a, FS3b noise reduction frame, _Mn-1 , _Mn noise reduction frame, MV motion vector, OC occlusion judgment result, PR priority RR Reference frame indefinite area information, SR correction target area information.

Claims (14)

One of a plurality of frames constituting a moving image is set as a frame of interest, and at least one of a temporally preceding frame and a temporally subsequent frame of the frame of interest is used as a motion detection reference frame. A motion detector that estimates a motion between a frame and the motion detection reference frame, and detects a motion vector in units of pixels constituting the frame of interest;
Using the noise reduction reference frame that is at least one of the motion vector detected by the motion detector, the frame of interest, a frame temporally preceding the frame of interest, and a frame temporally following the frame of interest. A noise reduction unit that generates a noise reduction frame in which the noise of the frame is reduced,
The noise reduction unit is
Motion vector boundary detection means for detecting boundary pixels of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixels;
A pixel value change rate analysis unit that analyzes a pixel value change rate between the frame of interest and the noise reduction reference frame, and generates analysis result information of the pixel value change rate;
A video processing apparatus comprising: correction means for generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate. The noise reduction unit uses the pixel value change rate analysis result information generated by the pixel value change rate analysis unit and the motion vector boundary pixel information generated by the motion vector boundary detection unit, and A frame blend rate calculating means for calculating a frame synthesis coefficient for each pixel of the frame;
The generation of the noise reduction frame by the correction unit is performed using the motion vector boundary pixel information, the analysis result information of the pixel value change rate, and the frame synthesis coefficient. Video processing device.   The analysis result information of the pixel value change rate generated by the pixel value change rate analyzing means includes a change in pixel value of a pixel having the same coordinate between the target frame and the noise reduction reference frame. The video processing apparatus according to claim 1, wherein the image processing apparatus is information indicating which of a plurality of predetermined pixel value change patterns is classified for each pixel. The noise reduction unit detects the motion based on the analysis result information of the pixel value change rate generated by the pixel value change rate analysis unit and the motion vector boundary pixel information generated by the motion vector boundary detection unit. A motion erroneous detection region correction coefficient calculating means for detecting a region where motion detection by the unit can be estimated to be a false detection and determining a correction coefficient for each pixel;
The video processing apparatus according to claim 4, wherein the correction unit generates the noise reduced image using a correction coefficient for each pixel. The motion error detection region correction coefficient calculation means is
A pixel value sudden change region detecting means for detecting a portion where the change rate of the pixel value is large and generating correction target region information indicating the detected portion;
A reference frame indefinite area detecting means for generating reference frame indefinite area information indicating an area in which it is impossible to determine whether it is a foreground part or a background in a frame adjacent to the frame of interest;
Motion vector boundary concentration area detecting means for generating motion vector concentration area information indicating an area where the boundary of the motion vector is concentrated;
5. The video processing apparatus according to claim 4, further comprising correction coefficient calculation means for determining the correction coefficient based on the correction target area information, the reference frame indefinite area information, and the motion vector concentration area information. A storage unit for storing the noise reduction frame;
The noise reduction unit reads at least one noise reduction frame temporally preceding the frame of interest generated in the past from the storage unit, and uses it as the noise reduction reference frame. 6. The video processing apparatus according to any one of items 1 to 5. One of a plurality of frames constituting a moving image is set as a frame of interest, and at least one of a temporally preceding frame and a temporally subsequent frame of the frame of interest is used as a motion detection reference frame. A motion detection step of estimating a motion between a frame and the motion detection reference frame, and detecting a motion vector in pixel units constituting the frame of interest;
Using the noise reduction reference frame that is at least one of the motion vector detected in the motion detection step, the frame of interest, and a frame temporally preceding and temporally following the frame of interest. A noise reduction step for generating a noise reduction frame in which the noise of the frame is reduced, and
The noise reduction step includes
A motion vector boundary detection step for detecting a boundary pixel of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixel;
A pixel value change rate analysis step of analyzing a pixel value change rate between the frame of interest and the motion detection reference frame, and generating analysis result information of the pixel value change rate;
And a noise reduction frame generation step of generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate. A broadcast receiving unit that receives a broadcast wave and decodes video information included in the broadcast wave;
A series of frames constituting a moving image is received from the broadcast receiving unit, one of the series of frames is set as a frame of interest, and a frame temporally preceding and temporally following the frame of interest Using at least one of them as a motion detection reference frame, estimating a motion between the target frame and the motion detection reference frame, and detecting a motion vector in units of pixels constituting the target frame;
Using the noise reduction reference frame that is at least one of the motion vector detected by the motion detector, the frame of interest, a frame temporally preceding the frame of interest, and a frame temporally following the frame of interest. A noise reduction unit that generates a noise reduction frame in which the noise of the frame is reduced,
The noise reduction unit is
Motion vector boundary detection means for detecting boundary pixels of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixels;
A pixel value change rate analyzing means for analyzing a pixel value change rate between the frame of interest and the motion detection reference frame, and generating analysis result information of the pixel value change rate;
A broadcast receiving apparatus comprising: correction means for generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate. A storage unit for storing the noise reduction frame;
The broadcast receiving apparatus according to claim 8, wherein the noise reduction unit reads at least one noise reduction frame generated in the past from the storage unit and uses the frame as the noise reduction reference frame. An imaging unit that shoots video and generates a series of frame data constituting a moving image;
A series of a plurality of frames constituting a moving image is received from the imaging unit, one of the series of frames is set as a frame of interest, and at least a frame temporally preceding and temporally following the frame of interest Using one of them as a motion detection reference frame, estimating a motion between the target frame and the motion detection reference frame, and detecting a motion vector in units of pixels constituting the target frame;
Using the noise reduction reference frame that is at least one of the motion vector detected by the motion detector, the frame of interest, a frame temporally preceding the frame of interest, and a frame temporally following the frame of interest. A noise reduction unit that generates a noise reduction frame in which the noise of the frame is reduced,
The noise reduction unit is
Motion vector boundary detection means for detecting boundary pixels of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixels;
A pixel value change rate analyzing means for analyzing a pixel value change rate between the frame of interest and the motion detection reference frame, and generating analysis result information of the pixel value change rate;
A video photographing apparatus comprising: correction means for generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate. A storage unit for storing the noise reduction frame;
The video imaging apparatus according to claim 10, wherein the noise reduction unit reads at least one noise reduction frame generated in the past from the storage unit and uses the read frame as the noise reduction reference frame. A storage unit for storing a series of frames constituting a moving image;
One of the series of frames stored in the storage unit is set as a target frame, and at least one of a temporally preceding frame and a temporally subsequent frame is used as a motion detection reference frame. A motion detection unit that estimates a motion between the frame of interest and the motion detection reference frame, and detects a motion vector in units of pixels constituting the frame of interest;
Using the noise reduction reference frame that is at least one of the motion vector detected by the motion detector, the frame of interest, a frame temporally preceding the frame of interest, and a frame temporally following the frame of interest. A noise reduction unit that generates a noise reduction frame in which the noise of the frame is reduced;
Have
The noise reduction unit is
Motion vector boundary detection means for detecting boundary pixels of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixels;
A pixel value change rate analyzing means for analyzing a pixel value change rate between the frame of interest and the motion detection reference frame, and generating analysis result information of the pixel value change rate;
A video storage device comprising: correction means for generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate.   The video accumulation apparatus according to claim 12, wherein the noise reduction unit reads at least one noise reduction frame generated in the past from the storage unit and uses the frame as the noise reduction reference frame. A program for causing a computer to execute processing,
The process is
One of a plurality of frames constituting a moving image is set as a frame of interest, and at least one of a temporally preceding frame and a temporally subsequent frame of the frame of interest is used as a motion detection reference frame. A motion detection process for estimating a motion between a frame and the motion detection reference frame and detecting a motion vector in units of pixels constituting the frame of interest;
Using the reference vector for noise reduction that is at least one of the motion vector detected in the motion detection process, the frame of interest, a frame temporally preceding and temporally following the frame of interest Including noise reduction processing for generating a noise reduction frame in which noise of the frame is reduced,
The noise reduction process
A motion vector boundary detection process for detecting a boundary pixel of the motion vector and generating motion vector boundary pixel information indicating the detected boundary pixel;
A pixel value change rate analysis process for analyzing a pixel value change rate between the frame of interest and the motion detection reference frame, and generating analysis result information of the pixel value change rate;
A noise reduction frame generation process for generating the noise reduction frame using the motion vector boundary pixel information and the analysis result information of the pixel value change rate.

Images