JP2010226646A - Video processing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To effectively reduce random noise even when frames of the same image are continued. <P>SOLUTION: A video processing apparatus includes: a first frame buffer 121 for storing an image of a frame preceding to an input frame; a comparison and discrimination section 123 for comparing an image of the input frame with the image stored in the first frame buffer 121 and discriminating whether or not both images are identical; and a cyclic three-dimensional (3D) noise reduction section 129 which uses a cyclic coefficient K to perform cyclic 3D noise cancellation processing on the image stored in the first frame buffer 121. When discrimination section 123 has discriminated that both images are identical, the comparison and discrimination section 123 sets the cyclic coefficient K to 1. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to a video processing apparatus, and more particularly to a video processing apparatus that can effectively reduce random noise even when the same frame image is continuous.

With the recent development of IT technology and network technology, networking of video surveillance systems is also progressing in the security field. In a networked video surveillance system, a network recorder is arranged on the surveillance side and video taken by a network camera is recorded widely.
In general, video surveillance systems often monitor at night or in dark places, so it is required to display dark video brightly and clearly. For example, Patent Document 1 describes that a low-luminance area of an image is provided with a gradation thickness so that an image in a dark place or a backlight is displayed easily. In this case, since noise components generated in the low luminance region are also emphasized, it is desirable to perform noise reduction processing separately.

  As a typical technique for reducing noise, two-dimensional noise reduction (2DNR) that performs noise reduction processing within one frame of image data and noise reduction processing that focuses on correlation between adjacent frames are performed. Three-dimensional noise reduction (3DNR) is known.

  In the three-dimensional noise reduction, there are widely used ones that improve the S / N by adding the image of the previous frame, or a cyclic type that multiplies the difference image from the image of the previous frame by a cyclic coefficient and subtracts it. FIG. 9 is a block diagram showing a basic configuration of cyclic three-dimensional noise reduction. As shown in this figure, the cyclic three-dimensional noise reduction 300 includes a frame buffer 301, and subtracts a difference between the image of the previous frame stored in the frame buffer 301 and the image of the input frame. Thus, a noise component newly generated in the input frame image is extracted. The extracted noise component is multiplied by the cyclic coefficient K by the multiplier 303, and further subtracted from the input image by the subtractor 304, thereby removing noise from the image of the input frame.

  Any of the three-dimensional noise reduction methods can obtain an effect on the random noise generated in the still areas of the preceding and following frames, but an afterimage of the previous frame image is generated in the area with motion.

  For this reason, a motion detection function is provided, and a cyclic coefficient is changed between a moving part and a stationary part in a frame. For example, in Patent Document 2, three-dimensional noise reduction is applied to an area where there is no motion in a frame, and the effect of three-dimensional noise reduction is set to be weak in an area where there is motion, and is used together with two-dimensional noise reduction. It is described. Japanese Patent Application Laid-Open No. 2004-259542 discloses a technique for improving the accuracy of motion detection and determining a cyclic coefficient by removing a high frequency noise component by applying a low pass filter to a difference image between an input frame and a noise reduction output frame. Is described. Further, in recent years, as a motion compensation type three-dimensional noise reduction, a method of calculating a motion vector of each pixel and performing the three-dimensional noise reduction after correction is proposed.

JP 2008-123472 A Japanese Patent Publication No. 7-110049 JP-A-5-328174

  By the way, in the network monitoring system, in order to reduce the load on the network and increase the video recording efficiency in the network recorder, the video compression rate is increased or the video frame rate is reduced and recorded in the network recorder. Is frequently done. At this time, the frame rate is also changed for each video importance and each monitoring target. When the video frame rate is lowered, when the video is displayed on the display device, the frame rate is adjusted to the frequency of the video device.

  For example, assume that the frame rate at the time of shooting is reduced to one third during network transmission. In this case, 10 frame images 1f to 10f obtained at the time of photographing as shown in FIG. 10A are four frames 1f, 4f, 7f, and 10f as shown in FIG. 10B. Will be transmitted to the network. That is, the nine frame images 1f to 9f are one third of the three frames 1f, 4f, and 7f. Note that small black dots in the frame image indicate random noise generated on the network camera side.

  When a video with a reduced frame rate is displayed on the display device at the same frequency as the frame rate at the time of shooting, as shown in FIG. 10C, three frames of the same image are continuously displayed. .

  For this reason, when noise reduction processing for random noise generated on the network camera side is not performed, the same noise is continuously displayed while the frames of the same image are continuous. Therefore, it is conceivable to apply the cyclic three-dimensional noise reduction shown in FIG.

  In the cyclic type three-dimensional noise reduction, when the cyclic coefficient K of the multiplier 303 is 1, newly generated noise can be removed. However, it is greatly affected by the noise of the previous frame image stored in the frame buffer 301. Therefore, by setting the cyclic coefficient in a range of 0 <K <1, improvement of S / N against random noise is achieved.

  When the conventional cyclic three-dimensional noise reduction is applied when the dropped frame rate is displayed again, as shown in FIG. 10C, the display device 1f → 4f, 4f → 7f, 7f → 10f Thus, when the content of the frame image changes, a sufficient S / N improvement can be obtained. However, for example, when 4f continues, there is no difference between the preceding and succeeding frame images, and since it is not regarded as random noise, the effect of noise reduction cannot be sufficiently obtained.

  Accordingly, an object of the present invention is to provide a video processing apparatus capable of effectively reducing random noise even when frames of the same image are continuous.

  In order to solve the above problems, a video processing apparatus according to the present invention includes a first frame memory that stores an image of a frame immediately before an input frame, an image of the input frame, and an image stored in the first frame memory. And a discriminating means for discriminating whether or not both images are the same, and cyclic processing for performing cyclic three-dimensional noise removal processing using a cyclic coefficient on the image stored in the first frame memory A three-dimensional noise reduction unit, wherein the cyclic coefficient is set to 1 when the comparison / determination unit determines that the two images are the same.

  Here, when the comparison determination unit determines that the two images are not the same, the cyclic coefficient can be set to a value larger than 0 and smaller than 1.

  Further, the cyclic three-dimensional noise reduction means includes a second frame memory for storing an image for one frame, and a difference between an image stored in the first frame memory and an image stored in the second frame memory. Calculating a difference between an image stored in the first frame memory and an output of the multiplier; a first subtractor that calculates the output; a multiplier that multiplies the output of the first subtractor by the cyclic coefficient; A second subtractor that outputs the image after noise removal and outputs the image to the second frame memory may be provided.

  ADVANTAGE OF THE INVENTION According to this invention, even when the frame of the same image continues, the video processing apparatus which can reduce random noise effectively is provided.

It is a block diagram which shows the structure of the network video monitoring system which concerns on this embodiment. It is a block diagram which shows the structure of a video processing apparatus. It is a block diagram which shows the structure of a noise reduction part. It is a figure which shows the example of a histogram. It is a figure explaining the specific example of the output signal of a block. It is a figure which shows the effect of this embodiment concretely. It is a figure which shows the another effect of this embodiment concretely. It is a block diagram which shows another structure of a noise reduction part. It is a block diagram which shows the structure of the conventional circulation type three-dimensional noise reduction. It is a figure which compares the frame at the time of imaging | photography, the frame at the time of transmission, and the frame at the time of a display.

  Embodiments of the present invention will be described in detail with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a network video monitoring system according to the present embodiment. As shown in the figure, the network video surveillance system includes a plurality of surveillance cameras 20 (20a, 20b, 20c, 20d), a hub device 40, a network decoder device 50, a video processing device 100, and a display device 200. It is configured with.

  The monitoring camera 20 takes a picture at a predetermined frame rate, and reduces the load on the communication path by reducing the frame rate when output to the outside. The hub device 40 receives video data from the plurality of monitoring cameras 20 and outputs the video data to the network decoder device 50. The network decoder device 50 decodes the received video data and outputs it to the video processing device 100. At this time, the network decoder device 50 increases the frame rate of the received video data in order to match the display frequency of the display device 200 and outputs it. For this reason, frames of the same image including a noise component are continuous. Here, it is assumed that the display frequency of the display device 200 is the same as the frame rate at the time of shooting by the monitoring camera 20. The video processing device 100 receives the video data output from the network decoder device 50, performs noise reduction processing, and outputs it to the display device 200.

  FIG. 2 is a block diagram illustrating a configuration of the video processing apparatus 100. As shown in this figure, the video processing device 100 receives video data from the network decoder device 50, and outputs a video receiving unit 110 that outputs each frame, and a noise component of a frame image output from the video receiving unit 110. The noise reduction part 120 and the video output part 130 which outputs the frame image in which the noise component was reduced to the display apparatus 200 as video data are provided. The video processing apparatus 100 can be configured using an information processing apparatus including a CPU, a memory, an interface, and the like.

  In the following, a case where video data is input from one monitoring camera 20 will be described as an example for the sake of simplicity, but video data may be input from a plurality of monitoring cameras 20 as shown in FIG. It shall be possible. The format of the video data is not limited, but can be RGB, YCbCr, or the like, for example.

  FIG. 3 is a block diagram illustrating a configuration of the noise reduction unit 120. As shown in the figure, the noise reduction unit 120 includes a first frame buffer 121, a second frame buffer 122, and a comparison determination unit 123. The comparison discriminating unit 123 includes a subtractor 124 and a discriminator 125, and the subtractor 124 calculates a difference between the image S 1 of the input frame and the image of the previous frame stored in the first frame buffer 121. Then, the discriminator 125 discriminates whether or not the images of both frames are the same.

  For example, the comparison determination unit 123 determines whether or not the input frame image and the image of the previous frame are the same including the noise component by using a histogram of the difference image as shown in FIG. Can do. Here, FIG. 4A shows a histogram of the difference image in the case of the same image, and FIG. 4B shows a histogram of the difference image in the case of different images. In this example, the pixel level of 256 gradations is divided into 16 sections, and the frequency of the pixels included in each section is graphed. If the same image as shown in FIG. 4A, the pixel level of the difference image is distributed to 0, and if it is a different image as shown in FIG. 4B, the pixel level of the difference image is distributed over a wide range. It will be. Therefore, the comparison / determination unit 123 can determine whether or not the images are the same in the preceding and succeeding frames according to the pixel level distribution state of the difference image.

  The second frame buffer 122 stores an image based on a frame immediately before the previous frame stored in the first frame buffer 121. Then, a noise component is extracted by calculating a difference between the image stored in the first frame buffer 121 and the image stored in the second frame buffer 122 by the subtractor 126. The extracted noise component is stored in the first frame buffer 121 by being multiplied by the cyclic coefficient K by the multiplier 127 and further subtracted from the image stored in the first frame buffer 121 by the subtractor 128. Noise removal is performed on the image, which is output as an output frame image Sout. That is, the second frame buffer 122, the subtractor 126, the multiplier 127, and the subtractor 128 constitute a cyclic three-dimensional noise reduction unit 129 similar to the conventional one.

  Here, when the discriminator 125 discriminates that the images are not identical in the preceding and succeeding frames, the cyclic coefficient K multiplied by the multiplier 127 at the time of the next frame processing is set to k1 satisfying 0 <k1 <1. Thereby, it becomes possible to improve the S / N against random noise. The cyclic coefficient K may be a fixed value or a variable value. The larger the cyclic coefficient K, the more newly generated noise can be reduced. Therefore, the cyclic coefficient K may be increased when it can be determined that there are many noise components with reference to the histogram shown in FIG.

  On the other hand, when the discriminator 125 discriminates that the images are the same in the preceding and following frames, the cyclic coefficient K multiplied by the multiplier 127 during the next frame processing is set to “1”. Thus, the second frame buffer 122 can maintain the same state as the previous frame. The output frame image Sout is also the same image as the previous output frame image Sout.

  As described above, the configuration shown in FIG. 3 changes the cyclic coefficient to be applied depending on whether or not the image of the latest frame is the same as the image of the previous frame. Specifically, if they are the same, the cyclic coefficient is 1, and if they are not the same, the cyclic coefficient is k1 that satisfies 0 <k1 <1. Thus, 3DRN can be effectively applied only when there is a change in the frame image, and the effect of three-dimensional noise reduction can be maintained when there is no change.

  Next, a specific example of the output signal of each block will be described with reference to FIG. Here, the image signal of the input frame is S1, and the output signal of the first frame buffer 121 is S2. Assume that the output signal of the discriminator 125 is S3, “X” indicates that the images of the previous and subsequent frames are not the same, and “0” indicates that the images of the previous and subsequent frames are the same. The output signal of the second frame buffer 122 is S4, the output signal of the subtractor 126 is S5, and the output signal of the multiplier 127 is S6. K is a cyclic coefficient in the multiplier 127.

  In addition, the input frame is assumed to be three frames of the same image, and is a still image including a noise component for the sake of simplicity. In the following, “A” represents a still image component, and “a”, “b”, and “c” represent noise components generated by the monitoring camera 20. Therefore, as the input frame signal S1, “A + a” is input from 1f to 3f, “A + b” is input from 4f to 6f, and “A + c” is input from 7f to 9f.

  First, the cyclic coefficient K is set to 0 for 2 frame times from the start of processing until the 1f frame is input to the second frame buffer 122. Since 3 frames of the same image continue, when 4f and 7f are input, the image is different from the image of the previous frame, and the output of S3 becomes “X”. As a result, the cyclic coefficient K applied when 5f and 8f are input is k1, and the cyclic coefficient K is 1 at other time points.

  For this reason, Sout becomes “A + a”, which is the same as the previous frame, in 3f and 4f where the cyclic coefficient is 1, and becomes A + b−N1 in 5f where the cyclic coefficient is k1. Here, N1 represents k1 × (b−a). After that, 6f and 7f with a cyclic coefficient of 1 are “A + b−N1”, which is the same as the previous frame, and A + c−N2 at 8f with a cyclic coefficient of k1. Here, N2 represents k1 × (c− (b−N1)). S4 is an output delayed by one frame from Sout, S5 can be obtained by S2-S4, and S6 can be obtained by S5 × K.

  The effect of this embodiment will be described using a more specific example. FIG. 6A is a diagram in which a part of FIG. 5 is extracted, and is a diagram when k1 of the cyclic coefficient K is halved. FIG. 6B shows an example of a conventional system in which the cyclic coefficient K is fixed to 1/2 for comparison.

  In this figure, at 5f, the noise component (ba) / 2 to be subtracted is obtained in both the method of the present embodiment and the conventional method, and in S4 of 6f where the subtraction result is accumulated, the noise components “a” and “b” are It can be seen that each is 1/2.

  Next, in 6f where the same frame image is input, according to the conventional method shown in FIG. 6B, the noise component “a” is reduced to ¼ as shown in S4 of 7f. However, the noise component “b” is 3/4, which is larger than the previous frame image. That is, if “A + b” is continuously input, the noise component “b” further increases and the noise reduction effect is impaired.

  On the other hand, as shown in FIG. 6A, when the method of this embodiment is used, the noise component “a” and the noise component “b” are equally held in S4 of 7f, so that noise reduction is achieved. The effect can be retained.

Further, in the present embodiment, by setting the cyclic coefficient to 1 when the images of the previous and subsequent frames are the same, there is also an effect of removing random noise generated on the transmission path. A specific example is shown in FIG. As shown in this figure, it is assumed that noise z is added on the transmission line at 6f. In this case, z is added to (b−a) / 2 in S5, but when the cyclic coefficient is 1, by subtracting from S2,
(A + b + z) − {z + (b−a) / 2} = A + (b + a) / 2
Thus, the noise z can be completely removed.

  As described above, according to the present embodiment, even when frames of the same image are continuous, the effect of three-dimensional noise reduction can be kept optimal, so that random noise can be effectively reduced. It becomes like this.

  FIG. 8 is a block diagram illustrating another configuration example of the noise reduction unit. The same blocks as those in FIG. 3 are denoted by the same reference numerals. In another configuration example shown in this figure, the noise reduction unit 120a extracts only a changing frame image based on the output signal from the discriminator 125 of the comparison discriminating unit 123 to extract a three-dimensional noise reduction (3DNR) unit. 141, three-dimensional noise reduction is performed, and the frame rate reduced in the output is converted again to the same frame rate as the display frequency of the display device 200. Thereby, the same effect as the noise reduction part 120 shown in FIG. 3 can be acquired. However, in this case, the first output frame buffer 143 and the second output frame buffer 144 are added, and the circuit scale increases. As described above, the configuration shown in FIG. 3 can effectively reduce random noise while minimizing the circuit scale.

  In the present embodiment, the video from the monitoring camera 20 has been described as a still image. When the present invention is applied to a moving image, for example, a known motion detection function is added to the comparison discriminating unit 123 to discriminate between a static region and a motion region in a frame, and the respective cyclic coefficients are set. Try to change. Thus, the present invention can be effectively applied to moving images.

DESCRIPTION OF SYMBOLS 20 ... Surveillance camera 40 ... Hub apparatus 50 ... Network decoder apparatus 100 ... Video processing apparatus 110 ... Video receiving part 112 ... Frame buffer 120 ... Noise reduction part 121 ... 1st frame buffer 122 ... 2nd frame buffer 123 ... Comparison discrimination | determination part 124 ... subtractor 125 ... discriminator 126 ... subtractor 127 ... multiplier 128 ... subtractor 129 ... cyclic 3D noise reduction unit 130 ... video output unit 141 ... 3D noise reduction unit 143 ... first output frame buffer 144 ... first 2-output frame buffer 200... Display device

Claims (3)

A first frame memory for storing an image of the previous frame of the input frame;
A comparison discriminating means for comparing the image of the input frame with the image stored in the first frame memory and discriminating whether or not both images are the same;
Cyclic 3D noise reduction means for performing cyclic 3D noise removal processing using a cyclic coefficient on the image stored in the first frame memory;
The video processing apparatus according to claim 1, wherein the cyclic coefficient is set to 1 when the comparison determination unit determines that the two images are the same. The video processing apparatus according to claim 1,
The video processing apparatus according to claim 1, wherein when the comparison determination unit determines that the two images are not the same, the cyclic coefficient is set to a value greater than 0 and less than 1. The video processing apparatus according to claim 1 or 2,
The cyclic three-dimensional noise reduction means includes:
A second frame memory for storing an image for one frame;
A first subtractor for calculating a difference between an image stored in the first frame memory and an image stored in the second frame memory;
A multiplier for multiplying the output of the first subtractor by the cyclic coefficient;
A difference between the image stored in the first frame memory and the output of the multiplier is calculated and output as a noise-removed image; and a second subtractor that outputs to the second frame memory. A video processing apparatus characterized by the above.