WO2010106739A1

WO2010106739A1 - Image processing device, image processing method, and image processing program

Info

Publication number: WO2010106739A1
Application number: PCT/JP2010/001122
Authority: WO
Inventors: 隅谷一徳; 芹沢正之
Original assignee: パナソニック株式会社
Priority date: 2009-03-16
Filing date: 2010-02-22
Publication date: 2010-09-23
Also published as: JP2010219807A

Abstract

An image processing device (1) provided with: a frame memory (2) which stores a recursive image signal from 1 frame before; a synchronization unit (3) which synchronizes the input image signal with the recursive image signal; a motion detection unit (4) which detects motion in the image from the input image signal and recursive image signal; a two-dimensional filter (6) which reduces the noise component in the input image signal according to the correlation of the pixels of interest in the image to the pixels surrounding the same; a three-dimensional filter (7) which uses the recursive coefficient, which is based on the motion in the image, to add the input image signal to the recursive image signal in a weighted manner; a signal synthesis unit (8) which uses a weighting coefficient, which is based on the motion in the image, to add the output image signals from the two-dimensional filter (6) and three-dimensional filter (7) in a weighted manner; and further provided with a noise reduction unit (5) which reduces the noise in the recursive image signal to be inputted to the three-dimensional filter (7). Due to this, the image processing device (1) can shorten the time taken for noise to be sufficiently reduced in an object which has changed from a motion state to a still state.

Description

Image processing apparatus, image processing method, and image processing program

The present invention relates to a digital image processing apparatus, and more particularly to an image processing apparatus capable of obtaining a high-quality image by reducing noise of an image signal and improving S / N.

Generally, an image signal is a signal in which image information is repeated at a frame period, and the autocorrelation between frames is very strong. On the other hand, since noise components contained in image signals generally have no autocorrelation, it is known that when image signals are temporally averaged for each frame period, only noise component energy is reduced, and noise can be reduced. It has been. Conventionally, a noise reduction device that performs noise reduction based on motion detection information is known as a frame cyclic noise reduction device (see, for example, Patent Document 1).

In this conventional noise reduction apparatus, still image / moving image determination is performed based on the motion detection information. When it is determined as a still image, noise reduction is mainly performed by a frame cyclic noise reduction device (three-dimensional filter). When it is determined as a moving image, an area selection type noise reduction device (2 Each noise reduction apparatus is controlled so that noise reduction is performed by a dimension filter.

However, in the conventional noise reduction device, it is possible to make the noise reduction effect of the three-dimensional filter much larger than the noise reduction effect of the two-dimensional filter by setting the cyclic coefficient large. If the circulation is not repeated to some extent, the effect of the three-dimensional filter cannot be fully exhibited.

For example, if it is determined that an object in the image is “moving”, the noise of the object part (moving image part) is reduced by the two-dimensional filter, and the part around the object (stationary) is reduced by the three-dimensional filter. The noise of the image portion is reduced. The S / N ratio of the noise-reduced part with the two-dimensional filter is not as good as that of the noise-reduced part with the three-dimensional filter, but there is “motion blur” for a moving subject. / N reduction is allowed. For example, when it is determined from the next frame that the object is “still”, the noise of the object portion is also reduced by the three-dimensional filter. However, in the conventional noise reduction device, immediately after the moving object stops, the effect of noise reduction of the three-dimensional filter for the object part (still image part) cannot be obtained, and the circulation is repeated to some extent. After that, the same noise reduction effect as that of the surrounding portion that was originally stationary can be obtained (the S / N becomes equivalent). In other words, for an object whose state has changed from “moving state” to “still state”, it takes time until the noise is sufficiently reduced and the S / N is improved. May stand out.

Therefore, it has been desired to improve the S / N as quickly as possible to make the noise inconspicuous for an object whose state has changed from "moving state" to "stationary state". In particular, in a camera having a PTZ (pan / tilt / zoom) function, it is determined that the entire screen is “moving” during the PTZ operation, but noise is reduced as soon as possible after the PTZ operation is completed. / N should be increased.

JP 2001-160909 A

The present invention has been made under the above background. An object of the present invention is to provide an image processing apparatus capable of shortening the time until noise is sufficiently reduced for an object whose state has changed from a moving state to a stationary state.

One aspect of the present invention is an image processing apparatus, which includes a frame memory unit that accumulates an image signal of one frame before as a cyclic image signal, an input image signal, and a cyclic image signal input from the frame memory unit. A synchronization unit that simultaneously synchronizes a plurality of lines, a motion detection unit that detects image motion based on the synchronized input image signal and cyclic image signal, and a correlation between a pixel of interest in the image and its surrounding pixels The input image signal and the cyclic image signal are weighted and added using a two-dimensional filter unit that reduces the noise component of the input image signal and a cyclic coefficient determined based on the detected motion of the image. An output image from the two-dimensional filter unit using a three-dimensional filter unit that reduces noise components of the input image signal and a weighting coefficient determined based on the detected motion of the image A signal synthesizer that weights and adds the signal and the output image signal from the three-dimensional filter unit, and a noise removal unit that performs a noise removal process on the cyclic image signal input to the three-dimensional filter unit, and the motion of the image is small In this case, the cyclic coefficient is determined to be smaller than the cyclic coefficient when the image motion is large.

Another aspect of the present invention is an image processing method, in which an image signal of one frame before is stored as a cyclic image signal, and an input image signal and the accumulated cyclic image signal are simultaneously stored for a plurality of lines. To perform synchronization processing, to perform motion detection processing to detect image motion based on the synchronized input image signal and cyclic image signal, and to correlate the target pixel in the image and its surrounding pixels Based on the two-dimensional filter processing for reducing the noise component of the input image signal, and the cyclic coefficient determined based on the detected image motion, where the image motion is small. A three-dimensional filter that reduces the noise component of the input image signal by weighted addition of the input image signal and the cyclic image signal using a cyclic coefficient determined to a smaller value when the movement of the image is large Signal synthesis processing for performing weighted addition of the output image signal after the two-dimensional filter processing and the output image signal after the three-dimensional filter processing using a weighting coefficient determined based on the motion of the detected image And performing noise removal processing on the cyclic image signal before the three-dimensional filter processing.

Another aspect of the present invention is an image processing program. This program stores the image signal of the previous frame as a cyclic image signal and the input image signal and the accumulated cyclic image signal for a plurality of lines simultaneously. Based on the correlation between the target pixel in the image and its surrounding pixels, the processing for performing the synchronization processing, the motion detection processing for detecting the motion of the image based on the synchronized input image signal and the cyclic image signal Two-dimensional filter processing for reducing the noise component of the input image signal and a cyclic coefficient determined based on the detected motion of the image, and when the motion of the image is smaller than when the motion of the image is greater Three-dimensional filter processing for reducing noise components of the input image signal by weighted addition of the input image signal and the cyclic image signal using a cyclic coefficient determined to be a small value; Signal synthesis processing for weighted addition of the output image signal after the two-dimensional filter processing and the output image signal after the three-dimensional filter processing using a weighting coefficient determined based on the detected motion of the image, and the three-dimensional filter processing The computer executes a noise removal process on the previous cyclic image signal.

As described below, there are other aspects of the present invention. Accordingly, this disclosure is intended to provide some aspects of the invention and is not intended to limit the scope of the invention described and claimed herein.

FIG. 1 is a block diagram illustrating a configuration of an image processing apparatus according to the first embodiment. FIG. 2 is a diagram illustrating an example of a noise reduction effect by a two-dimensional filter. FIG. 3 is a diagram illustrating an example of a noise reduction effect by a three-dimensional filter. FIG. 4 is a diagram showing an example of noise removal processing by LPF as a comparative example. FIG. 5 is a diagram illustrating an example of noise removal processing according to the second embodiment.

The detailed description of the present invention will be described below. However, the following detailed description and the accompanying drawings do not limit the invention. Instead, the scope of the invention is defined by the appended claims.

The image processing apparatus according to the present invention includes a frame memory unit that stores an image signal of one frame before as a cyclic image signal, and a synchronization that simultaneously synchronizes an input image signal and a cyclic image signal input from the frame memory unit for a plurality of lines. A motion detection unit that detects the motion of the image based on the synchronized input image signal and the cyclic image signal, and a noise component of the input image signal based on a correlation between the target pixel in the image and its surrounding pixels The noise component of the input image signal is reduced by weighted addition of the input image signal and the cyclic image signal using a two-dimensional filter unit for reducing the image and a cyclic coefficient determined based on the detected motion of the image. The output image signal from the two-dimensional filter unit and the output from the three-dimensional filter unit using the three-dimensional filter unit and a weighting coefficient determined based on the detected motion of the image A signal synthesis unit that performs weighted addition of the force image signal, and a noise removal unit that performs noise removal processing on the cyclic image signal input to the three-dimensional filter unit, and the cyclic coefficient when the motion of the image is small is It has a configuration that is determined to be smaller than the cyclic coefficient when the motion is large.

With this configuration, noise removal processing is performed on the cyclic image signal input to the three-dimensional filter unit. Therefore, it is possible to reduce the time until the noise is sufficiently reduced for an object whose state has changed from a moving state to a stationary state. The image can be shortened and a high S / N image can be obtained quickly.

In the image processing apparatus of the present invention, the noise removal unit may have a configuration that is a median filter, a weighted median filter, or a stack filter.

With this configuration, even when the noise removal process is repeatedly performed on the cyclic image signal input to the three-dimensional filter unit, the necessary image signal is not lost, and an appropriate noise reduction process is performed. A high-quality image can be obtained.

In the image processing apparatus of the present invention, the noise removing unit compares the signal levels of the target pixel and the surrounding pixels, and the signal level of the target pixel is determined based on the maximum signal level of the surrounding pixels. The pixel value of the pixel of interest may be replaced when it is larger than the upper limit value of the pixel or when it is smaller than a predetermined lower limit value determined based on the minimum signal level of the surrounding pixels.

The image processing method of the present invention accumulates the image signal of the previous frame as a cyclic image signal, performs a synchronization process for synchronizing the input image signal and the accumulated cyclic image signal for each of a plurality of lines, Based on the synchronized input image signal and the cyclic image signal, performing a motion detection process for detecting the motion of the image, and based on the correlation between the target pixel in the image and its surrounding pixels, the noise component of the input image signal Noise component of the input image signal by performing a two-dimensional filter process for reducing the input image signal and weighted addition of the input image signal and the cyclic image signal using a weighting coefficient determined based on the detected motion of the image And a cyclic coefficient determined on the basis of the detected motion of the image, where the motion of the image is smaller. Using a cyclic coefficient determined to be a smaller value when larger, performing signal synthesis processing for weighted addition of the output image signal after two-dimensional filter processing and the output image signal after three-dimensional filter processing, and three-dimensional filter processing Performing noise removal processing on the previous cyclic image signal.

Also with this method, as described above, noise removal processing is performed on the cyclic image signal used for the three-dimensional filter processing, so that noise is sufficiently reduced for an object whose state has changed from a moving state to a stationary state. It is possible to shorten the time required to obtain a high S / N image.

The image processing program of the present invention includes a process for accumulating the image signal of the previous frame as a cyclic image signal, a process for performing a synchronization process for simultaneously synchronizing the input image signal and the accumulated cyclic image signal for a plurality of lines, Based on the synchronized input image signal and the cyclic image signal, the motion detection process for detecting the motion of the image, and the noise component of the input image signal is reduced based on the correlation between the target pixel in the image and its surrounding pixels. Two-dimensional filter processing and a cyclic coefficient determined based on the detected image motion, which is determined to be smaller when the image motion is small and smaller than when the image motion is large Thus, the input image signal and the cyclic image signal are weighted and added to reduce the noise component of the input image signal and based on the detected motion of the image. Using a weighted coefficient, a signal synthesis process for weighted addition of the output image signal after the two-dimensional filter process and the output image signal after the three-dimensional filter process, and noise removal for the cyclic image signal before the three-dimensional filter process Processing is executed by a computer.

Also in this program, noise removal processing is performed on the cyclic image signal used for the three-dimensional filter processing as described above, so that noise is sufficiently reduced for an object whose state has changed from a moving state to a stationary state. It is possible to shorten the time until the image is received and to obtain a high S / N image quickly.

The present invention provides a noise removal unit that performs noise removal processing on a cyclic image signal input to a three-dimensional filter unit, until noise is sufficiently reduced for an object whose state has changed from a moving state to a stationary state. Can be shortened.

Hereinafter, an image processing apparatus according to an embodiment of the present invention will be described with reference to the drawings. In the present embodiment, an example of an image processing apparatus used for image processing of an image taken by a camera having a PTZ function is illustrated. This image processing function may be realized by a program stored in an HDD or a memory of the image processing apparatus.

(First embodiment)
The configuration of the image processing apparatus according to the first embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing the configuration of the image processing apparatus according to the present embodiment. As shown in FIG. 1, an image processing apparatus 1 includes a frame memory 2 that accumulates image signals as cyclic image signals, a synchronization unit 3 that performs a process of synchronizing an input image signal and a cyclic image signal, A motion detection unit 4 that performs processing for detecting motion and a noise removal unit 5 that performs processing for removing noise components of the cyclic image signal are provided. The image processing apparatus 1 also includes a two-dimensional filter 6 that removes noise components of the input image signal based on the correlation of the pixels in the image, and weighted addition of the input image signal and the cyclic image signal to thereby calculate the input image signal. A three-dimensional filter 7 that removes noise components, and a signal synthesis unit 8 that performs weighted addition of output image signals from the two-dimensional filter 6 and the three-dimensional filter 7 are provided.

The frame memory 2 receives the image signal of the previous frame output from the signal synthesizer 8. The image signal of the previous frame is temporarily stored as a cyclic image signal in the frame memory 2 and read out in synchronization with the input signal of the next frame.

The synchronization unit 3 is composed of a line memory and the like. An input image signal and a cyclic image signal are input to the synchronizer 3, and the input image signal and the cyclic image signal for a plurality of lines are respectively synchronized. As a result, the image signal of the target pixel and its surrounding pixels can be processed two-dimensionally (planarly).

The motion detection unit 4 receives the input image signal and the cyclic image signal output from the synchronization unit 3, and detects the motion of the image based on the synchronized input image signal and the cyclic image signal. . For example, the amount of motion for each pixel is calculated based on the difference between the input image signal and the cyclic image signal.

The two-dimensional filter 6 has a function of reducing the noise component of the input image signal based on a two-dimensional correlation between the target pixel in the image and its surrounding pixels. That is, it can be said that the two-dimensional filter 6 functions as an area-selective noise reduction device that performs two-dimensional noise reduction processing on the input image signal to improve S / N.

The three-dimensional filter 7 performs a three-dimensional (planar and temporal) noise reduction process on the input image signal by weighted addition of the input image signal and the cyclic image signal, and reduces the noise component of the input image signal. It has a function to reduce. In other words, the three-dimensional filter 7 functions as a frame cyclic noise reduction device that improves the S / N according to the height of the autocorrelation of the video by weighted addition of the cyclic image signal at the same position as the input image signal. I can say that.

The signal synthesizer 8 receives the image signal output from the two-dimensional filter 6 and the image signal output from the three-dimensional filter 7. The signal synthesizer 8 performs a process of performing signal synthesis by weighted addition of these two image signals.

The cyclic coefficient used for the weighted addition in the three-dimensional filter 7 and the weighting coefficient (also referred to as a weighting coefficient) used for the weighted addition in the signal synthesis unit 8 are determined based on the motion amount calculated by the motion detection unit 4. Is done. The motion detection unit 4 can calculate a motion amount for each pixel. For example, when the motion amount calculated by the motion detection unit 4 is large (in the case of a moving image), the cyclic coefficient is set to be small, the input image signal is frequently used, and the motion amount calculated by the motion detection unit 4 is When it is small (in the case of a still image), the cyclic coefficient is set to be large and a cyclic image signal is often used. Also, for example, when the amount of motion calculated by the motion detection unit 4 is large (in the case of a moving image), the weighting coefficient is set to be small, and the two-dimensional filter output signal is frequently used and is calculated by the motion detection unit 4. When the amount of motion is small (in the case of a still image), the weighting coefficient is set large and a three-dimensional filter signal is often used.

The noise removal unit 5 performs noise removal processing using a filter on the cyclic image signal input to the three-dimensional filter 7 (cyclic image signal one frame before output from the frame memory 2). In the noise removal unit 5 of the present embodiment, a median filter is used as a filter for this noise removal processing.

The median filter is a filter that outputs a median value of a plurality of input image signals, and has a function of removing noise components from the image signals. Note that in order to obtain an input signal to the median filter, signals from pixels around the target pixel are required. In order to obtain the signals of the surrounding pixels, it is usually necessary to perform a synchronization process for a plurality of lines using a line memory or the like (a separate synchronization unit 3 must be provided). However, in the present embodiment, since the synchronization unit 3 is already provided for the motion detection unit 4, it is not necessary to separately provide the synchronization unit 3 by using this, thereby increasing the circuit scale. Can be suppressed.

The operation of the image processing apparatus 1 configured as described above will be described with reference to the drawings.

When one point on the image is a moving pixel, noise reduction processing by the two-dimensional filter 6 is performed. FIG. 2 is a diagram illustrating an example of noise reduction processing by the two-dimensional filter 6. In FIG. 2, the time change of the signal level of one point on the image is shown (the vertical axis is the signal level and the horizontal axis is the time). In FIG. 2, the original signal (image signal before noise reduction processing) is indicated by a symbol “rhombus”, and a signal obtained by applying the two-dimensional filter 6 to the original signal is indicated by a symbol “square”. ing. In this case, the true zero signal level without noise is “0”, and noise is added around “0”. 2 that the noise component of the image signal is removed to some extent by the noise reduction processing by the two-dimensional filter 6.

When this pixel becomes a still pixel (from a moving pixel), noise reduction processing by the three-dimensional filter 7 is performed. FIG. 3 is a diagram illustrating an example of noise reduction processing by the three-dimensional filter 7. In FIG. 3, the time change of the signal level of this pixel is shown (the vertical axis is the signal level and the horizontal axis is the time). In FIG. 3, a signal obtained by applying the three-dimensional filter 7 after performing the noise removal processing using the median filter is indicated by a symbol “square”, and the three-dimensional filter 7 is applied without performing the noise removal processing. The signal (that is, the comparative example) is indicated by the symbol “diamond”. From the results shown in FIG. 3, when the original signal is subjected to the three-dimensional filter 7 without performing noise removal processing as in the prior art, it takes time to converge to “0” (especially when the number of cyclic frames increases). On the other hand, when the three-dimensional filter 7 is applied to the original signal after performing the noise removal processing as in the present embodiment, the time until convergence to “0” is shortened even when the moving state changes to the stationary state. It can be seen that it has been (converged quickly).

In this case, the noise removal unit 5 also performs noise removal processing on the cyclic image signal for a still image. In the stationary state, the cyclic coefficient of the cyclic image signal is set large, so that the cyclic image signal passes through the noise removal unit 5 many times and is subjected to noise removal processing many times. Therefore, an S / N improvement effect can be obtained even in a still image.

According to the image processing apparatus 1 of the first embodiment as described above, by providing the noise removal unit 5 that performs noise removal processing on the cyclic image signal input to the three-dimensional filter 7, the moving state is changed to the stationary state. The time until the noise is sufficiently reduced for the object whose state has changed can be shortened.

In other words, in the present embodiment, noise removal processing is performed on the cyclic image signal input to the three-dimensional filter 7, so that noise is sufficiently reduced for an object whose state has changed from a moving state to a stationary state. Time (see FIG. 3), and a high S / N image can be obtained quickly.

Here, the operational effects of the image processing apparatus 1 of the present invention will be described in more detail in comparison with a conventional apparatus.

In the conventional apparatus, the image quality of the portion from the “motion state” to the “still state” due to the difference in performance between the two-dimensional filter on the moving portion of the image and the three-dimensional filter on the stationary portion is There was a problem that the image quality of the still part deteriorated.

Even in the conventional apparatus, the S / N is improved to some extent by the two-dimensional filter in the “first first frame” where the subject has changed from the “moving state” to the “still state” (the S / N may be improved only to some extent). I can say). Since the motion detection result is “with motion” at the time of the “first first frame”, the cyclic coefficient is set to a low value, and the weighted addition coefficient is weighted so as to be almost a two-dimensional filter output. The output signal is approximately the current input signal with a two-dimensional filter applied.

However, in the “next frame”, since the motion detection result is “no motion”, the cyclic coefficient is set high, the weighted addition coefficient is weighted so as to be almost a three-dimensional filter output, and the output signal is A signal close to the cyclic image signal is output. That is, an image having a poor S / N (an image in which the S / N is improved only to some extent) in the “first frame” after the “motion state” is changed to the “still state” (the “next frame”) ", Since the cyclic coefficient is set high and the weighted addition coefficient is almost a three-dimensional filter output), it remains for a long time. As a result, the" motion state "changes to the" stationary state ". The image quality of the affected area is deteriorated compared to the image quality of the surrounding still area.

As a method for solving such a problem, conventionally, (1) lowering the cyclic coefficient setting value when stationary and (2) improving the performance of the two-dimensional filter have been proposed. However, in the method (1), although the time until the image quality of the portion from the “motion state” to the “stationary state” becomes equal to the surroundings is improved, the image quality at the time of stillness is lowered. In the method (2), there is a limit to simply improving the performance of the filter in two dimensions (particularly, when the gain amount is large, the effect of the two-dimensional filter is limited). A complicated filter circuit with high performance has a problem of increasing the circuit scale. Therefore, conventionally, the method (1) is often used, and in that case, the image quality of the portion where the “motion state” is changed to the “stationary state” is “time until it becomes equal to the surroundings” and “ It was inevitable that the “image quality at rest” would be a trade-off.

Also in the image processing apparatus 1 of the present invention, the S / N of the “first first frame” image that has changed from the “motion state” to the “still state” is improved to some extent by the two-dimensional filter as in the conventional case. At the time of this “first first frame”, the motion detection result is “with motion”, the cyclic coefficient is set to a low value, and the weighted addition coefficient is weighted so as to be approximately a two-dimensional filter output, and the output signal Is approximately the current input signal with a two-dimensional filter applied. This output signal is input to the frame memory 2 and becomes a cyclic image signal of the next frame.

In the image processing apparatus 1 of the present invention, in the “next frame”, the cyclic image signal passes through the noise removal unit 5 and noise removal processing is performed (S / N is improved). At the time of this “next frame”, the motion detection result is “no motion”, the cyclic coefficient is set high, and the weighted addition coefficient is weighted so as to be an approximately three-dimensional filter output, and the output signal Outputs a signal close to the cyclic image signal. This output signal is input to the frame memory 2 and becomes a cyclic image signal of the next frame.

In the image processing apparatus 1 of the present invention, such processing is repeatedly performed. That is, in the still state image, the cyclic coefficient is set high, and the weighted addition coefficient is weighted so as to be almost a three-dimensional filter output, and the signal passes through the noise removing unit 5 many times, and the S / N is reduced. There is an effect that it is improved. Such an S / N improvement effect is significantly different from that of simply strengthening the two-dimensional filter.

The basic performance of the image processing apparatus 1 is that an S / N image as high as possible is required even in a normal still state image (not only immediately after changing from a motion state to a still state). As described above, since the noise removal unit 5 also performs noise removal processing on the cyclic image signal for a still image, the S / N is also improved for a still image.

Further, in the present embodiment, since the noise removing unit 5 is a median filter, an FIR filter, an IIR filter, etc., even if the noise removal processing is repeatedly performed on the cyclic image signal input to the three-dimensional filter 7 means. As in the case of LPF, a necessary image signal is not lost, and a high-quality image with appropriate noise reduction processing can be obtained.

For example, LPFs such as FIR filters and IIR filters are common as filters, but these filters are not suitable for use in the noise removing unit 5 of the present embodiment. The reason will be described with reference to FIG.

As described above, when the image is stationary, the cyclic coefficient of the three-dimensional filter 7 increases, and the ratio of the cyclic signal from the frame memory 2 to the output signal increases. In FIG. 4, for the sake of simplicity, the first cyclic image signal when the cyclic coefficient is set to 100% is indicated by the symbol “rhombus”, and the second cyclic image signal is “square”. The third cyclic image signal is indicated by a “triangle” symbol. As shown in FIG. 4, it can be seen that the image signal spreads and the signal level decreases as the first, second, and third cycles are repeated. In other words, if the tour is repeated, the image signal will eventually disappear, or even if it is not so extreme, the signal spreads, and the stationary part is mistakenly detected as the moving part near the contour of the stationary object. There is a risk of failure.

Since the median filter is an order selection type filter, the signal does not spread and the above-mentioned problems do not occur. Further, since the median filter is an order selection type filter, even if the circulation is repeated with a cyclic coefficient of 100%, the output value converges when the process is repeated to some extent, so that the image signal is not lost.

In the above description, the example in which the noise removing unit 5 is configured with a median filter has been described. However, the scope of the present invention is not limited to this, and the noise removing unit 5 may be, for example, a weighted median filter or a stack filter. It may be constituted by.

The weighted median filter is an improvement of the median filter and improves the preservation of the fine line portion of the image. The weighted median filter is preferable from the viewpoint of improving the image quality if a certain increase in circuit scale can be allowed.

The stack filter is a filter that can obtain an output equivalent to the median filter by dividing the input signal into threshold values, performing Boolean algebra processing for each bit, and then performing inverse transformation of the threshold division. In this stack filter, the characteristics of a median filter that preserves a specific structure such as a weighted median filter or an edge can be configured. Therefore, the stack filter is preferable from the aspect of image quality improvement if an increase in circuit scale can be allowed to some extent. .

Even when the noise removal unit 5 is configured with such a weighted median filter or a stack filter, the noise removal process is repeatedly performed on the cyclic image signal input to the three-dimensional filter 7 means, as in the above embodiment. Even in such a case, a necessary image signal is not lost like an LPF such as an FIR filter or an IIR filter, and a high-quality image subjected to appropriate noise reduction processing can be obtained.

(Second Embodiment)
Next, an image processing apparatus 1 according to a second embodiment of the present invention will be described with reference to the drawings. Here, the present embodiment will be described with a focus on differences from the first embodiment. In other words, unless otherwise specified here, the configuration and operation of the present embodiment are the same as those of the first embodiment.

In the image processing apparatus 1 according to the present embodiment, the noise removing unit 5 compares the signal levels of the target pixel and the surrounding pixels, and the signal level of the target pixel is determined based on the maximum signal level of the surrounding pixels. Greater than a predetermined upper limit (maximum value of surrounding pixels + threshold) or smaller than a predetermined lower limit (minimum value of surrounding pixels−threshold) determined based on the minimum signal level of the surrounding pixels, It is characterized in that a process for replacing the pixel value of the target pixel is performed.

Here, the features of this embodiment will be described in detail with reference to FIG. FIG. 5 is a diagram illustrating an example of noise removal processing according to the present embodiment. FIG. 5 shows an example in which noise removal processing is performed by comparing the signal levels of a pixel of interest (i, j) and its surrounding pixels (3 × 3 pixels) in magnitude.

As shown in FIG. 5, in this case, first, values x (i−1, j−1) and x (i−1) of the surrounding 3 × 3 pixels excluding the target pixel value x (i, j). , J), x (i-1, j + 1), x (i, j-1), x (i-1, j + 1), x (i + 1, j-1), x (i + 1, j), x (i + 1) , J + 1) is determined.

Then, the upper limit value obtained by adding the threshold value to the maximum value, the lower limit value obtained by subtracting the threshold value from the minimum value, and the target pixel value x (i, j) are compared. When the target pixel value x (i, j) is larger than the upper limit value (= maximum value of surrounding pixels + threshold value), the target pixel value x (i, j) is replaced with the upper limit value. On the other hand, when the target pixel value x (i, j) is smaller than the lower limit value (= minimum value of surrounding pixels−threshold value), the target pixel value x (i, j) is replaced with the lower limit value.

The threshold value may be a fixed value or may be controlled according to the signal gain amount. Further, a value proportional to the average value of the surrounding pixel signals may be added to the threshold as a signal proportional threshold component.

Since the noise removal processing according to the present embodiment is also a signal selection type filter, the signal does not spread even if the processing is repeated by circulation. Similarly to the first embodiment, even if the cycle is repeated with a cyclic coefficient of 100%, the output value converges when processing is performed to some extent, so that the image signal is not lost.

In the above example, the description has been made using surrounding 3 × 3 pixels, but the scope of the present invention is not limited to this. That is, the surrounding pixels used for the noise removal process are not limited to 3 × 3 pixels.

Also with the image processing apparatus 1 according to the second embodiment, the same effects as those of the first embodiment can be obtained.

That is, even in the noise removing unit 5 of the present embodiment, even if the noise removal processing is repeatedly performed on the cyclic image signal input to the three-dimensional filter 7 means, like the LPF such as the FIR filter or the IIR filter. Necessary image signals are not lost, and a high-quality image with appropriate noise reduction processing can be obtained.

The embodiments of the present invention have been described above by way of example, but the scope of the present invention is not limited to these embodiments, and can be changed or modified according to the purpose within the scope of the claims. is there.

Although the presently preferred embodiments of the present invention have been described above, it will be understood that various modifications can be made to the present embodiments and are within the true spirit and scope of the present invention. It is intended that the appended claims include all such variations.

As described above, the image processing apparatus according to the present invention has an effect that the time until noise is sufficiently reduced for an object whose state has changed from a moving state to a stationary state can be shortened. It is useful for image processing of images taken with cameras equipped with functions.

DESCRIPTION OF SYMBOLS 1 Image processing apparatus 2 Frame memory 3 Synchronization part 4 Motion detection part 5 Noise removal part 6 Two-dimensional filter 7 Three-dimensional filter 8 Signal composition part

Claims

A frame memory unit for accumulating the image signal of the previous frame as a cyclic image signal;
A synchronization unit that simultaneously synchronizes the input image signal and the cyclic image signal input from the frame memory unit for a plurality of lines;
A motion detector that detects the motion of an image based on the input image signal and the cyclic image signal that are synchronized;
A two-dimensional filter unit that reduces a noise component of the input image signal based on a correlation between a target pixel in the image and surrounding pixels;
A three-dimensional filter unit that reduces a noise component of the input image signal by weighted addition of the input image signal and the cyclic image signal using a cyclic coefficient determined based on the detected motion of the image; ,
Using a weighting coefficient determined based on the detected motion of the image, a signal synthesizer that weights and adds the output image signal from the two-dimensional filter unit and the output image signal from the three-dimensional filter unit;
A noise removing unit that performs a noise removing process on the cyclic image signal input to the three-dimensional filter unit;
With
The image processing apparatus, wherein the cyclic coefficient when the image motion is small is determined to be smaller than the cyclic coefficient when the image motion is large.
The image processing apparatus according to claim 1, wherein the noise removing unit is a median filter, a weighted median filter, or a stack filter.
The noise removing unit compares the signal levels of the target pixel and the surrounding pixels, and the signal level of the target pixel is larger than a predetermined upper limit value determined based on the maximum signal level of the surrounding pixels. 2. The image processing according to claim 1, wherein the pixel value of the target pixel is replaced when the pixel value is smaller than a predetermined lower limit value determined based on a minimum signal level of the surrounding pixels. apparatus.
Storing the image signal of the previous frame as a cyclic image signal;
Performing a synchronization process for simultaneously synchronizing the input image signal and the accumulated cyclic image signal for each of a plurality of lines;
Performing a motion detection process for detecting a motion of an image based on the synchronized input image signal and the cyclic image signal;
Performing a two-dimensional filter process for reducing a noise component of the input image signal based on a correlation between a pixel of interest in the image and surrounding pixels;
The cyclic coefficient determined based on the detected motion of the image, and the cyclic coefficient determined to be a smaller value when the motion of the image is smaller than when the motion of the image is large, Performing a three-dimensional filter process for reducing a noise component of the input image signal by weighted addition of the input image signal and the cyclic image signal;
Performing a weighted addition of the output image signal after the two-dimensional filter processing and the output image signal after the three-dimensional filter processing using a weighting coefficient determined based on the detected motion of the image When,
Performing noise removal processing on the cyclic image signal before the three-dimensional filter processing;
An image processing method comprising:
A process of storing the image signal of the previous frame as a cyclic image signal;
A process of performing a synchronization process for simultaneously synchronizing the input image signal and the accumulated cyclic image signal for a plurality of lines;
A motion detection process for detecting a motion of an image based on the synchronized input image signal and the cyclic image signal;
Two-dimensional filter processing for reducing the noise component of the input image signal based on the correlation between the pixel of interest in the image and its surrounding pixels;
The cyclic coefficient determined based on the detected motion of the image, and the cyclic coefficient determined to be a smaller value when the motion of the image is smaller than when the motion of the image is large, Three-dimensional filter processing for reducing noise components of the input image signal by weighted addition of the input image signal and the cyclic image signal;
Using a weighting coefficient determined based on the detected motion of the image, a signal synthesis process for weighted addition of the output image signal after the two-dimensional filter processing and the output image signal after the three-dimensional filter processing;
Noise removal processing for the cyclic image signal before the three-dimensional filter processing;
An image processing program for causing a computer to execute.