KR101133520B1 - Color noise reduction method and device - Google Patents

Abstract

PURPOSE: A device for removing color noise and a method thereof are provided to compensate for the difference between the current frame image and the previous frame image by motion compensation. CONSTITUTION: A global noise removing unit(130) acquires a global noise removed image by estimating a color difference signal value for each pixel based on a block unit average value of the previous frame. A local noise removing unit(150) acquires a local noise removed image by using a magnetic component mask. A result combining unit(160) mixes the global noise removed image and the local noise removed image in a ratio corresponding to a weighted value.

Description

Color noise reduction method and device

The present invention relates to a method and apparatus for removing three-dimensional color noise using a noise level.

An image sensor is a semiconductor device that converts an optical image into an electrical signal. Among these, a charge coupled device (CCD) is a device in which charge carriers are stored and transported in a capacitor while individual metal-oxide-silicon (MOS) capacitors are located in close proximity to each other. Complementary MOS (CMOS) image sensor uses CMOS technology that uses a control circuit and a signal processing circuit as a peripheral circuit to make as many MOS transistors as the number of pixels, and sequentially detects the output of the pixels using the same. A device employing a switching method.

Portable devices (such as digital cameras, mobile communication terminals, etc.) equipped with such image sensors have been developed and sold. The image sensor is configured as an array of small photosensitive diodes called pixels or photosites. The pixels themselves usually do not extract color from light, but only convert photons from a broad spectral band into electrons. In order to record color images with a single sensor, the sensor is filtered so that different pixels receive different color illumination. This type of sensor is known as a color filter array. Different color filters are arranged in a predefined pattern across the sensor.

The image data provided from the image sensor is output as a luminance signal and a color signal through various signal processing such as color interpolation, luminance processing, color processing, and color format conversion through an image signal processing apparatus.

Since the color signal generated as described above includes noise according to digital signal processing, there is a need for a method of removing noise included in the color signal.

1 is a diagram illustrating an input mask used in a conventional method of removing color noise.

Referring to FIG. 1, 3x3, 3x5, and 3x7 structures are formed around a center pixel C24 among 3x7 mask inputs, and color noises are removed by using respective median values Cm1, Cm2, and Cm3.

However, since the color noise exists in a band form in a specific area, the size of the mask should be large. If color noise larger than the size of the 3x7 mask used by the conventional method occurs, it is difficult to remove the color noise.

The above-described background technology is technical information that the inventor holds for the derivation of the present invention or acquired in the process of deriving the present invention, and can not necessarily be a known technology disclosed to the general public prior to the filing of the present invention.

The present invention removes the color noise of the input mask unit in real time by using the overall color information of the previous frame image, and removes the color noise capable of correcting the difference between the previous frame image and the current frame image through motion compensation. To provide a method and apparatus.

The present invention also provides a method and apparatus for removing color noise that can effectively remove color noise in a large area as well as a small area by combining a color noise removing image in a mask unit by referring to the overall color information of the corrected image. It is to provide.

In addition, the present invention is to provide a color noise removal method and apparatus that can calculate the noise level to give a weight corresponding to the noise level and the user can adjust the color noise removal intensity through an adjustable variable.

Another object of the present invention is to provide a method and apparatus for removing color noise, which can remove color noise of a dark portion more strongly than other brightness regions by using brightness information of a previous frame image.

Other objects of the present invention will be readily understood through the following description.

According to an aspect of the present invention, an apparatus for removing color noise of an input image, the apparatus comprising: a global noise remover configured to predict a color difference signal value of each pixel by using a block unit average value of a previous frame to obtain a global noise remove image; A local noise removing unit configured to obtain a local noise removing image by using a magnetic component mask obtained by separating only a magnetic component corresponding to a center pixel among input masks having a predetermined size among the current frames; And a result combination unit configured to perform color noise removal by mixing the global noise removal image and the local noise removal image at a ratio corresponding to a weight.

And a motion compensator for predicting a motion vector between the previous frame and the current frame, and obtaining a motion compensated global noise canceled image by applying the motion vector to the global noise canceled image. The global noise canceled image may be mixed with the local noise canceled image.

The global noise remover may read the block unit average value of four block center points adjacent to the pixel position from a memory and predict the color difference signal value at the position by using bilinear interpolation. The global noise remover may calculate a color difference value that is a determination criterion of the mixing ratio in the result combination unit by using a difference between a block unit average value of the four block center points and the predicted color difference signal value.

The motion compensator calculates a frame motion vector prediction value from a difference between a block unit average value at a predetermined position of the current frame and a block unit average value at the same position of the previous frame, and calculates the frame difference vector value and the color difference value to the frame. Motion compensation may be performed by applying a motion vector prediction value.

The local noise canceling unit may include a contour preserving noise removing unit configured to perform contour preserving noise removal on a color difference signal of a pixel belonging to the magnetic component mask; A white point / black point corrector for performing white point / black point correction by using a color difference signal of a center mask including some pixels among the magnetic component masks; A local noise removal value calculator configured to calculate a local noise removal value by mixing the contour preservation noise removal value that is a result of the contour preservation noise removal unit and the white point / black point correction value that is a result of the white point / black point correction unit; can do.

The contour preservation noise removal unit may obtain the contour preservation noise removal value by weighting the surrounding pixels having a value similar to that of the center pixel through bidirectional filtering. The white point / black point correcting unit may calculate the white point / black point correction value by arranging the difference between pixel values of the center pixel and the center pixel in ascending order and averaging a predetermined number of small values.

The local noise removing unit averages a difference between a pixel value corresponding to a center pixel of the magnetic component mask and a color difference signal value of pixels belonging to the magnetic component mask among the motion compensated global removal images, and then adjusts the ratio of the mixing ratio in the result combination unit. The apparatus may further include a noise level predictor configured to calculate a noise level predicted value as a determination criterion.

The weight is calculated by using a color noise removal intensity that varies according to the magnitude of the luminance signal of the current frame, a first reference value calculated by the motion compensation unit, and a second reference value calculated by the local noise removal unit. Can be.

The first reference value is calculated using the difference between the block unit average value of the four block center points adjacent to the pixel position in the global noise removing unit and the predicted color difference signal value, and the motion compensation unit compensates for the motion. The second reference value is the color difference value of the pixel value corresponding to the center pixel of the magnetic component mask and the color difference signal value of the pixels belonging to the magnetic component mask in the motion compensated global removal image. The noise level may be calculated by averaging the difference.

The apparatus may further include a block average calculator configured to receive the color difference signal of the current frame and store the average of color difference signal values of pixels belonging to the block for each block having a predetermined size in a memory as a block unit average value of the current frame. The block average value may be used to remove color noise for the next frame.

Meanwhile, according to another aspect of the present invention, a method of removing color noise of an input image in a color noise removing apparatus and a recording medium on which a program for performing the same are recorded are provided.

According to an embodiment, there is provided a color noise removing method comprising: (a) obtaining a global noise removing image by predicting a color difference signal value of each pixel by using a block unit average value of a previous frame; (b) obtaining a local noise removing image by using a magnetic component mask obtained by separating only magnetic components corresponding to a center pixel among input masks having a predetermined size among the current frames; And (c) mixing the global noise removing image and the local noise removing image at a ratio corresponding to a weight to perform color noise removing.

(a1) predicting a motion vector between the previous frame and the current frame, and applying the motion vector to the global noise canceled image to obtain a motion compensated global noise canceled image, the step (c) May mix the motion compensated global noise canceled image and the local noise canceled image.

In the step (a), the block unit average value of four block center points adjacent to the pixel position may be read from a memory, and the color difference signal value at the position may be predicted using bilinear interpolation. Step (a) may calculate a color difference value which is a determination standard of the mixing ratio in step (c) by using the difference between the block unit average value of the four block center points and the predicted color difference signal value.

In the step (a1), a frame motion vector prediction value is calculated from a difference between a block unit mean value at a predetermined position of the current frame and a block unit mean value at the same position of the previous frame, and the color difference signal value and the color difference value are calculated. Motion compensation may be performed by applying the frame motion vector prediction value.

The step (b) comprises: (b1) performing contour preservation noise cancellation on the color difference signal of the pixel belonging to the magnetic component mask; (b2) performing white point / black point correction by using a color difference signal of a center mask including some pixels among the magnetic component masks; (b3) calculating a local noise removal value by mixing the contour preservation noise removal value that is the result of the step (b1) and the white point / black point correction value that is the result of the step (b2). .

In the step (b1), the contour preservation noise removal value may be obtained by weighting the surrounding pixels having a value similar to that of the center pixel through bidirectional filtering. The step (b2) may calculate the white point / black point correction value by arranging the difference between pixel values of the center pixel and the center pixel in ascending order, and then averaging a predetermined number of small values.

In the step (c), the step (b) is performed by averaging the difference between the pixel value corresponding to the center pixel of the magnetic component mask and the color difference signal value of the pixels belonging to the magnetic component mask among the motion compensated global removal images. The noise level prediction value used as a determination criterion of the mixing ratio can be calculated.

The weight is calculated by using a color noise removal intensity that varies according to the magnitude of the luminance signal of the current frame, a first reference value calculated by the motion compensation unit, and a second reference value calculated by the local noise removal unit. Can be.

The first reference value is calculated using the difference between the block unit average value of the four block center points adjacent to the pixel position in step (a) and the predicted color difference signal value, and the motion compensation in step (a1). The second reference value is a color difference value of the pixel value corresponding to the center pixel of the magnetic component mask and the color difference signal value of the pixels belonging to the magnetic component mask in the motion compensated global removal image in step (b). The noise level may be calculated by averaging the difference.

Receiving the color difference signal of the current frame and storing the average of the color difference signal values of pixels belonging to the block for each block having a predetermined size as a block unit average value of the current frame, in a block unit of the current frame. The average value can be used to remove color noise for the next frame.

Other aspects, features, and advantages will become apparent from the following drawings, claims, and detailed description of the invention.

According to an exemplary embodiment of the present invention, color noise in an input mask unit may be removed in real time by using the overall color information of the previous frame image, and the difference between the previous frame image and the current frame image may be corrected through motion compensation.

In addition, it is possible to effectively remove color noise in a large area as well as a small area by combining the color noise removing image in a mask unit by referring to the overall color information of the corrected image.

In addition, the noise level may be calculated to give a weight corresponding to the noise level, and the color noise removal intensity may be adjusted through a user adjustable variable.

Also, by using the brightness information of the previous frame image, color noise of the dark part may be more strongly removed than other brightness areas.

1 is a view showing an input mask used in the conventional method for removing color noise;
2 is a view showing a schematic configuration of an apparatus for removing color noise according to an embodiment of the present invention;
3 is a conceptual diagram related to a global noise removing method according to an embodiment of the present invention;
4 is a view for explaining a method of obtaining a motion vector according to an embodiment of the present invention;
5 is a view showing a schematic configuration of a local noise removing unit according to an embodiment of the present invention;
6 illustrates a magnetic component mask separated into an input mask and a magnetic component according to an embodiment of the present invention;
FIG. 7 is a view illustrating an input image and a motion compensated global noise reduction image used for contour preservation noise cancellation according to an embodiment of the present invention; FIG.
8 is a graph applied to an image resulting from mixing the contour preservation noise removal value and the white point / black point correction value according to an embodiment of the present invention;
9 is a graph relating to color noise removal intensity setting according to an embodiment of the present invention;
10 is a graph showing a relationship between a weight and a color noise removal value (final result) according to an embodiment of the present invention;
11 is a flowchart of a method for removing color noise according to an embodiment of the present invention;
12 is a view showing an original image and a result image of performing color noise removal according to an embodiment of the present invention.

BRIEF DESCRIPTION OF THE DRAWINGS The present invention is capable of various modifications and various embodiments, and specific embodiments are illustrated in the drawings and described in detail in the detailed description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all transformations, equivalents, and substitutes included in the spirit and scope of the present invention. In the following description of the present invention, if it is determined that the detailed description of the related known technology may obscure the gist of the present invention, the detailed description thereof will be omitted.

The terms first, second, etc. may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another.

The terminology used herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, action, component, part, or combination thereof described on the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

Also, the terms " part, "" module," and the like, which are described in the specification, refer to a unit for processing at least one function or operation, and may be implemented by hardware or software or a combination of hardware and software.

In the following description, well-known functions or constructions are not described in detail since they would obscure the invention in unnecessary detail.

Hereinafter, an embodiment of the present invention will be described in detail with reference to related drawings. In the present specification, the current frame is referred to as the nth frame, and the previous frame is referred to as the (n-1) th frame ((n-1). It is assumed that (th frame).

2 is a view showing a schematic configuration of a color noise removing apparatus according to an embodiment of the present invention, Figure 3 is a conceptual diagram of a global noise removing method according to an embodiment of the present invention, Figure 4 is a view of the present invention FIG. 5 is a diagram illustrating a method of obtaining a motion vector according to an embodiment. FIG. 5 is a diagram illustrating a schematic configuration of a local noise canceling unit according to an embodiment of the present invention. FIG. FIG. 7 is a diagram illustrating an input mask and a magnetic component mask separated into magnetic components, and FIG. 7 is a diagram illustrating an input image and a motion compensated global noise canceling image used for removing contour preservation noise according to an embodiment of the present invention. FIG. 8 is a graph applied to an image resulting from mixing the contour preservation noise removal value and the white point / black point correction value according to an embodiment of the present invention. FIG. A graph of the color noise reduction intensity setting in accordance with an embodiment of the invention, Figure 10 is a graph showing the relationship between the weight and the color noise reduction value (the end result) according to an embodiment of the present invention.

The color noise removing apparatus 100 according to an embodiment of the present invention removes global noise using a previous frame image, performs motion compensation using a predicted motion vector, and removes local noise using magnetic components. It is possible to remove color noise larger than the mask by mixing the respective result images.

The color noise removing apparatus 100 according to the present embodiment will be described on the assumption that the format of the input video signal is YCbCr. For input video signals of other formats (e.g., RGB signals, YUV signals, etc.), a format converter (not shown) is provided at the front of the color noise removing device 100, and the format is converted into a YCbCr signal. The color noise removing process according to the invention may be performed.

Referring to FIG. 2, the color noise removing apparatus 100 may include a block average calculator 110, a memory 120, a global noise remover 130, a motion compensator 140, a local noise remover 150, The result combiner 160 is included.

In the present exemplary embodiment, the input image signal may be divided into a luminance signal Y_in and a color difference signal C_in = Cb, Cr, and the luminance signal Y_in is input to the result combination unit 160, and the color difference signal ( C_in) is input to the block average calculator 110 and the local noise remover 150.

The block average calculation unit 110 calculates a block unit average value avg_data (nth) of a block of size NxN with respect to the color difference signal C_in (nth) among the input image signals of the current frame, and stores the average value in the memory 120. Save it. N is a natural number and may be one of the common factors of the horizontal length and the vertical length of the input image (for example, 128), and the average value of each block may be an average value of color difference signals of pixels belonging to the corresponding block.

The block unit average value stored in the memory 120 is regarded as a color difference signal value representative of the block, and used to remove global noise for the next frame ((n + 1) th frame).

The global noise remover 130 is the target of the current color noise removal among the block average values of the previous frame ((n-1) th frame) stored in the memory 120 to remove the global noise with respect to the current frame. Four block unit average values (a, b, c, d) adjacent to the pixel position to be read are read to predict the color difference signal value at the corresponding position.

Color difference signal value prediction at the pixel position P1 may be performed through, for example, bi-linear interpolation.

Referring to FIG. 3, it can be seen that the input image 1 is divided into a plurality of N × N sized blocks, and a block unit average value of the previous frame for each block is stored in the memory 120. In this case, the block average value of each block is considered to be a color difference signal value at the center pixel position of the block.

Assume that the pixel position P1 for which the color difference signal value is to be predicted is (x, y). In this case, the square 10 located at the center of each block is found so that the corresponding position is located therein. In the drawing illustrated in FIG. 3, the rectangles 10 connecting the centers of blocks 1 (B1), 2 (B2), 3 (B3), and 4 (B4) are blocks 1 to 4 (B1 to B4). The average value of each block is assumed to be a, b, c, and d as to the previous frame ((n-1) th frame) stored in the memory 120.

The interpolation may be performed in proportion to the distance between the current pixel position (x, y) and the four vertices (a, b, c, d) of the rectangle 10. This will be described in detail as follows.

The block unit average value of the blocks 1 to 4 (B1 to B4) is read from the memory 120, and from the position coordinates (x, y) of the current pixel position P1 to the sides of the rectangle 10 according to Equation 1 below. The distance ratio of x_length and y_length is obtained. Here, the distance ratio is a value representing the distance from the current position (x, y) of the four sides of the quadrangle 10 to the two closest sides as a ratio between 0 and 1.

[Equation 1]

Where% is the remainder operation and / is the division operation.

Then, two virtual averages av1 and av2 are obtained by interpolation from the distance ratio according to Equation 2 below.

[Equation 2]

A color difference signal value PD = avg_out ((n-1) th) of the current position is predicted from two virtual averages according to Equation 3 below.

&Quot; (3) "

PD = (1-y_length) * av1 + y_length * av2

The global noise removing unit 130 may predict the color difference signal values through the above-described processes (Equations 1 to 3) for all pixels of the current frame of the input image.

In addition, the global noise remover 130 uses the difference between the four neighboring values a, b, c, and d and the predicted color difference signal value PD as shown in Equation 4 below. difference value)

&Quot; (4) "

CDValue = (| PD-a | + | PD-b | + | PD-c | + | PD-d |) / 4

Referring back to FIG. 2, the motion compensator 140 estimates and compensates for a motion generated due to a time interval between a previous frame (n−1) th frame and a current frame (nth frame).

Since the global noise canceller 130 uses data of the previous frame, a phenomenon caused by the movement of the subject or the camera due to the time interval between the previous frame and the current frame (for example, motion blur Due to blurring), etc.) may occur. Therefore, it is necessary to predict the motion between the previous frame and the current frame and compensate for it.

In the color difference signal C_in used in the global noise canceller 130, the shape information of the object is markedly less than that of the luminance signal Y_in. That is, the low frequency information of the image sensitive to the human eye can be said. Therefore, even if the motion vector between the previous frame and the current frame is roughly predicted and compensated accordingly, the motion compensation unit 140 indicates that the error in the current frame is not large. I use it.

Referring to FIG. 4, the motion compensator 140 may block the average value of the unit of the current frame (avg_out _k (nth) at each of a predetermined number (eg, four) positions when a current frame (nth frame) is input). ), And the block motion vector prediction value mve _k is calculated based on a difference between the x-axis and y-axis of the block unit average value avg_out _k ((n-1) th) of the previous frame (see Equation 5).

[Equation 5]

mve _k = avg_out _k (nth)-avg_out _k ((n--th), (k = 1, 2, 3, 4)

As the frame motion vector prediction value MVE ', a median of the block motion vector prediction values calculated by Equation 5 is used (see Equation 6). Here, the frame motion vector prediction value is a value representing the degree of shaking.

&Quot; (6) "

MVE '= median (mve _k )

The motion compensation is performed by applying the frame motion vector predicted value MVE 'to the global noise canceled image obtained by the global noise remover 130, that is, the block average value avg_out, and the frame motion is also applied to the color difference value CDValue. The motion compensation is performed by applying the vector prediction value MVE '(see Equation 7).

[Equation 7]

Here, m_CDValue is a motion compensated color difference value used in the result combining unit 160 to determine a blending ratio for mixing the global noise removal value and the local noise removal value, which will be described later.

In addition, the global noise elimination image m_avg_out on which motion compensation is performed may be obtained through Equation 7.

Referring back to FIG. 1, the local noise remover 150 extracts a magnetic component mask among the input masks to perform local noise removal.

An example of a magnetic component mask separating only the input mask and the magnetic component for local noise removal is shown in FIG. 6. (a) is an input mask of 9x9 size, and (b) is a 5x5 size magnet obtained by separating only the magnetic component (hatched portion of the input mask of (a)) which is the same color component signal as the center pixel c1_55 of the input mask. Ingredient mask. Here, one of c1 and c2 is one of the color difference signals Cb and Cr, and the other is the other color difference signal. That is, c2 = Cr if c1 = Cb, and c2 = Cb if c1 = Cr. In addition, the size of the input mask is 9x9 and the size of the magnetic component mask is 5x5 is just one example, and various sizes may be applied according to the level of local noise cancellation.

The local noise remover 150 removes the local noise by performing contour preservation noise removal and white point / black point correction on the color difference signal belonging to the magnetic component mask and mixing the results, and in the result combination unit 160 which will be described later. The noise level, which is a reference value for determining the mixing ratio, is predicted.

Referring to FIG. 5, the local noise removing unit 150 includes the contour preserving noise removing unit 152, the white / black point correcting unit 154, the local noise removing value calculating unit 156, and the noise level predicting unit 158. Include.

The contour preserving noise removal unit 152 performs noise removal in which the contour is preserved using, for example, a bilateral filter. When the noise is removed using the bidirectional filter, first, the color difference signal corresponding to the magnetic component mask corresponds to C1 ₁₁ to C1 ₅₅ shown in FIG. 7A and has a peripheral pixel value similar to that of the center pixel C1 ₃₃ . Weight for.

[Equation 8]

Referring to Equation 8, a noise removal process using a bidirectional filter is described. Where cnr_c1_nrf_str is the noise removal intensity for c1. If the center pixel is c2, it may be replaced by cnr_c2_nrf_str, that is, the noise removal intensity for c2. cnr_c1_nrf_str and cnr_c2_nrf_str are parameters that can be adjusted by the user and can be determined statistically and experimentally. Diff _ij is the square of the difference between the i-th row of the magnetic component mask, the j-th column pixel C1 _ij , and the center pixel C1 ₃₃ , and gm _ij is the i-th row, j-th column pixel of the magnetic component mask. Gm_in _ij is a product of weights and pixel values for the i-th row and j-th column pixels of the magnetic component mask, and out_1st is a contour conservation noise removal value.

The white point / black point corrector 154 is used when it is necessary to remove popping noise such as white point or black point, and performs white point / black point correction. During white point / black point correction, a center mask 20 (here, a 3x3 mask) having a predetermined size including the center pixel C133 among the magnetic component masks shown in FIG. 7 is used.

[Equation 9]

S [a] is a value obtained by ascending the pixel value difference between the i-th row, j-th column pixel C1 _ij and the center pixel C1 ₃₃ of the center mask 20, and out_2nd is a white / black point correction value. It is the median value in [a].

The local noise removal value calculation unit 156 applies a weight (orf_weight) to the contour preservation noise removal value out_1st and the white point / black point correction value out_2nd and mixes them at an appropriate ratio, thereby removing the local noise as shown in FIG. 7. Compute the value (cnr_1st_out).

[Equation 10]

orf_weight is the product of the average of the four smaller values of S [a] and the noise reduction strength (cnr_orf_str). The round function rounds off below the decimal point. Here, the popping noise canceling intensity (cnr_orf_str) is a parameter that can be adjusted by the user and can be determined statistically and experimentally. Also, in calculating orf_weight, two, three or five or six other than the small four of S [a] may be selected and the average may be used.

Referring to FIG. 8, a function for obtaining a local noise removing value cnr_1st_out according to Equation 10 is illustrated.

The local noise remover 150 mixes the contour preservation noise removal value and the white point / black point correction value at an appropriate ratio to remove the popping noise (white point / black point) that could not be removed by the simple bidirectional filter, and corrects the white point / black point. And it is possible to prevent the step phenomenon that can occur at the boundary of the contour preservation noise removal.

The noise level prediction unit 158 calculates a noise level prediction value NEValue at the pixel position using the motion compensated global noise reduction image m_avg_out (Equation 11).

[Equation 11]

Referring to FIG. 7B, the magnetic component mask is applied to the motion compensated global noise canceled image m_avg_out, and then the motion compensated global noise canceled value m_avg33 corresponding to the center pixel is applied to the magnetic component mask. The noise level prediction value may be calculated by averaging the differences between the color difference signal values C11, C12,..., And C55 of the pixels to which the pixels belong.

Referring back to FIG. 2, the result combination unit 160 is output through the global noise canceller 130 and the motion compensator 140 obtained according to the result of comparing the motion compensated color difference value and the noise level prediction value. The resultant image (motion compensated global noise removal value) and the resultant image (local noise removal value) output through the local noise removal unit 150 are mixed at an appropriate ratio, and the size of the input mask (for example, Color noise larger than 9x9 size shown in (a) is removed.

First, the result combining unit 160 adjusts the color noise removing intensity by the magnitude of the luminance signal Y_in, that is, the luminance Y_Luma_value. A method of setting the color noise removal intensity (cnr_c1_str or cnr_c2_str) is shown in FIG. In the present exemplary embodiment, the center pixel of the input mask is c1 and is described based on c1. However, the same content is also applicable to c2.

Referring to FIG. 9, color noise removal intensity may be set differently for each luminance region to efficiently remove color noise in a low luminance or high luminance region.

cnr_LL_thr is a threshold that distinguishes low and medium luminance areas, cnr_HL_thr is a threshold that distinguishes medium and high luminance areas, and cnr_c1_luma_width is the width interpolated between each luminance when connecting by applying intensity between each luminance area. , cnr_c1_LL_mix_str, cnr_c1_ML_mix_str, and cnr_c1_HL_mix_str are mixed intensities for c1 in the low luminance region, the medium luminance region, and the high luminance region, respectively.

cnr_LL_thr, cnr_HL_thr, cnr_c1_luma_width, cnr_c1_LL_mix_str, cnr_c1_ML_mix_str, cnr_c1_HL_mix_str are user adjustable parameters and can be determined statistically and experimentally. In addition, cnr_c1_luma_width, cnr_c1_LL_mix_str, cnr_c1_ML_mix_str, cnr_c1_HL_mix_str are parameters for c1 and may be adjusted to values for c2 when the center pixel of the input mask is c2.

The result combination unit 160 performs motion compensation using the color noise removal intensity cnr_c1_str determined according to the luminance region to which the target pixel belongs, the motion compensated color difference value m_CDValue, and the noise level prediction value NEValue. Set the FinalWeight to adjust the mixing ratio of the global noise reduction value and the local noise reduction value.

[Equation 12]

The weight may be set as in Equation 12. That is, after comparing the motion-compensated color difference value (m_CDValue) and the noise level prediction value (NEValue), a large value is set as a middle weight, and the intermediate weight is multiplied by the color noise removal intensity (cnr_c1_str) adjusted for each luminance region. By setting the weight (FinalWeight) can be set.

The result combiner 160 mixes the motion compensated global noise canceled image m_avg_out and the local noise canceled image cnr_1st_out according to the set weight, and finally calculates and outputs the color noise canceled value CNR_out. It is possible to remove color noise larger than the size of.

The relationship between the weight FinalWeight and the color noise removal value CNR_out is shown in FIG. 10, and may be expressed as a first-order function in which the weight is an independent variable and the color noise removal value is a dependent variable, as shown in Equation 13 below.

[Equation 13]

The color noise removing apparatus 100 according to the present embodiment is one of the components of an image signal processor, and is a color that is one of signal processing performed on an image signal input through an image sensor. It may be a component that performs noise removal.

Hereinafter, a method of removing color noise performed by the color noise removing apparatus 100 will be described with reference to FIG. 11.

11 is a flowchart illustrating a method of removing color noise according to an embodiment of the present invention. Each step described below may be performed by each internal component of the color noise removing apparatus 100.

First, in step S200, an image signal of a current frame, which is a color noise removal target, is input. The image signal is divided into a luminance signal Y_in and a chrominance signal C_in, and the chrominance signal C_in is used for global noise removal and local noise removal, and the luminance signal Y_in is generated according to global noise removal and local noise removal. It is used to blend each resultant image.

In operation S210, the block average calculator 110 receives the color difference signal C_in of the current frame, calculates a block unit average value, and stores the average value in the block 120 in the memory 120. The block unit average value of the current frame stored in the memory 120 is used to remove the global noise for the next frame.

Next, in step S220, the global noise remover 130 removes global noise by using a block unit average value for the previous frame stored in the memory 120.

For example, the color difference signal value avg_out for each pixel is predicted using bilinear interpolation. A color difference value (CDValue) is obtained by using a difference between a color difference signal value and a block unit average value of four points adjacent to the corresponding pixel.

Next, in operation S230, the motion compensator 140 predicts and compensates for the motion between the previous frame and the current frame with respect to the color difference signal value avg_out and the calculated color difference value CDValue for the previous frame.

To this end, the motion compensator 140 calculates a block motion vector prediction value from the difference between the average value of the block unit at a predetermined position of the current frame and the average value of the block unit at the same position of the previous frame (step S232), and calculates the intermediate values thereof. The frame motion vector predicted value is used (step S234). Then, motion compensation is performed by applying the frame motion vector prediction value to the color difference signal value and the color difference value obtained in step S220 to obtain a motion compensated global noise canceled image m_avg_out and a motion compensated color difference value m_CDValue (step S236). ).

In operation S240, the local noise remover 150 applies an input mask to the color difference signal, extracts a magnetic component mask, and performs local noise removal.

In operation S242, the contour preservation noise removing unit 152 removes the contour preservation noise with respect to the color difference signal of the pixel belonging to the magnetic component mask. Contour preservation noise removal may be achieved through bi-directional filtering, for example, and has been described with reference to Equation 8 above.

In operation S244, the white point / black point corrector 154 performs white point / black point correction by using a color difference signal of some pixels including the center pixel among the magnetic component masks. When it is necessary to remove the popping noise such as the white point or the black point, the white point / black point correction is performed, which has been described with reference to Equation 9 above.

In step S246, the local noise removal value calculator 156 mixes the resultant image (contour preservation noise removal value out_1st and the white point / black point correction value out_2nd) performed in the previous step S242 at an appropriate ratio to reduce the local noise removal value. Compute (cnr_1st_out).

In operation S248, the noise level predictor 158 calculates a noise level predicted value NEValue at the pixel position using the motion compensated global noise canceled image. This has been described with reference to Equation 11 above.

In operation S250, the result combining unit 160 compares the motion compensated color difference value m_CDValue and the noise level prediction value NEValue with the motion compensated global noise reduction image m_avg_out and the local noise reduction image cnr_1st_out. To remove the color noise.

Here, the mixing ratio of the motion compensated global noise canceled image m_avg_out and the local noise canceled image cnr_1st_out is adjusted according to the magnitude of the luminance signal Y_in and the color noise canceled intensity cnr_1st_str and the motion compensated color difference value ( m_CDValue) and the noise level predicted value NEValue.

It is apparent that the above-described method for removing color noise may be performed by an automated procedure according to a time series sequence by a software program or the like embedded in the color noise removing apparatus 100. The codes and code segments that make up the program can be easily deduced by a computer programmer in the field. In addition, the program is stored in a computer-readable information storage medium, and the program is read and executed by a computer to implement the method. The information storage medium includes a magnetic recording medium, an optical recording medium, and a carrier wave medium.

12 is a view showing an original image and a result image of color noise removal performed according to an embodiment of the present invention.

FIG. 12A illustrates an original image in which color noise exists, and FIG. 12B illustrates a result image in which color noise removal is performed according to the present invention. Through this, it can be confirmed that color noise that is larger than the size of the input mask (for example, 9x9) and spreads widely throughout the screen is effectively removed.

According to the present invention, the color information of the entire region and the noise level of the local region are measured with respect to the input image (color difference signal image), and the color noise is removed based thereon. Since color information of the entire area is calculated with respect to the input image of the previous frame, real-time application is possible, and motion compensation is additionally performed to reduce an error with the current frame.

By weighting the combined color information in the entire region and the information from which the color noise is removed in the local region, it is possible to effectively remove the color noise even if it is widely spread larger than the size of the input mask.

In addition, in the process of removing color noise in the local area, noise is removed by weighting neighboring pixels having a value similar to that of the center pixel, thereby minimizing the influence due to surrounding pixels, thereby minimizing problems such as color bleeding. Can be.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit or scope of the invention as defined in the following claims And changes may be made without departing from the spirit and scope of the invention.

100: color noise removing device
110: block average calculation unit
120: memory
130: global noise canceller
140: motion compensation unit
150: local noise removing unit
160: result combination
152: contour preservation noise removing unit
154: white point / black point correction unit
156: local noise removal value calculator
158: noise level prediction unit

Claims (25)

A device for removing color noise of an input image,
A global noise removing unit configured to obtain a global noise removing image by predicting a color difference signal value of each pixel by using a block unit average value of a previous frame;
A local noise removing unit configured to obtain a local noise removing image by using a magnetic component mask obtained by separating only a magnetic component corresponding to a center pixel among input masks having a predetermined size among the current frames; And
And a result combination unit configured to perform color noise removal by mixing the global noise removing image and the local noise removing image at a ratio corresponding to a weight.
The method of claim 1,
The apparatus may further include a motion compensator configured to predict a motion vector between the previous frame and the current frame, and obtain a motion compensated global noise canceled image by applying the motion vector to the global noise canceled image.
And the result combination unit mixes the motion compensated global noise canceled image with the local noise canceled image.
The method of claim 2,
The global noise removing unit reads the block unit average value of four block center points adjacent to the pixel position from a memory and predicts the color difference signal value at the position by using bilinear interpolation. Noise reduction device.
The method of claim 3,
The global noise canceller calculates a color difference value that is a determination criterion of the mixing ratio in the result combination unit by using a difference between a block unit average value of the four block center points and the predicted color difference signal value. Color noise reduction device.
The method of claim 4, wherein
The motion compensator calculates a frame motion vector prediction value from a difference between a block unit average value at a predetermined position of the current frame and a block unit average value at the same position of the previous frame, and calculates the frame difference vector value and the color difference value to the frame. Color noise removal apparatus characterized in that to perform a motion compensation by applying the motion vector prediction value.
The method of claim 1,
The local noise removing unit,
An outline preservation noise removal unit for performing outline preservation noise removal on the color difference signal of the pixel belonging to the magnetic component mask;
A white point / black point corrector for performing white point / black point correction by using a color difference signal of a center mask including some pixels among the magnetic component masks;
A local noise removal value calculator configured to calculate a local noise removal value by mixing the contour preservation noise removal value that is a result of the contour preservation noise removal unit and the white point / black point correction value that is a result of the white point / black point correction unit; Color noise removal device characterized in that.
The method of claim 6,
And the contour preserving noise removing unit obtains the contour preserving noise removing value by weighting the surrounding pixels having a value similar to that of the center pixel through bidirectional filtering.
The method of claim 6,
And the white point / black point correcting unit calculates the white point / black point correction value by averaging a predetermined number of small values after ascending the pixel value difference between the peripheral pixel of the center mask and the center pixel.
The method of claim 6,
The local noise removing unit averages the difference between the pixel value corresponding to the center pixel of the magnetic component mask and the color difference signal value of the pixels belonging to the magnetic component mask in the motion compensated global noise removing image to determine the mixing ratio of the result combination unit. And a noise level predictor for calculating a noise level predicted value as a determination criterion.
The method of claim 2,
The weight is calculated by using a color noise removal intensity that varies according to the magnitude of the luminance signal of the current frame, a first reference value calculated by the motion compensation unit, and a second reference value calculated by the local noise removal unit. Color noise removal device characterized in that the.
The method of claim 10,
The first reference value is calculated using the difference between the block unit average value of the four block center points adjacent to the pixel position in the global noise removing unit and the predicted color difference signal value, and the motion compensation unit compensates for the motion. Color difference,
The second reference value is calculated by averaging a difference between a pixel value corresponding to a center pixel of the magnetic component mask and a color difference signal value of pixels belonging to the magnetic component mask among the motion compensated global removal images by the local noise removing unit. It is a noise level prediction value, The color noise removal apparatus characterized by the above-mentioned.
The method of claim 1,
The apparatus may further include a block average calculator configured to receive the color difference signal of the current frame and store the average of the color difference signal values of pixels belonging to the block for each block having a predetermined size in a memory as a block unit average value of the current frame.
The block unit average value of the current frame is used to remove the color noise for the next frame.
As a method of removing color noise of an input image by a color noise removing device,
(a) obtaining a global noise reduction image by predicting a color difference signal value of each pixel by using a block unit average value of a previous frame;
(b) obtaining a local noise removing image by using a magnetic component mask obtained by separating only magnetic components corresponding to a center pixel among input masks having a predetermined size among the current frames; And
and performing color noise removal by mixing the global noise removal image and the local noise removing image at a ratio corresponding to a weight.
The method of claim 13,
(a1) predicting a motion vector between the previous frame and the current frame and obtaining the motion compensated global noise canceled image by applying the motion vector to the global noise canceled image,
In the step (c), the color noise removing method comprises mixing the motion compensated global noise removing image and the local noise removing image.
The method of claim 14,
The step (a) is characterized in that the block unit average values for the four block center points adjacent to the pixel position are read out from a memory and the color difference signal value is predicted at the position using bilinear interpolation. How to remove color noise.
16. The method of claim 15,
The step (a) is to calculate the color difference value that is the determination standard of the mixing ratio in the step (c) by using the difference between the block unit average value of the four block center points and the predicted color difference signal value Color noise reduction method.
The method of claim 16,
In the step (a1), a frame motion vector prediction value is calculated from a difference between a block unit mean value at a predetermined position of the current frame and a block unit mean value at the same position of the previous frame, and the color difference signal value and the color difference value are calculated. And performing motion compensation by applying the frame motion vector prediction value.
The method of claim 13,
The step (b)
(b1) performing contour preservation noise reduction on the color difference signal of the pixel belonging to the magnetic component mask;
(b2) performing white point / black point correction by using a color difference signal of a center mask including some pixels among the magnetic component masks;
(b3) calculating a local noise removal value by mixing the contour preservation noise removal value that is the result of the step (b1) and the white point / black point correction value that is the result of the step (b2). Color noise reduction method.
The method of claim 18,
In the step (b1), the contour preservation noise removing value is obtained by weighting the surrounding pixels having a value similar to that of the center pixel through bidirectional filtering.
The method of claim 18,
In the step (b2), the white noise / black dot correction value is calculated by arranging the difference between pixel values of the center pixel and the center pixel in ascending order and then averaging a predetermined number of small values.
The method of claim 18,
In the step (c), the step (b) is performed by averaging the difference between the pixel value corresponding to the center pixel of the magnetic component mask and the color difference signal value of the pixels belonging to the magnetic component mask among the motion compensated global noise canceled images. And calculating a noise level prediction value that is a determination criterion of the blending ratio.
The method of claim 13,
The weight is calculated by using a color noise removal intensity that varies according to the magnitude of the luminance signal of the current frame, a first reference value calculated by the motion compensation unit, and a second reference value calculated by the local noise removal unit. A color noise removing method characterized by the above-mentioned.
The method of claim 22,
The first reference value is calculated using the difference between the block unit average value of the four block center points adjacent to the pixel position in step (a) and the predicted color difference signal value, and the motion compensation in step (a1). Color difference,
The second reference value is calculated by averaging a difference between a pixel value corresponding to a center pixel of the magnetic component mask and a color difference signal value of pixels belonging to the magnetic component mask among the motion compensated global removal images in step (b). It is a noise level prediction value, The color noise removal method characterized by the above-mentioned.
The method of claim 13,
Receiving the color difference signal of the current frame and storing the average of the color difference signal values of pixels belonging to the block for each block having a predetermined size in a memory as a block unit average value of the current frame,
The block unit average value of the current frame is used to remove the color noise for the next frame.
A recording medium on which a program of instructions that can be executed by a digital processing apparatus for performing the color noise removing method according to any one of claims 13 to 24 is tangibly embodied and can be read by the digital processing apparatus.

Images