WO2014087909A1 - Image processing device - Google Patents

Abstract

Provided is a technology whereby artifacts, which arise when carrying out noise reduction, sharpening, or other image enhancement processes on an image, are reduced. An image processing device comprises: a setting unit which, using signal values of each pixel in an inputted image signal, sets a likelihood with respect to a predetermined object; a parameter output unit which outputs a parameter for image processing, wherein a first parameter which is employed in image processing with respect to the object and a second parameter which is employed in image processing with respect to another object, are mixed at a mixing ratio based on the likelihood; and an image processing unit which, using the parameter for image processing which is outputted from the parameter output unit, carries out an image process on the signal values of each pixel.

Description

Image processing device

The present invention relates to an image processing apparatus, and more particularly to a technique for performing image quality improvement processing such as noise reduction processing and sharpening processing according to an image.

Japanese Patent Application Laid-Open No. 2010-152518 discloses that a sharpening process for a specific color area such as a face image area is reduced from other areas, thereby sharpening an image such as a landscape and a natural image of a human face. A technique for performing a sharpening process is disclosed. In this technique, sharpness processing is performed by setting a sharpness processing parameter for a specific color region lower than a level set for a region other than the specific color region.

Japanese Patent Laid-Open No. 2010-152518 switches sharpness processing between a specific color region and a region other than the specific color region. For this reason, when the boundary between the specific color area and the area other than the specific color area is ambiguous, sharpness processing is finely switched at the boundary portion, which may cause artifacts. On the other hand, when image quality improvement processing such as noise reduction processing or sharpening processing is performed uniformly on the entire image, artifacts may occur depending on the object included in the image. For example, in the case of an image that contains fine linear textures at random, such as turf and green trees, the linear texture is misrecognized as noise such as jaggy around the contour and smoothed. May result in an unnatural image. Further, in such an image area, the effect of the sharpening process is felt weak.

An object of the present invention is to provide a technique for reducing artifacts generated when image processing such as noise reduction or sharpening is performed on an image.

An image processing apparatus according to a first aspect of the present invention includes an image processing unit that performs image processing on a signal value of each pixel in an image signal using an input image processing parameter, and a signal value of each pixel in the image signal To set the likelihood of the pixel for a predetermined object, a predetermined first parameter for use in the image processing for the object, and for use in the image processing for another object A parameter output unit that outputs the image processing parameter obtained by mixing the predetermined second parameter with a mixture ratio based on the likelihood set by the setting unit to the image processing unit.

In a second aspect based on the first aspect, the setting unit converts the signal value of each pixel into a feature amount representing the size of the component in a feature space composed of a plurality of components representing the feature of the object. Conversion is performed to obtain a scalar value based on the feature amount, and the likelihood is set based on the scalar value.

In a third aspect based on the second aspect, the feature space is a color space including a first component and a second component constituting a color.

In a fourth aspect based on the second aspect or the third aspect, the feature space has a frequency space including a predetermined frequency component included in the object.

In a fifth aspect based on the third aspect, the setting unit converts the signal value of each pixel into the feature amount in the color space, and uses a vector representing the range of the object in the feature space. The scalar value mainly representing the first component and the second component is obtained.

In a sixth aspect based on the third aspect, the setting unit converts the signal value of each pixel into the feature amount in the color space, and a plurality of regions representing the range of the object in the color space. A distance between each determined reference point and the feature amount is obtained as the scalar value.

In a seventh aspect based on the fourth aspect, the setting unit obtains the frequency component of the pixel in the frequency space from the signal value of each pixel, and sets the obtained frequency component as the scalar value of the pixel.

According to an eighth invention, in any one of the first to seventh inventions, the image processing unit performs a smoothing process on the signal value in the image signal as the image processing, and the image At least one of second image processing for performing sharpening processing on the signal value in the signal is performed.

According to the configuration of the present invention, it is possible to reduce artifacts generated when image processing such as noise reduction and sharpening is performed on an image.

FIG. 1 is a block diagram illustrating a schematic configuration of the image processing apparatus according to the first embodiment. FIG. 2 is a circuit diagram of the feature space conversion unit in the first embodiment. FIG. 3A is a diagram illustrating the color of the detection target object on the UV plane. FIG. 3B is a diagram illustrating a range that the scalar value S1 can take on the UV plane. FIG. 3C is a diagram illustrating a possible range of the scalar value S2 on the UV plane. FIG. 4 is a diagram illustrating a region on the UV plane corresponding to the scalar value. FIG. 5 is a schematic circuit diagram of the parameter output unit in the first embodiment. FIG. 6 is a diagram illustrating a schematic configuration of a noise reduction processing unit in the first embodiment. FIG. 7 is a block diagram illustrating a configuration of the difference image generation unit in the first embodiment. FIG. 8 is a block diagram illustrating a schematic configuration of the image processing apparatus according to the second embodiment. FIG. 9A is a diagram illustrating a region on the UV plane representing the color of the detection target object in the second embodiment. FIG. 9B is a diagram illustrating a region on the UV plane representing the color of the detection target object in the second embodiment. FIG. 9C is a diagram illustrating a region on the UV plane representing the color of the detection target object in the second embodiment. FIG. 10 is a diagram illustrating a region having a high likelihood on the UV plane in the second embodiment. FIG. 11 is a block diagram illustrating a schematic configuration of the image processing apparatus according to the modification (1). FIG. 12 is a block diagram illustrating a schematic configuration of the sharpening processing unit in the modified example (1). FIG. 13 is a block diagram illustrating a configuration example of the high-frequency component output unit in Modification Example (1). FIG. 14 is a block diagram illustrating a configuration example of the nonlinear filter in the modification example (1). FIG. 15 is a block diagram illustrating a configuration example of the two-dimensional high-pass filter unit in the modification (1). FIG. 16 is a block diagram illustrating a configuration example of the nonlinear arithmetic unit in the modification example (1).

An image processing apparatus according to an embodiment of the present invention includes: an image processing unit that performs image processing on a signal value of each pixel in an image signal using an input image processing parameter; and A setting unit that sets the likelihood of the pixel for a predetermined object using a signal value, a predetermined first parameter for use in the image processing for the object, and the image processing for another object A parameter output unit that outputs the image processing parameter, which is obtained by mixing a predetermined second parameter for use with a mixture ratio based on the likelihood set by the setting unit, to the image processing unit ( First configuration). According to the first configuration, when an object that may cause artifacts during image processing is included in the image signal, the likelihood indicating the object likeness is set for each pixel in the image signal, Image processing is performed on each pixel using a parameter corresponding to the likelihood. As a result, since it is possible to perform image processing suitable for the object, artifacts due to image processing can be reduced as compared to the case where image processing is performed using uniform parameters for the entire image.

According to a second configuration, in the first configuration, the setting unit converts the signal value of each pixel into a feature amount that represents the size of the component in a feature space that includes a plurality of components that represent the feature of the object. Conversion may be performed to obtain a scalar value based on the feature amount, and the likelihood may be set based on the scalar value. According to the second configuration, the likelihood for an object is set for each pixel in a feature space that represents a feature of the object that may cause an artifact during image processing. Therefore, it is possible to perform appropriate image processing for the pixels having the object characteristics.

In a third configuration, the feature space may be a color space including a first component and a second component constituting a color in the second configuration. According to the third configuration, the present invention can be applied to an object having a color feature.

In a fourth configuration according to the second or third configuration, the feature space may include a frequency space including a predetermined frequency component included in the object. According to the fourth configuration, the likelihood can be set using the frequency component included in the object.

According to a fifth configuration, in the third configuration, the setting unit converts the signal value of each pixel into the feature amount in the color space, and uses a vector representing the range of the object in the feature space. The scalar value mainly representing the first component and the second component may be obtained.

According to a sixth configuration, in the third configuration, the setting unit converts the signal value of each pixel into the feature amount in the color space, and a plurality of regions representing the range of the object in the color space. The distance between each determined reference point and the feature amount may be obtained as the scalar value.

According to a seventh configuration, in the fourth configuration, the setting unit obtains a frequency component of the pixel in the frequency space from a signal value of each pixel, and uses the obtained frequency component as the scalar value of the pixel. It is good.

According to an eighth configuration, in any one of the first to seventh configurations, the image processing unit performs a smoothing process on the signal value in the image signal as the image processing, and the image It is good also as performing at least one of the 2nd image processing which performs a sharpening process about the said signal value in a signal. According to the eighth configuration, it is possible to perform at least one of the smoothing process or the sharpening process on the signal value of each pixel in the image signal using a parameter corresponding to the likelihood for the object. Therefore, for example, if the object is trees or grass with a fine texture, adjust the image processing parameters so that the intensity of the smoothing process is weakened for the pixels that contain the object in the image signal. Can do. As a result, it is possible to reduce artifacts due to the smoothing process as compared to the case where the smoothing process is uniformly performed on the entire image. In addition, since the image processing parameters can be adjusted so as to increase the sharpening processing strength for the pixel containing the object, the sharpening effect of the pixel portion is strengthened compared to other pixels. Is possible.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the drawings, the same or corresponding parts are denoted by the same reference numerals and description thereof will not be repeated.

<First Embodiment>
(Overview)
In this embodiment, for each pixel value of the image signal input to the image processing apparatus, the likelihood for the detection target object is set according to the degree to which the characteristics of a predetermined object (hereinafter, detection target object) are included. Then, image quality improvement processing (image processing) according to the likelihood is performed. The likelihood indicates the likelihood of the detection target object of each pixel value in the image signal. In the present embodiment, as an example of the detection target object, a specific color area (here, a green area such as grass or trees) in the image signal will be described as an example.

(Constitution)
FIG. 1 is a block diagram illustrating a schematic configuration of an image processing apparatus according to the present embodiment. As shown in FIG. 1, the image processing apparatus 1 includes a feature space conversion unit 10, an RGB-YUV conversion unit 15, a setting unit 20A, a parameter output unit 30, an image processing unit 40, and a YUB-RGB conversion unit 50. In the present embodiment, the feature space conversion unit 10 and the setting unit 20A are examples of a setting unit. Hereinafter, each part will be described.

The RGB-YUV conversion unit 15 converts the input RGB signal into a YUV signal according to the following equation (1) and outputs the YUV signal to the image processing unit 40.

The feature space conversion unit 10 converts an image signal in the RGB color space (hereinafter, RGB signal) into an image signal in the YUV color space (hereinafter, YUV signal), and outputs the YUV signal to the setting unit 20A and the image processing unit 40. To do. As described above, the detection target object is an image of grass, trees, and the like, and has a green feature. When the detection target object has a feature in color, the feature of the detection target object can be obtained more accurately by converting the input signal into a signal value in a color space composed of components representing the color. In this embodiment, in order to obtain a UV signal that does not depend on luminance, the input signal is converted into a YUV signal in the YUV color space, but the input signal is converted into xyY, L ^* a ^* b ^* , You may make it convert into the signal of color spaces, such as L ^* u ^* v ^* . In the present embodiment, the YUV color space is an example of a feature space, and the YUV signal is an example of a feature amount that represents the size of a component in the feature space. That is, the input signal may be converted into a feature amount in a feature space made up of components representing the features of the detection target object in accordance with the features of the detection target object.

FIG. 2 is a circuit diagram for converting an input RGB signal into a YUV signal in the feature space conversion unit 10. The R, G, and B signals input to each multiplier shown in FIG. 2 are multiplied by coefficients (a ₁₁ to a ₃₃ ) corresponding to the multipliers, and added for each R, G, and B signal. . That is, the RGB signal is converted into a YUV signal by the conversion equation shown in the following equation (1-1).

Returning to FIG. 1, the description will be continued. The setting unit 20A includes a vector calculation processing unit 21, a minimum value calculation processing unit 22, and a numerical value limiting processing unit 23. The vector calculation processing unit 21 converts the signal value (Y _i , U _i , V _i ) of each pixel of the YUV signal output from the feature space conversion unit 10 into a plurality of scalar values on the UV plane with Y = 1. To do. In the UV plane shown in FIG. 3A, the closer to U = 0 and V = 0, the closer to the color (green) of the detection target object. In the present embodiment, the vector calculation processing unit 21 converts the signal value of each pixel in the YUV signal into scalar values S1 and S2 on the UV plane using the following equations (2) and (3).

In the UV plane shown in FIG. 3B, the closer to the V-axis direction from the broken line L1, the closer to green, and in the UV plane shown in FIG. 3C, the closer to the U-axis direction from the broken line L2, the closer to green. The vector (w11, w12, w13) in Expression (2) is determined according to the broken line L1 in FIG. 3B, and the vector (w21, w22, w23) in Expression (3) is determined according to the broken line L2 in FIG. 3C. These vectors are vectors representing the range of the detection target object in the color space. In this embodiment, for example, (w11, w12, w13) = (− 1.00, −0.25, 155) / 4, (w21, w22, w23) = (0.25, −1.00, 128). ) / 4.

That is, the scalar value S1 represents the color difference from the detection target object mainly as the U component (an example of the first component), and the scalar value S2 represents the color difference from the detection target object as V This mainly represents the component (an example of the second component). The scalar values S1 and S2 are set so as to monotonously increase as U = 0 and V = 0 in the UV plane.

Returning to Fig. 1, the explanation will be continued. The minimum value calculation processing unit 22 selects the scalar value of the minimum value (hereinafter referred to as the minimum value S) from the scalar values S1 and S2 calculated by the vector calculation processing unit 21 and outputs the selected value to the numerical value limiting processing unit 23.

The numerical value calculation processing unit 23 converts the minimum value S output from the minimum value calculation processing unit 22 into likelihood P. In this embodiment, the likelihood P is represented by a numerical value from the minimum value 0 to the maximum value 16. When the minimum value S is 16 or more, the numerical calculation processing unit 23 sets the maximum likelihood P = 16. When the minimum value S is 0 or less, the minimum value likelihood P = 0 is set. When the minimum value S is greater than 0 and less than 16, likelihood P = S is set.

FIG. 4 is a diagram illustrating a region on the UV plane corresponding to the scalar value (minimum value S). Curves L31 and L32 indicated by broken lines in the UV plane of FIG. 4 are obtained by connecting a line segment parallel to the line segment L1 shown in FIG. 3B and a line segment parallel to the line segment L2 shown in FIG. 3C. Yes, it represents the boundary of the color (green) of the detection target object. In other words, the region P1 is a region showing green and includes a color having a scalar value S of 16 or more. The region P2 is a region including a green component, and includes a color having a scalar value S greater than 0 and less than 16. Region P ₃ is not in the green region, include color scalar value S becomes 0 or less. Therefore, the color included in the region P ₁ is the highest likelihood, color included in the region P ₃ most likelihood is low. Color included in the region P ₂ takes a value between the maximum value and the minimum value of the likelihood.

Returning to Fig. 1, the explanation will be continued. The parameter output unit 30 calculates a parameter to be output to the image processing unit 40 based on the likelihood P output from the numerical value limiting processing unit 23 of the setting unit 20A. In the present embodiment, in the parameter output unit 30, a first parameter K1 is arbitrarily set as a parameter suitable for the detection target object, and a second parameter K2 is arbitrarily set as a parameter suitable for another object. The parameter output unit 30 calculates a parameter K (an example of an image processing parameter) obtained by mixing the first parameter K1 and the second parameter K2 at a ratio corresponding to the likelihood P. In this embodiment, K1 = 0 and K2 = 1 are set in advance.

FIG. 5 shows a circuit diagram for specifying the parameter K in the parameter output unit 30. As shown in FIG. 5, the parameter output unit 30 uses the output likelihood P, and calculates the mixture ratio α (= (P−Pmin) / (Pmax−Pmin)) of the first parameter K1 and the second parameter K2. calculate. The parameter K is expressed by the following formula (4).

The parameter output unit 30 calculates the value of the parameter K by substituting the calculated mixing ratio α into the above equation (4). When the likelihood P = Pmax = 16, α = 1 and the parameter K = K1. In this case, the likelihood P is the maximum value, and there is a high possibility that the pixel to be processed substantially matches the color of the detection target object. Therefore, a parameter K (K = K1 = 0) suitable for the detection target object is output for such a pixel.

When the likelihood P = Pmin = 0, α = 0 and the parameter K = K2. In this case, the likelihood P is the minimum value, and there is a high possibility that the pixel to be processed does not match the color of the detection target object. Therefore, a parameter K (K = K2 = 1) suitable for other objects is output for such a pixel.

For example, when the likelihood P is an intermediate value between the maximum value and the minimum value (= (Pmax + Pmin) / 2), α = 0.5 and the parameter K = (K1 + K2) / 2. In this case, the likelihood P is an intermediate value, and it is ambiguous to determine whether the processing target pixel matches the color of the detection target object. Therefore, a parameter K (K = 0.5) obtained by adding an intermediate value between the first parameter K1 and the second parameter K2 is output for such a pixel.

Returning to Fig. 1, the explanation will be continued. The image processing unit 40 includes a noise reduction processing unit 41. The image processing unit 40 uses the parameter K output from the parameter output unit 30 to perform noise reduction processing on the Y signal output from the RGB-YUV conversion unit 15 by the noise reduction processing unit 41 for each pixel. . The image processing unit 40 outputs the YUV signal for each pixel subjected to the noise reduction processing to the YUV-RGB conversion unit 50. The YUV-RGB conversion unit 50 converts the YUV signal output from the image processing unit 40 into an RGB signal by the following equation (1-2) and outputs the RGB signal.

Here, a configuration example of the noise reduction processing unit 41 in the image processing unit 40 will be described. FIG. 6 is a block diagram illustrating a schematic configuration of the noise reduction processing unit 41. As illustrated in FIG. 6, the noise reduction processing unit 41 includes a difference image generation unit 410, a parameter adjustment unit 416, and a synthesis calculation unit 417. The difference image generation unit 410 generates, for each pixel, a first difference signal (ΔY) obtained by performing a smoothing process on the input Y signal. The difference image generation unit 410 only needs to perform a smoothing process on the Y signal for each pixel and output the difference signal ΔY after the smoothing process. Here, an example of the difference image generation unit 410 will be described. FIG. 7 is a block diagram illustrating a configuration of the difference image generation unit 410. As illustrated in FIG. 7, the difference image generation unit 410 includes a contour direction estimation unit 411, a reference region load processing unit 412, a preprocessing unit 413, a direction evaluation unit 414, and a product-sum operation unit 415.

(Contour direction estimation unit 411)
The contour direction estimation unit 411 estimates the contour direction for each pixel based on the signal value (luminance value) for each pixel represented by the Y signal output from the RGB-YUV conversion unit 15. The contour direction is a direction orthogonal to the normal line of the contour line, that is, a tangential direction of the contour line. The contour line refers to a line indicating a space where the signal value is substantially constant, and may be a curve or a straight line. Therefore, the contour is not limited to a region where the signal value changes rapidly according to a change in position. The relationship between the contour line and the signal value corresponds to the relationship between the contour line and the altitude. The position of each pixel is given discretely or is affected by noise around the contour to be improved in this embodiment, such as jaggy, dot disturbance, mosquito noise, etc. In some cases, the direction of the contour cannot be determined using the line passing through as a line forming the contour. Here, it is assumed that the signal value is differentiable (that is, continuous) in the space representing the coordinates for each pixel. The contour direction estimation unit 411 calculates the contour direction θ based on, for example, Expression (5) based on the difference value of the signal value in the horizontal direction or the vertical direction for each pixel.

In Expression (5), the contour direction θ is a counterclockwise angle with respect to the horizontal direction (x direction). x and y are horizontal and vertical coordinates, respectively. Y (x, y) is a signal value at coordinates (x, y). That is, the contour direction θ is calculated as an angle that gives a tangent value obtained by dividing the partial differentiation of the signal value Y (x, y) in the x direction by the partial differentiation of the signal value Y (x, y) in the y direction. The Expression (5) can be derived from the relationship that the signal value Y (x, y) is constant even if the coordinates (x, y) are different. Here, Gx (x, y) and Gy (x, y) represent partial differentiation in the x direction and partial differentiation in the y direction of the signal value Y (x, y), respectively. In the following description, Gx (x, y) and Gy (x, y) may be referred to as x-direction partial differentiation and y-direction partial differentiation, respectively. In the following description, unless otherwise specified, the position (coordinates) of the pixel (i, j) indicates the barycentric point of the pixel. Further, the variable a at the position of the pixel is represented as a (i, j) or the like.

The contour direction estimation unit 411 uses, for example, equations (6) and (7), respectively, and the x-direction partial differential Gx (i, j) and y of the signal value Y (i, j) at each pixel (i, j). A directional partial differential Gy (i, j) is calculated.

In Expressions (6) and (7), i and j are integer values indicating indexes of the pixel of interest in the x direction and y direction, respectively. A pixel of interest is a pixel that attracts attention as a direct processing target. Wx (u ′, v ′) and Wy (u ′, v ′) indicate filter coefficients of the difference filter in the x direction and the y direction, respectively. u and v are integer values indicating indices of the reference pixel in the x and y directions, respectively. The reference pixel is a pixel that is within a range determined by a predetermined rule with the target pixel as a reference, and is a pixel that is referred to when processing the target pixel. The reference pixel includes a target pixel. u ′ and v ′ are integer values indicating indices in the x and y directions of the reference pixel when the target pixel is the origin. Therefore, u = i + u ′ and v = j + v ′.

For example, the difference filter described above may include filter coefficients Wx (u ′, u ′, v ′ for each of u ′, v ′ of 2n + 1 in the x direction and 2n + 1 in the y direction (total (2n + 1) · (2n + 1)) reference pixels. v ′) and Wy (u ′, v ′). In the following description, an area to which a reference pixel to which the filter coefficient is given belongs may be referred to as a reference area. n is an integer value greater than 1 (for example, 2). Here, the filter coefficients Wx (u ′, v ′) and Wy (u ′, v ′) are 1 with respect to the reference pixel in the positive direction based on the target pixel, and the same differential direction (x direction) as the target pixel. 0 for a reference pixel having a coordinate value of −1, and −1 for a reference pixel in the negative direction with reference to the target pixel. That is, the filter coefficient Wx (u ′, v ′) of the differential filter in the x direction is 1 (0 <u ′ ≦ n), 0 (u ′ = 0), −1 (0> u ′ ≧ −n). is there. The filter coefficients Wy (u ′, v ′) of the difference filter in the y direction are 1 (0 <v ′ ≦ n), 0 (v ′ = 0), and −1 (0> v ′ ≧ −n). Further, n is an integer value that is equal to or larger than the enlargement ratio of the image.

As a result, the signal value is smoothed in each of the positive direction and the negative direction with respect to the target pixel. Therefore, when estimating the direction of the contour, the influence of noise around the contour such as jaggy, mosquito noise, and dot interference It becomes difficult to receive. However, if reference pixels that are large and away from the target pixel are taken into account, a partial differential value that is originally a local value may not be calculated correctly. Therefore, n is set to a value smaller than a predetermined maximum value, for example, an integer value equal to the enlargement rate, an integer value obtained by rounding up the digits after the decimal point of the enlargement rate, or any of these integer values. It is determined that the value is larger.

The contour direction estimation unit 411 quantizes and quantizes the contour direction θ (i, j) calculated based on the calculated x-direction partial differential Gx (i, j) and y-direction partial differential Gy (i, j). A quantized contour direction D (i, j) representing the contour direction is calculated. The contour direction estimation unit 411 uses, for example, Expression (8) when calculating the quantized contour direction D (i, j).

In equation (8), round (...) Is a rounding function that gives an integer value obtained by rounding off decimal places of real numbers. N _d is a constant representing the number of quantized contour directions (number of quantized contour directions). The quantization contour direction number _Nd is, for example, any value between 8 and 32. That is, the quantized contour direction D (i, j) is represented by any integer from 0 to N _d−1 by rounding the value obtained by dividing the contour direction θ by the quantization interval by π / N _d. . As a result, the degree of freedom in the contour direction θ is restricted, and the processing load described later is reduced.

In order to avoid division by zero, the absolute value | Gx (i, j) | of the x-direction partial differential Gx (i, j) is smaller than a predetermined minute real value (for example, 10 ⁻⁶ ). Tan ⁻¹ is π / 2. Depending on the arithmetic processing system, there is a case where a tangent function having two arguments, Gx and Gy, is prepared in order to avoid the above-described error due to division and zero division. By using this, tan ⁻¹ is obtained. Also good.

The contour direction estimation unit 411 outputs quantized contour direction information representing the calculated quantized contour direction D (i, j) to the reference region load processing unit 412.

(Reference area load processing unit 412)
The reference region load processing unit 412 performs, for each pixel of interest (i, j), based on the quantized contour direction D (i, j) for each pixel represented by the quantized contour direction information input from the contour direction estimation unit 411. Define reference area load information. The reference area load information is a weighting factor R (D (i, j), u ′, v ′) for each reference pixel (u ′, v ′) belonging to a reference area centered on a certain target pixel (i, j). ). This weighting factor may be referred to as a reference area load. The size of the reference area in the reference area load processing unit 412 is determined in advance so as to be equal to the size of the reference area in the direction evaluation unit 414.

Based on the quantized contour direction D (i, j) for each pixel represented by the quantized contour direction information input from the contour direction estimating unit 411, the reference region load processing unit 412 performs weighting factor R (D (i, j ), U ′, v ′). The reference area load processing unit 412 is a reference in which (D (i, j), u ′, v ′) takes a value other than zero among the weighting factors R (D (i, j), u ′, v ′). A weighting factor R (D (i, j), u ′, v ′) for each pixel (u ′, v ′) is selected. Such a reference pixel is called a contour direction reference pixel because it is located in a contour direction or a direction approximate to the contour direction from the target pixel (i, j). The reference area load processing unit 412 generates reference area load information representing the weighting factor R (D (i, j), u ′, v ′) related to each contour direction reference pixel, and accumulates the generated reference area load information. The result is output to the sum calculation unit 415.

The reference region load processing unit 412 extracts quantized contour direction information representing the quantized contour direction D (u, v) related to each contour direction reference pixel from the input quantized contour direction information. The reference area load processing unit 412 outputs the extracted quantized contour direction information to the direction evaluation unit 414. The reference area load processing unit 412 extracts a luminance signal representing the signal value Y (u, v) related to each contour direction reference pixel from the Y signal output from the RGB-YUV conversion unit 15. The reference area load processing unit 412 outputs the extracted luminance signal to the preprocessing unit 413.

(Pre-processing unit 413)
For each pixel of interest (i, j), the pre-processing unit 413, for each pixel of interest (i, j), from the luminance signal input from the reference region load processing unit 412, each reference pixel belonging to the reference region (i, j) is the center. A luminance signal representing the signal value Y (u, v) of u, v) is extracted. The pre-processing unit 413 subtracts the signal value Y (i, j) of the target pixel from the signal value Y (u, v) of the reference signal represented by the extracted luminance signal, and obtains the difference signal value Y (u, v) − Y (i, j) is calculated. The preprocessing unit 413 generates a difference signal representing the calculated difference signal value, and outputs the generated difference signal to the product-sum operation unit 415.

(Direction evaluation unit 414)
The direction evaluation unit 414 calculates the direction evaluation value of each reference pixel belonging to the reference region centered on the target pixel for each target pixel based on the quantized contour direction D (i, j) for each pixel. Here, the direction evaluation unit 414 determines the difference between the quantization contour direction D (i, j) of the target pixel (i, j) and the quantization contour direction D (u, v) of the reference pixel (u, v). The direction evaluation value of the reference pixel is determined so that the smaller the value is, the larger the direction evaluation value is. For example, the direction evaluation unit 414 calculates the difference value ΔD = between the quantization contour direction D (i, j) for the pixel of interest (i, j) and the quantization contour direction D (u, v) for the reference pixel (u, v). D (u, v) −D (i, j) is calculated. Here, when the difference value ΔD is 0, that is, when D (u, v) and D (i, j) are equal, the direction evaluation value F (| ΔD |) is determined as the maximum value 1. When the difference value ΔD is not 0, that is, when D (u, v) and D (i, j) are not equal, the direction evaluation value F (| ΔD |) is determined as the minimum value 0.

The direction evaluation unit 414 determines that the quantization contour direction D (i, j) for the target pixel (i, j) approximates the quantization contour direction D (u, v) for the reference pixel (u, v), that is, the difference. The direction evaluation value F (ΔD) may be determined so as to increase as the absolute value | ΔD | of the value ΔD decreases. For example, the direction evaluation unit 414 calculates F (0) = 1, F (1) = 0.75, F (2) = 0.5, F (3) = 0.25, F (| ΔD |) = 0. (| ΔD |> 3).

However, when one of the quantization contour direction D (i, j) and the quantization contour direction D (u, v) is larger than N _d / 2 and the other is smaller than N _d / 2, Although the contour direction indicated by is approximated, the absolute value | ΔD | becomes large, so that an incorrect direction evaluation value F (ΔD) may be calculated. For example, when D (i, j) = 7 and D (u, v) = 0, | ΔD | = 7. However, the difference in the quantization contour direction is π / 8, and should be defined as | ΔD | = 1. Therefore, when one of the quantization contour direction D (i, j) and the quantization contour direction D (u, v) is larger than N _d / 2, the direction evaluation unit 414 determines the other quantization contour. A correction value is calculated by adding _Nd to the direction value. The direction evaluation unit 414 calculates an absolute value for a difference value between the calculated correction value and one quantized contour direction. The intended direction evaluation value is determined by using the absolute value thus calculated as | ΔD |. In the product-sum operation unit 415 described later, by using the above-described direction evaluation value F (| ΔD |), the contour direction of the pixel of interest (i, j) is different from the reference pixel (u, v) having a different contour direction. The effect can be ignored or neglected.

Also in the direction evaluation unit 414, the size of the reference region to which the reference pixel (u, v) belongs, that is, the number of pixels in the horizontal direction or the vertical direction may be 2n + 1 or larger. Note that the size of the reference region in the direction evaluation unit 414 may be different from the size of the reference region in the contour direction estimation unit 411. For example, the number of pixels in the horizontal direction and the vertical direction of the reference area in the direction evaluation unit 414 is 7 respectively, whereas the number of pixels in the horizontal direction and the vertical direction of the reference area in the contour direction estimation unit 411 is 5 respectively. There may be. The direction evaluation unit 414 outputs direction evaluation value information representing the direction evaluation value F (ΔD) of each reference pixel (u, v) to the product-sum operation unit 415 for each pixel of interest (i, j).

(Product-sum operation unit 415)
For each pixel of interest (i, j), the product-sum operation unit 415 receives the direction evaluation value information from the direction evaluation unit 414, the reference region load information from the reference region load processing unit 412, and the difference signal from the preprocessing unit 413. Entered. The product-sum operation unit 415 performs the direction evaluation value F (| ΔD |) represented by the direction evaluation value information, the reference region load, with respect to the difference signal value Y (u, v) −Y (i, j) represented by the difference signal. A smoothing difference value ΔY (i, j) is calculated by multiply-adding the reference region load R (D (i, j), u ′, v ′) represented by the information. The product-sum operation unit 415 uses, for example, Expression (9) when calculating the smoothed difference value ΔY (i, j).

In Equation (9), Rs (D (i, j)) represents a function (region selection function) for selecting a contour direction reference pixel among the reference pixels related to the target pixel (i, j). That is, u ′, v′εRs (D (i, j)) indicates a contour direction reference pixel. Expression (9) is obtained by calculating the direction evaluation value F (| ΔD |), the reference region load R (D (i, j), u ′, v ′), and the difference signal value Y (u, v) −Y represented by the difference signal. This indicates that the product of (i, j) is calculated for each reference pixel, and the sum between the reference pixels belonging to the reference area of the calculated product is calculated. Equation (9) represents that the smoothed difference value ΔY (i, j) is calculated by dividing the calculated sum by the reference area N (i, j) (the number of reference pixels belonging to the reference region). . The product-sum operation unit 415 generates a smoothed difference signal representing the calculated smoothed difference value ΔY (i, j), and synthesizes the generated smoothed difference signal (ΔY (i, j)) as the first difference signal. The result is output to the calculation unit 417.

As described above, the difference image generation unit 410 according to the present embodiment calculates the contour direction of each pixel, and is the contour direction of the target pixel or a direction that approximates the direction, and the contour direction is the same as or approximate to the target pixel. The target pixel is smoothed using the signal value relating to the reference pixel which is the direction in which the pixel is to be moved.

Returning to FIG. 6, the description will be continued. The parameter adjustment unit 416 multiplies the first difference signal (ΔY (i, j)) for each pixel output from the product-sum operation unit 415 of the difference image generation unit 410 by the parameter K output from the parameter output unit 30. The second difference signal (K · ΔY (i, j)) is output to the composition calculation unit 417.

The composition calculation unit 417 outputs, for each pixel, a Y ′ signal obtained by adding the second difference signal output from the parameter adjustment unit 416 and the Y signal output from the RGB-YUV conversion unit 15. When the parameter K = 0, a Y signal equivalent to the luminance signal of the original image is output, and the effect of the noise reduction process is minimized. When the parameter K = 1, a Y signal obtained by adding the smoothed difference signal to the luminance signal of the original image is output, and the effect of the noise reduction processing is maximized. That is, noise reduction is suppressed for fine texture portions such as trees in the input image signal, and noise is reduced for other object portions. Further, when the parameter K is 0 <K <1, the second difference signal corresponding to the likelihood for the detection target object is added to the Y signal, so that the parameter K is rapidly switched between 1 and 0. Artifacts due to are reduced.

In the first embodiment described above, the scalar value of the signal value of each pixel in the image signal is determined based on a plurality of components constituting the color of the detection target object in the color space (UV plane) according to the characteristics of the detection target object. Ask. Then, a likelihood corresponding to the magnitude of the scalar value is set for each pixel, and image processing is performed using a parameter corresponding to the likelihood of the pixel. With this configuration, even if a pixel is vaguely determined whether or not it substantially matches the detection target object, image processing is performed using parameters according to the likelihood of the pixel. The image processing suitable for is performed. As a result, it is possible to reduce artifacts due to image processing as compared with the case where image processing is performed using uniform parameters for the entire image.

Second Embodiment
In the first embodiment, the signal value of the pixel having the color (feature) of the detection target object is expressed as a scalar value on the UV plane, and the likelihood for the detection target object is obtained based on the scalar value. Hereinafter, another method for obtaining the likelihood for the detection target object will be described.

FIG. 8 is a block diagram showing a schematic configuration of the image processing apparatus according to the present embodiment. The image processing device 1a is different from the setting unit 20A of the first embodiment in that the setting unit 20B includes a distance calculation processing unit 24 and a maximum value calculation processing unit 25. Hereinafter, each part of the setting unit 20B will be described.

The distance calculation processing unit 24 calculates the distance R between the reference point in the plurality of regions on the UV plane corresponding to the color (feature) of the detection target object and the position indicated by the pixel signal value. The distance calculation processing unit 24 converts the value of the distance R so that the value becomes larger as the distance R to the reference point is smaller. In the present embodiment, the distance R is an example of a scalar value for the color of the detection target object.

In this embodiment, for example, as shown in FIGS. 9A to 9C, substantially elliptical regions f1, f2, and f3 on the UV plane are set as a color range (here, green) corresponding to the detection target object. . In this example, the diameter SUn in the U-axis direction and the diameter S _{Vn in the} V-axis direction of the region f1 are (S _U1 , S _V1 ) = (20, 40), and the center point q1 is the reference of the region f1 Is a point. The coordinates (U ₁ , V ₁ ) of the reference point q1 on the UV plane are (16, 32). The diameter of the region f2 is (S _U2 , S _V2 ) = (10, 20), and the center point q2 is the reference point of the region f2. The coordinates of the reference point q2 on the UV plane are (U ₂ , V ₂ ) = (32, 64). The diameter of the region f3 is (S _U3 , S _V3 ) = (5, 10), and the center point q3 is the reference point of the region f3. The coordinates of the reference point q3 on the UV plane are (U ₃ , V ₃ ) = (16, 32). The larger the diameters S _Un and S _{Vn of the} regions f1 to f3, the larger the regions, and the wider the regions that are determined to be closer to the reference point.

The distance calculation processing unit 24 uses the following equation (10) to calculate the distance between the signal value (U, V) of each pixel in the YUV signal output from the feature space conversion unit 10 and the reference points q1, q2, q3. calculate.

The distance calculation processing unit 24 converts the distance Rn between the signal value (U, V) of each pixel and the reference points q1, q2, q3 by the following formula (11) or formula (12). Expressions (11) and (12) are functions that increase as the distance Rn decreases.

The maximum value calculation processing unit 25 selects the maximum value distance Rn ′ from the distances R1 ′, R2 ′, R3 ′ calculated by the distance calculation processing unit 24, and outputs it to the numerical value limiting processing unit 23. By selecting the maximum distance Rn ′, the logical sum of the regions f1, f2, and f3 can be obtained as a result. As illustrated in FIG. 10, the region f obtained by superimposing the regions f1, f2, and f3 is a region having a higher likelihood for the detection target object than the region other than the region f.

As in the first embodiment, the numerical value limit processing unit 23 sets the distance Rn ′ output from the maximum value calculation processing unit 25 to be within the range between the predetermined maximum value Pmax and minimum value Pmin. Convert to likelihood P. That is, when the distance Rn ′ is equal to or greater than a predetermined maximum value, the distance Rn ′ = Pmax = P is converted, and when the distance Rn ′ is a predetermined minimum value, the distance Rn ′ = Convert to Pmin = P. When the distance Rn ′ is greater than the minimum value and less than the maximum value, the distance Rn ′ = P is converted.

In the second embodiment described above, the reference points defined in a plurality of areas indicating the predetermined color (feature) range of the detection target object in the color space (UV plane), and the positions indicated by the pixel signal values in the color space The likelihood with respect to the detection target object is represented by a multi-value based on the distance between the two. With this configuration, even if the signal value of the pixel has the characteristics of the detection target object and it is unclear whether it matches the detection target object, the parameter corresponding to the likelihood is used. Image processing. As a result, appropriate image processing corresponding to the object is performed, and artifacts due to image processing can be reduced as compared with a case where image processing is performed using uniform parameters for the entire image.

As mentioned above, although embodiment of this invention was described, embodiment mentioned above is only the illustration for implementing this invention. Therefore, the present invention is not limited to the above-described embodiment, and can be implemented by appropriately modifying the above-described embodiment without departing from the spirit thereof. Hereinafter, modifications of the present invention will be described.

<Modification>

(1) In the first embodiment described above, the example in which the image processing unit 40 performs the smoothing process on the signal value of each pixel using the parameter K by the noise reduction processing unit 41 has been described. Sharpening processing may be performed using a parameter corresponding to the likelihood. FIG. 11 is a block diagram showing a configuration of an image processing apparatus according to this modification. As shown in FIG. 11, the image processing apparatus 1b includes a parameter output unit 30A and an image processing unit 40A instead of the parameter output unit 30 and the image processing unit 40 in the first embodiment.

The parameter output unit 30A obtains the mixing ratio α corresponding to the likelihood P output from the setting unit 20A, as in the first embodiment. Further, in the same manner as the above-described equation (4), a parameter K ′ (an example of an image processing parameter) is calculated and output to the image processing unit 40A. In the present modification, sharpening processing is performed as image processing. Therefore, the sharpening effect is enhanced for pixels whose detection target object includes fine textures such as trees. Therefore, in the parameter output unit 30A, K1 ′ = 1 is set as the first parameter K1 ′ used for the sharpening process for the detection target object, and K2 ′ = 0 is set as the second parameter K2 ′ used for the sharpening process for the other object in advance. To do.

When the likelihood P = Pmax = 16, α = 1 and the parameter K ′ = K1 ′. In this case, the likelihood P is the maximum value, and there is a high possibility that the pixel to be processed substantially matches the color of the detection target object. Therefore, a parameter K ′ (K1 ′ = 1) suitable for the sharpening process of the detection target object is output for such a pixel. When the likelihood P = Pmin = 0, α = 0 and the parameter K ′ = K2 ′. In this case, the likelihood P is the minimum value, and there is a high possibility that the pixel to be processed does not match the color of the detection target object. Therefore, a parameter K ′ (K2 ′ = 0) suitable for other objects is output for such a pixel. For example, when the likelihood P is an intermediate value between the maximum value and the minimum value (= (Pmax + Pmin) / 2), α = 0.5 and the parameter K ′ = (K1 ′ + K2 ′) / 2. . In this case, the likelihood P is an intermediate value, and it is ambiguous to determine whether the processing target pixel matches the color of the detection target object. Therefore, for such a pixel, a parameter K ′ (= 0.5) obtained by adding an intermediate value between the first parameter K1 ′ and the second parameter K2 ′ is output.

The image processing unit 40A includes a sharpening processing unit 42. FIG. 12 is a diagram illustrating a schematic configuration of the sharpening processing unit 42. As illustrated in FIG. 12, the sharpening processing unit 42 includes a high frequency component output unit 420, a parameter adjustment unit 428, and a synthesis calculation unit 429. The high frequency component output unit 420 performs a sharpening process on the Y signal output from the RGB-YUV conversion unit 15 and outputs a high frequency component value NL _2D of the Y signal.

FIG. 13 is a block diagram showing an example of the high-frequency component output unit 420. As illustrated in FIG. 13, the high frequency component output unit 420 includes a contour direction estimation unit 411, a low-pass filter unit 420 a, and a high frequency extension unit 427. The low-pass filter unit 420 a includes a preprocessing unit 413, a direction evaluation unit 414, a reference region load processing unit 412, a product-sum operation unit 425, and a synthesis operation unit 426. In FIG. 13, the same reference numerals as those in the first embodiment are assigned to the same configurations as those in the first embodiment. Hereinafter, description of the same configuration as in the first embodiment will be omitted, and the product-sum operation unit 425, the synthesis operation unit 426, and the high-frequency extension unit 427 will be described.

(Product-sum operation unit 425)
For each pixel of interest (i, j), the product-sum operation unit 425 receives the direction evaluation value information from the direction evaluation unit 414, the reference region load information from the reference region load processing unit 412, and the luminance signal from the preprocessing unit 413. Entered. The product-sum operation unit 425 includes a direction evaluation value F (| ΔD |) represented by the direction evaluation value information, a reference region load R (D (i, j), u ′, v ′) represented by the reference region load information, and a luminance signal. The product-sum value S (i, j) is calculated using the equation (13), for example, based on the signal value Y (u, v) represented by.

Equation (13) is the product of the direction evaluation value F (| ΔD |), the reference region load R (D (i, j), u ′, v ′), and the signal value Y (u, v) represented by the luminance signal. The calculation is performed for each reference pixel, and the sum between reference pixels belonging to the reference area of the calculated product is calculated as a product sum value S (i, j). That is, the equation (13) indicates that the product sum value S (i, j) is the product of the direction evaluation value F (| ΔD |) and the reference region load R (D (i, j), u ′, v ′). Can be considered to be calculated by weighted addition of the signal value Y (u, v), using as a weighting factor. The product of the direction evaluation value F (| ΔD |) and the reference region load R (D (i, j), u ′, v ′) may be referred to as a direction evaluation region load. The product-sum operation unit 425 is based on the direction evaluation value F (| ΔD |) represented by the direction evaluation value information and the reference region load R (D (i, j), u ′, v ′) represented by the reference region load information. For example, the load area C (i, j) is calculated using Equation (14).

Expression (14) calculates the product of the direction evaluation value F (| ΔD |) and the reference area load R (D (i, j), u ′, v ′) for each reference pixel, and the reference area of the calculated product This represents that the sum between reference pixels belonging to is calculated as a load area C (i, j). That is, the load area C (i, j) is a value obtained by weighting the reference region load R (D (i, j), u ′, v ′) with the direction evaluation value F (| ΔD |) for each reference pixel. That is, it represents the number of reference pixels that are substantially referred to in the product-sum operation of Expression (13). In other words, equation (14) indicates that the load area C (i, j) is calculated by taking the sum of the above-described direction evaluation region loads within the reference region. In addition, the product-sum operation unit 425 calculates the sum between reference pixels belonging to the reference region of the reference region load R (D (i, j), u ′, v ′) represented by the reference region load information as a reference area N (i, j). The reference area N (i, j) represents the number of reference pixels that are referred to for nominal purposes in the product-sum operation of Equation (13). The product-sum operation unit 425 includes product-sum value information representing the product-sum value S (i, j) calculated for each pixel of interest (i, j), load area information representing the load area C (i, j), and reference area. Reference area information representing N (i, j) is output to the composition calculation unit 426.

(Composition operation unit 426)
The sum calculation unit 426 receives product sum value information, load area information, and reference area information from the product sum calculation unit 425. The synthesis calculation unit 426 divides the product sum value S (i, j) represented by the product sum value information by the load area C (i, j) represented by the load area information to thereby smooth the direction smoothing value Y ′ (i, j). Is calculated. That is, the calculated direction smoothing value Y ′ (i, j) is a reference pixel in the quantization contour direction of the pixel of interest (i, j) or a direction approximating the direction, and the contour direction is of interest. It represents a signal value smoothed between reference pixels that are equal to or approximate to the pixel contour direction.

The composition calculation unit 426 calculates the mixture ratio w (i, j) by dividing the load area C (i, j) by the reference area N (i, j) represented by the reference area information. The mixture ratio w (i, j) approximates whether the contour direction is equal to the contour direction of the pixel of interest among the reference pixels in the quantization contour direction of the pixel of interest (i, j) or the direction approximating the direction. This represents the number ratio of reference pixels.

The composition calculation unit 426 converts the direction smoothed value Y ′ (i, j) and the signal value Y (i, j) represented by the input luminance signal (Y signal) into the mixing ratios w (i, j) and ( 1-w (i, j)) is added with weight (addition operation) to calculate the low-pass signal value Y ″ (i, j). This weighted addition is expressed by equation (15).

The synthesis calculation unit 426 outputs the luminance signal Y ″ representing the calculated low-pass signal value Y ″ (i, j) to the high frequency extension unit 427 and the synthesis calculation unit 429. FIG. 14 is a block diagram illustrating a configuration of the high-frequency extension unit 427. As illustrated in FIG. 14, the high-frequency extension unit 427 includes a two-dimensional high-pass filter unit 381 and a nonlinear calculation unit 382.

(Two-dimensional high-pass filter unit 381)
The two-dimensional high-pass filter unit 381 receives the luminance signal Y ″ from the synthesis calculation unit 426 and the quantized contour direction information from the contour direction estimation unit 411. The two-dimensional high-pass filter unit 381 has a quantized contour direction D (i, j) represented by the quantized contour direction information with respect to the low-pass signal value Y ″ (i, j) represented by the luminance signal Y ″. calculating a contour direction component signal W _2D representing the high frequency component according to. The two-dimensional high-pass filter unit 381 outputs the calculated contour direction component signal _W2D to the non-linear operation unit 382.

FIG. 15 is a schematic diagram illustrating the configuration of the two-dimensional high-pass filter unit 381. The two-dimensional high-pass filter unit 381 includes a delay memory 3811, a direction selection unit 3812, a multiplication unit 3813, a filter coefficient memory 3814, and a synthesis calculation unit 3815. The delay memory 3811 includes delay elements 3811-1 to 3811-2n + 1 that delay the input signal by 2n + 1 W _x samples. Delay elements 2811-v-1 ~ 2811- v-2n + 1 , respectively an input signal _{W x -2n, W x -2n +} 1, ..., W x samples delayed by 2n + 1 samples direction selection unit delay signal including a signal value of It outputs to 3812.

The delay elements 3811-1 to 3811-2n + 1 are connected in series. The low-pass signal value Y ″ (i, j) represented by the luminance signal Y ″ is input to one end of the delay element 3811-1, and the other end of the delay element 3811-1 is delayed by W _x samples. The signal is output to one end of the delay element 3811-2. One end of each of the delay elements 3811-2 to 3811-2n + 1 receives a delay signal delayed by W _x to 2n · W _x samples from the other end of each of the delay elements 3811-1 to 3811-2n. The other end of the delay element 3811-2n + 1 outputs a delayed signal delayed by (2n + 1) · W _x samples to the direction selection unit 3812. Accordingly, the signal value representing the luminance signal Y ″ and the signal values of (2n + 1) · (2n + 1) pixels adjacent to each other in the horizontal direction and the vertical direction around the target pixel are output to the direction selection unit 3812. Pixels corresponding to these signal values are reference pixels belonging to a reference region centered on the target pixel.

Based on the quantized contour direction D (i, j) for each pixel represented by the quantized contour direction information input from the contour direction estimating unit 411, the direction selecting unit 3812 starts the quantization contour from the target pixel (i, j). A reference pixel (u ′, v ′) in the direction D or a direction approximating that direction is selected. The selected reference pixel is, for example, a reference pixel that satisfies the following condition. (1) A line segment extending in the quantization contour direction from the center of the target pixel (i, j) is a reference pixel (u ′, v ′) that passes through the region. (2) The horizontal or vertical length through which the line segment passes is longer than 0.5 pixels or 0.5 pixels. (3) One reference pixel is selected for each of 2n + 1 coordinates in at least one of the horizontal direction and the vertical direction. Hereinafter, the selected reference pixel may be referred to as a selected reference pixel.

The direction selection unit 3812 determines whether the direction in which one reference pixel is selected for each of the 2n + 1 coordinates (hereinafter sometimes referred to as a selected coordinate direction) is the horizontal direction, the vertical direction, or both. For example, the direction selection unit 3812 outputs the signal values related to 2n + 1 selected reference pixels to the multiplication unit 3813 in descending order of the index in the selected coordinate direction. Note that the direction selection unit 3812 includes a storage unit in which selection reference pixel information representing a selection reference pixel for each quantization contour direction is stored in advance, and a quantization contour direction D (i, j) of the pixel of interest (i, j). The selected reference pixel information corresponding to () may be selected. Accordingly, the direction selection unit 3812 outputs the signal value related to the selected reference pixel represented by the selected selected reference pixel information to the multiplication unit 3813.

The multiplication unit 3813 includes 2n + 1 multipliers 3813-1 to 3813-2n + 1. Multipliers 3813-1 to 3813-2 n + 1 multiply each signal value input from direction selection unit 3812 by each filter coefficient read from filter coefficient memory 3814, and output the multiplied value to synthesis operation unit 3815. Here, the signal values are inputted so that the order of the multipliers 3813-1 to 3813-2n + 1 matches the order of the signal values inputted to each of them (descending order of the index in the selected coordinate direction). The filter coefficient memory 3814 stores 2n + 1 filter coefficients a _D−n , a _{D−n + 1} , and a _{D + n} used in the multipliers 3813-1 to 3813-2 _{n + 1} in advance. The filter coefficients a _D−n , a _{D−n + 1} , to a _{D + n} are high-pass filter coefficients that realize a high-pass filter by a product-sum operation with a signal value. The high pass filter coefficients, the filter coefficients of the above _{a D-n, a D-} n + 1, has a high pass characteristic similar to _{~ a D} + _n, may be a value having a property of blocking the DC component .

Combining unit 3815 adds the 2n + 1 multiplications value input from the multiplication unit 3813, and generates a contour direction component signal W _2D having a signal value which is their total. The composition calculation unit 3815 outputs the generated contour direction component signal _W2D to the nonlinear calculation unit 382.

Nonlinear operator 382 performs a nonlinear operation on the signal values representing the direction component signal W _2D input from two-dimensional high-pass filter 381. The nonlinear calculation unit 382 outputs the high frequency component value NL _2D represented by the calculated nonlinear output value to the parameter adjustment unit 428.

(Nonlinear calculation unit 382)
FIG. 16 is a block diagram illustrating a configuration of the nonlinear arithmetic unit 382. The nonlinear calculation unit 382 includes an absolute value calculation unit 2821-A, a power calculation unit 2822-A, a filter coefficient memory 2823-A, a multiplication unit 2824-A, a synthesis calculation unit 2825-A, a code detection unit 2826-A, and a multiplication unit. 2827-A.

The non-linear operation unit 382 generates an odd function sgn | W | · (c ₁ · | W |) of l (el) order (where l is an integer greater than 1) with respect to the input signal value W (= W _2D ). + C ₂ · | W | ² +... + C _l · | W | ^l ) is output as the nonlinear output value NLA. c ₁ , c ₂ ,..., c ₁ are 1, 2,.

The absolute value calculation unit 2821-A calculates the absolute value | W | of the signal value W indicated by the input direction component signal, and outputs the calculated absolute value | W | to the power operation unit 2822-A. The power calculation unit 2822-A includes l-1 multipliers 2822-A-2 to 2822-A-l, and multiplies the absolute value | W | input from the absolute value calculation unit 2821-A by the multiplier 2824-. Output to A.

Multiplier 2822-A-2 calculates absolute square value | W | ² by multiplying absolute values | W | input from absolute value calculation unit 2821-A. The multiplier 2822-A-2 outputs the calculated absolute square value | W | ² to the multiplier 2822-A-3 and the multiplier 2824-A.

Multipliers 2822-A-3 to 2822-A-1-1 are absolute square values | W | ² to absolute l-2 power values inputted from multipliers 2822-A-2 to 2822-A-1-2. | ^{W |} to ^l-2, the absolute value calculating unit 2821-a is input from the absolute value | a ^{l-1 |} W | absolute cubed value by multiplying the | W ^{| 3} ~ absolute l-1 squared value | ^W Calculate each. The multipliers 2822-A-3 to 2822-Al-1 use the calculated absolute cube values | W | ³ to absolute l-1 power values | W | ^l-1 respectively as multipliers 2822-A-4 to 2822-. The data is output to A-l and the multiplier 2824-A. The multiplier 2822-A-l receives the absolute l-1 power value | W | ^l-1 input from the multiplier 2822-A ^-1 and the absolute value | W input from the absolute value calculation unit 2821-A. | multiplied by the absolute l square value ^| W | to calculate the ^l. The multiplier 2822-A-l outputs the calculated absolute l-th power value | W | ^l to the multiplication unit 2824-A.

The filter coefficient memory 2823-A includes l storage elements 2823-A-1 to 2823-A-l. The storage device 2823-A-1 ~ 2823- A-l, respectively 1 ~ l following coefficients _c 1 ~ _{c l} are stored.

The multiplication unit 2824-A includes l multipliers 2824-A-1 to 2824-A-l. Multipliers 2824-A-1 ~ 2824- A-l , the absolute value is input from the respective power calculating section 2822-A | W | ~ absolute l squared | W ^| to ^l, the memory element 2823-A-1 ~ 2823-a-l to by multiplying the stored 1 ~ l following coefficients _c 1 ~ _{c l} to calculate a multiplication value.

Multipliers 2824-A-1 to 2824-A-l output the calculated multiplication values to the synthesis operation unit 2825-A, respectively. The synthesis operation unit 2825-A adds the multiplication values respectively input from the multipliers 2824-A-1 to 2824-A-1 to calculate a synthesis value. The combination calculation unit 2825-A outputs the calculated combination value to the multiplication unit 2827-A.

The sign detection unit 2826-A detects the sign of the signal value W indicated by the direction component signal input from the two-dimensional high-pass filter 381, that is, positive / negative. When the signal value is smaller than 0, the code detection unit 2826-A outputs −1 as a code value to the multiplication unit 2827-A. When the signal value is 0 or larger than 0, the code detection unit 2826-A outputs 1 as the code value to the multiplication unit 2827-A.

The multiplication unit 2827-A multiplies the combined value input from the combining calculation unit 2825-A and the code value input from the code detection unit 2826-A to calculate the high frequency component value NL _2D . The multiplier 2827-A outputs the calculated high frequency component value to the parameter adjuster 428 (see FIG. 12). The non-linear operation unit 382 having the above-described configuration has a relatively large circuit scale, but can adjust the output high-frequency component value using a small number of coefficients. If the coefficient values c ₁ to c _l−1 other than the highest order l are 0, the configuration related to the product-sum operation of these orders may be omitted. The configurations that can be omitted are the storage elements 2823-A-1 to 2823-A-1 and the multipliers 2824-A-1 to 2824-A-1 in the nonlinear arithmetic unit 382. For example, in the case of f (W) = sgn (W) | W | ² , the storage element 2823-A-1 and the multiplier 2824-A-1 may be omitted.

Returning to FIG. 12, the description will be continued. The parameter adjustment unit 428 multiplies the high frequency component value NL _2D (first high frequency component value) output from the high frequency component output unit 420 by the parameter K output from the parameter output unit 30 (K × NL). _2D ) (hereinafter referred to as the second high-frequency component value) is output to the composition calculation unit 429.

(Combining operation unit 429)
The synthesis calculation unit 429 uses the second high-frequency component value output from the parameter adjustment unit 428 and the low-pass signal value Y ″ (i, j) output from the synthesis calculation unit 426 of the high-frequency component output unit 420. Addition (combination) is performed to calculate the high-frequency extension signal value Z (i, j). The synthesis calculation unit 429 generates a luminance signal Z representing the calculated high-frequency extension signal value Z (i, j), updates the luminance signal Y input from the feature space conversion unit 10 to the luminance signal Z, and outputs the luminance signal Z And an image signal including the color difference signals U and V are output.

In the case of the parameter K ′ = 1, that is, for the pixel that substantially matches the detection target object, the luminance signal Z in which the high frequency component is combined with the low-pass signal is output. Further, when the parameter K ′ = 0, that is, for a pixel that does not match the detection target object, the luminance signal Z in which the high frequency component is not combined with the low-pass signal is output. As a result, in the input image signal, fine texture parts such as trees are sharpened more effectively than other parts, and fine lines such as trees are emphasized, and the state where trees are thicker is more appropriate. Expressed. When the parameter K ′ is 0 <K ′ <1, the luminance signal Z in which the high-frequency component value corresponding to the likelihood for the detection target object is combined with the low-pass signal is output. Artifacts due to the rapid switching of 'to 1 and 0 are reduced.

(2) In the first embodiment described above, noise reduction processing is performed by the noise reduction processing unit 41 in the image processing unit 40, and in the second embodiment, sharpening processing is performed by the sharpening processing unit 42 in the image processing unit 40A. However, the image processing unit may be configured to perform noise reduction processing and sharpening processing. For example, the configuration of the image processing unit 40 in the first embodiment is configured to include the high-frequency extension unit 427 of the sharpening processing unit 42 in the second embodiment. In this case, the high frequency extension unit 427 uses the luminance signal Y ′ (i, j) output from the noise reduction processing unit 41 and the quantized contour direction D (i, j) output from the contour direction estimation unit 411. To calculate the high-frequency component value.

(3) In the first and second embodiments described above, the likelihood based on the scalar value in the UV plane based on the signal value of the pixel or the distance between the color of the pixel in the UV plane and the color reference point of the detection target object. Although the example of adjusting the image processing parameters according to the degree has been described, the adjustment may be performed using the number of taps of the filter used for the noise reduction processing or the sharpening processing, the coefficient, the overshoot limit value, or the like. Good.

(4) In the first and second embodiments described above, the pixel having the detection target object is specified in the image signal, and the pixel signal value is converted into the scalar value of the pixel on the UV plane and the distance to the detection target object. You may make it perform the process which sets likelihood using a fuzzy inference apparatus.

(5) In the first and second embodiments described above, the color space has been described as an example of the feature space of the detection target object. However, a frequency space composed of frequency components (low frequency components and high frequency components) may be used. . In this case, when the detection target object has a specific frequency component, the likelihood for each pixel may be set based on the frequency component of the signal value of each pixel in the input signal and the specific frequency component. . For example, in the setting unit, the absolute value Gx of the difference between the luminance value Y (i, j) of the pixel to be processed and the luminance value Y (i + 1, j) of the pixel located in the horizontal direction of the pixel, and the pixel The absolute value Gy of the difference between the luminance value Y (i, j + 1) and the luminance value Y (i, j) of the pixel located in the vertical direction is obtained, and the result of adding the absolute values Gx and Gy (hereinafter, absolute The frequency component of the pixel to be processed (an example of a scalar value) may be used. In this case, the frequency component of the pixel is a feature value in the feature space and represents a scalar value. The setting unit may set the likelihood for the detection target object according to the absolute value G. For example, the likelihood may be set for each pixel so that the likelihood increases as the difference between the absolute value G and the threshold value increases. Note that an FIR filter may be used as a difference circuit that extracts a difference between luminance values of pixels.

As another example, the setting unit obtains the degree of coincidence (an example of a scalar value) between the frequency pattern of the detection target object and the frequency pattern of the signal value of each pixel using a pattern matching circuit. The likelihood corresponding to the degree may be set.

(6) In the first embodiment and the second embodiment described above, a color space is described as an example of the feature space of the detection target object. In the modification (5), a frequency space is used as the feature space of the detection target object. As described in the example, the feature space is not limited to this as long as it is a multidimensional scalar field composed of components representing the features of the detection target object. For example, the setting unit obtains a scalar value for the color of the detection target object for each pixel in the same manner as in the first and second embodiments described above, and each pixel in the same manner as in the modification (5) described above. Is obtained as a scalar value. In this example, the feature space is composed of color and frequency components representing the features of the detection target object. The setting unit sets the likelihood based on the scalar value for the color for each pixel in the same manner as in the first or second embodiment. When the frequency component that is the scalar value is equal to or greater than the threshold, the likelihood is set. May be adjusted according to the difference from the threshold. Even if the determination of whether or not the pixel to be processed is a detection target object is vague based only on the color of the detection target object, this configuration appropriately determines whether or not it is a detection target object. It becomes possible to judge.

In the above example, the frequency component of the pixel is a feature value and a scalar value. For example, a feature value (UV value, frequency component) in a feature space including a YUV component and a frequency (F) component is used. May be obtained as a scalar value. In this case, each scalar value (S1 ', S2', S3 ') is obtained from the feature quantity of each pixel by the following equations (16), (17), and (18). In this way, weights (vectors) for converting feature values into scalar values may be set so that each feature value of a pixel becomes a scalar value, or as in the first and second embodiments. The weight (vector) may be set based on the feature amount of the detection target object in the feature space.

The present invention can be industrially used as an image processing apparatus mounted on a display device such as a television or a PC.

Claims (9)

1. An image processing unit that performs image processing on the signal value of each pixel in the image signal using an input image processing parameter;
   A setting unit that sets the likelihood of the pixel with respect to a predetermined object using the signal value of each pixel in the image signal;
   A parameter output unit that calculates and outputs the image processing parameter based on the likelihood set by the setting unit;
   An image processing apparatus comprising:
2. The parameter output unit uses a predetermined first parameter for use in the image processing for the object and a predetermined second parameter for use in the image processing for another object as the likelihood. The image processing apparatus according to claim 1, wherein the image processing parameter is calculated by mixing with a mixing ratio based on the mixing ratio.
3. The setting unit converts a signal value of each pixel into a feature amount representing the size of the component in a feature space including a plurality of components representing the feature of the object, and converts a scalar value based on the feature amount. The image processing apparatus according to claim 1, wherein the likelihood is determined and the likelihood is set based on the scalar value.
4. 4. The image processing apparatus according to claim 3, wherein the feature space is a color space including a first component and a second component constituting a color.
5. The image processing device according to claim 3 or 4, wherein the feature space has a frequency space including a predetermined frequency component included in the object.
6. The setting unit converts the signal value of each pixel into the feature amount in the color space, and mainly uses the first component and the second component by using a vector representing the range of the object in the feature space. The image processing apparatus according to claim 4, wherein the represented scalar value is obtained.
7. The setting unit converts the signal value of each pixel into the feature amount in the color space, and each reference point defined in a plurality of regions representing the range of the object in the color space; the feature amount; The image processing apparatus according to claim 4, wherein the distance is calculated as the scalar value.
8. The image processing apparatus according to claim 5, wherein the setting unit obtains a frequency component of the pixel in the frequency space from a signal value of each pixel, and uses the obtained frequency component as the scalar value of the pixel.
9. The image processing unit includes at least one of first image processing that performs smoothing processing on the signal value in the image signal and second image processing that performs sharpening processing on the signal value in the image signal as the image processing. The image processing apparatus according to any one of claims 1 to 8, wherein:

Images