JP2008171059A - Image processing circuit, semiconductor device, and image processor - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing circuit for properly removing noise without damaging sharpness of an output image in the case of generating a desired output image by applying luminance conversion processing to an input image. <P>SOLUTION: This image processing circuit is provided with: a luminance conversion processing part 11 for applying luminance conversion processing to each of pixels configuring an input image; a noise removal processing part 13 for applying noise removal processing to each of those pixels configuring the output image of the luminance conversion processing part 11; and a filter coefficient setting part 14 for calculating a difference value between the luminance value of each of those pixels configuring the input image and the luminance value of each of those pixels configuring the output image of the luminance conversion processing part 11, and for setting the filter coefficients of noise removal processing according to the calculation result. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to an image processing circuit that generates a desired output image by performing luminance conversion processing (luminance dynamic range correction processing) on an input image, a semiconductor device formed by integrating the image processing circuit, and an image processing device using the image processing circuit. Is.

  Conventionally, luminance conversion processing is generally used as one of image processing for making an input image look more beautiful. In this luminance conversion process, for example, a luminance conversion coefficient is calculated based on the luminance distribution in the entire image, and the luminance conversion is performed by multiplying each pixel by the luminance conversion coefficient.

  More specifically, first, a luminance distribution (frequency of each luminance range, that is, distribution of the number of pixels belonging to each predetermined luminance range) for the entire input image is obtained. Based on this luminance distribution, for each luminance range in the input image, a higher luminance frequency is assigned to the output image, while a lower luminance frequency is assigned to the output image. As described above, the luminance conversion coefficient is calculated. Therefore, the luminance conversion coefficient is a separate value for each luminance or for each predetermined luminance range.

  As a result, the contrast (brightness difference) becomes more ambiguous in the part of the luminance range that is less frequent in the input image, but the contrast is clearer in the part of the luminance range that is frequent in the input image. It becomes. Therefore, when viewed as an entire image, the contrast can be made clearer than at the time of input.

  In addition, patent documents 1-4 etc. can be mentioned as a prior art relevant to the above.

Further, Patent Document 5 discloses a means for extracting a luminance component of an image, a creating means for creating a blurred image based on the luminance component, a correcting means for correcting a light / dark difference of the image according to the blurred image, There has been disclosed and proposed an image processing apparatus comprising a removing unit that removes noise from the corrected image in accordance with a correction amount of the contrast difference.
JP-T-2004-530368 JP 2000-013625 A JP 2000-036043 A JP 2004-145399 A JP 2006-65676 A

  Certainly, with the above-described conventional image processing apparatus, when viewed as an entire image, the contrast of the output image can be made clearer than the input image, so that the image quality and visibility can be improved. .

  However, in the above conventional image processing apparatus, the contrast of the output image can be clarified by the luminance dynamic range correction processing, but on the other hand, unnecessary noise components are emphasized, and there is a possibility that the image quality and visibility of the output image are impaired. . For example, for an input image having a high frequency of low-luminance pixels (that is, a dark input image), an image sensor (particularly, CMOS [ The noise of Complementary Metal-Oxide Semiconductor] and CCD [Charge Coupled Devices] sensors may be brightly emphasized.

  Conventionally, various techniques for removing the noise have been disclosed and proposed, but the conventional noise removal processing is one in which a filter coefficient is set based on the luminance value (brightness / darkness) of the input image. The correction amount of the luminance dynamic range correction process was not reflected. For this reason, depending on the input image, a pixel with a small correction amount (a pixel in which the noise is not so emphasized) is unnecessarily strong, and the output image may be blurred and the sharpness of the output image may be impaired. It was.

  In view of the above problems, the present invention appropriately removes noise without impairing the sharpness of an output image when generating a desired output image by performing luminance conversion processing (luminance dynamic range correction processing) on the input image. An object of the present invention is to provide an image processing circuit capable of improving the image quality and visibility of an output image, a semiconductor device in which the image processing circuit is integrated, and an image processing device using the same.

  In order to achieve the above object, an image processing circuit according to the present invention includes a luminance conversion processing unit that performs luminance conversion processing on each pixel constituting an input image; and each of the output image of the luminance conversion processing unit. A noise removal processing unit that performs noise removal processing on the pixel; a difference value between a luminance value for each pixel constituting the input image and a luminance value for each pixel constituting the output image of the luminance conversion processing unit And a filter coefficient setting unit that sets a filter coefficient for the noise removal processing according to the calculation result (first configuration).

  In the image processing circuit having the first configuration, the filter coefficient setting unit sets the filter coefficient so as to increase the noise removal processing as the difference value increases (second configuration). It is good to.

  By adopting such a configuration, a portion with a large correction amount (the difference value) by the luminance conversion processing (a portion where noise may be emphasized) is subjected to a strong noise removal processing, while the correction amount is For small portions, it is possible to weaken the noise removal process and prioritize the sharpness in view of the fact that noise is less noticeable. Therefore, when the input image is subjected to luminance conversion processing (luminance dynamic range correction processing) to generate a desired output image, noise is appropriately removed without impairing the sharpness of the output image, and the image quality and visibility of the output image are reduced. Can be increased.

  Alternatively, the image processing circuit according to the present invention includes a luminance conversion processing unit that performs luminance conversion processing on each pixel constituting the input image; and noise removal for each pixel constituting the output image of the luminance conversion processing unit. A noise removal processing unit that performs processing; an edge enhancement processing unit that performs edge enhancement processing on each pixel that constitutes an output image of the noise removal processing unit; a luminance value for each pixel that constitutes the input image; A filter coefficient setting unit that calculates a difference value from the luminance value of each pixel constituting the output image of the luminance conversion processing unit, and sets a filter coefficient of the edge enhancement processing according to the calculation result; (The third configuration).

  In the image processing circuit having the third configuration, the filter coefficient setting unit sets the filter coefficient so that the edge enhancement processing is strengthened as the difference value is smaller (fourth configuration). It is good to.

  By adopting such a configuration, the edge enhancement processing is applied to a portion where the correction amount (the difference value) by the luminance conversion processing is small (a portion that may be subjected to unnecessary strong noise removal processing). On the other hand, it is possible to prioritize the noise removal over the blur suppression of the output image by weakening the edge enhancement processing for the portion with a large correction amount. Therefore, when the input image is subjected to luminance conversion processing (luminance dynamic range correction processing) to generate a desired output image, noise is appropriately removed without impairing the sharpness of the output image, and the image quality and visibility of the output image are reduced. Can be increased.

  The semiconductor device according to the present invention has a configuration (fifth configuration) in which the image processing circuits having any one of the first to fourth configurations are integrated.

  An image processing apparatus according to the present invention includes the semiconductor device having the fifth configuration and an imaging unit that forms an optical image of a subject and images the subject, and the input image Is a configuration (sixth configuration) that is an image obtained by the imaging. With such a configuration, it is possible to take an image of a subject while enjoying the benefits obtained by any of the above configurations.

  An image processing apparatus according to the present invention includes the semiconductor device having the fifth configuration and a broadcast receiving unit or a media reproducing unit, and the input image is obtained by the broadcast receiving unit. A configuration (seventh configuration) that is a received image or a reproduction image obtained by the media reproduction means may be used. With such a configuration, it is possible to receive broadcasts or reproduce media while enjoying the benefits obtained by any of the above configurations.

  As described above, an image processing circuit according to the present invention, a semiconductor device integrated with the image processing circuit, and an image processing device using the image processing circuit can perform optimum noise removal processing or edge enhancement for each pixel of the input image. Since it can be processed, when the input image is subjected to luminance conversion processing (luminance dynamic range correction processing) to generate the desired output image, noise is appropriately removed and output without impairing the sharpness of the output image. Image quality and visibility can be improved.

  In the following, a detailed description will be given by taking as an example a case where the present invention is applied to an image processing apparatus that performs luminance conversion processing on an image of each frame obtained by moving image capturing.

  First, a first embodiment of an image processing apparatus according to the present invention will be described in detail.

  FIG. 1 is a block diagram showing a first embodiment of an image processing apparatus according to the present invention.

  As shown in this figure, the image processing apparatus 1A according to the present embodiment includes an imaging unit 10, a luminance conversion processing unit 11, a luminance conversion coefficient calculation unit 12, a noise removal processing unit 13, and a filter coefficient setting unit 14. , And an output unit 15.

  The imaging unit 10 includes a predetermined lens, an imaging device (CMOS sensor, CCD sensor), and the like, and performs imaging processing of the subject (for example, moving image imaging at 30 frames per second) by forming an optical image of the subject. Means. Note that image data for each frame obtained by the imaging unit 10 is sequentially output to the luminance conversion processing unit 11, the luminance conversion coefficient calculation unit 12, and the filter coefficient setting unit 14.

  The luminance conversion processing unit 11 performs predetermined luminance conversion processing (for each pixel constituting the image data (input image) input from the imaging unit 10 based on the luminance conversion coefficient obtained by the luminance conversion coefficient calculation unit 12 ( (Luminance dynamic range correction processing).

  The luminance conversion coefficient calculation unit 12 is a unit that calculates the luminance conversion coefficient based on the luminance distribution of the input image. More specifically, the luminance conversion coefficient calculation unit 12 first performs gamma correction on the input image, and then, based on the corrected data, the luminance distribution for the entire input image (for each luminance range). Frequency, that is, a distribution of the number of pixels belonging to each predetermined luminance range), and based on this luminance distribution, for each luminance range in the input image, the higher the frequency, the wider the luminance range in the output image, Conversely, the luminance conversion coefficient is calculated so that the lower the frequency, the narrower the luminance range is assigned to the output image. Therefore, the luminance conversion coefficient is a separate value for each luminance or for each predetermined luminance range.

  The noise removal processing unit 13 is means for performing predetermined noise removal processing on each pixel constituting the output image of the luminance conversion processing unit 11 based on the filter coefficient obtained by the filter coefficient setting unit 14. As an example of the noise removal process, the luminance value of the pixel that is the target of the noise removal process and the luminance value of the surrounding pixels (for example, the upper, lower, left, and right pixels) are referred to, and a predetermined weighting calculation is performed to obtain an image. It is conceivable to apply a low pass filter process spatially. By such noise removal processing, it is possible to reduce the luminance difference between adjacent pixels and make the noise inconspicuous.

  The filter coefficient setting unit 14 has a luminance value for each pixel constituting the input image (a luminance value before correction) and a luminance value for each pixel constituting the output image of the luminance conversion processing unit 11 (a luminance value after correction). ), And a filter coefficient for noise removal processing is set according to the calculation result.

  The output unit 15 includes a display such as an LCD [Liquid Crystal Display], and is a means for displaying an output image that has been subjected to noise removal processing by the noise removal processing unit 13.

  Of the above-described components, the luminance conversion processing unit 11, the luminance conversion coefficient calculation unit 12, the noise removal processing unit 13, and the filter coefficient setting unit 14 are preferably integrated in a semiconductor device.

  The image processing apparatus 1A having the above configuration performs luminance conversion processing based on the luminance conversion coefficient on the input image obtained by moving image capturing, and displays this on the display. By performing such luminance conversion processing, it is possible to make the contrast of a portion of the luminance range with high frequency in the input image clearer.

  In addition, the image processing apparatus 1A according to the present embodiment is not configured to set the filter coefficient for noise removal processing based on the luminance value (brightness) of the input image, but the difference between the luminance value before correction and the luminance value after correction. The filter coefficient of the noise removal process is set based on the value (that is, the correction amount by the luminance conversion process).

  More specifically, in the image processing circuit 1A of the present embodiment, as schematically shown in FIG. 2, the filter coefficient setting unit 14 increases the noise removal processing as the correction amount by the luminance conversion processing increases. In addition, a filter coefficient for noise removal processing is set with a linear or curved correlation. In FIG. 2, the target portions X1 to X4 of the luminance dynamic range correction process correspond to the target portions Y1 to Y4 of the noise removal process, respectively.

  In this way, by adopting a configuration in which the noise removal processing is performed by reflecting the correction amount of the luminance dynamic range correction processing, a portion where the correction amount (the difference value) by the luminance conversion processing is large (the noise may be emphasized). With respect to a certain part), the noise removal process is strongly applied. On the other hand, in the part with a small correction amount, the noise removal process is weakened in view of the fact that the noise is not conspicuous. . Therefore, in the image processing apparatus 1A according to the present embodiment, when a desired output image is generated by performing luminance conversion processing (luminance dynamic range correction processing) on the input image, noise is not affected without impairing the sharpness of the output image. It is possible to appropriately remove the image quality and visibility of the output image.

  Next, a second embodiment of the image processing apparatus according to the present invention will be described in detail.

  FIG. 3 is a block diagram showing a second embodiment of the image processing apparatus according to the present invention.

  As shown in the figure, the image processing apparatus 1B of the present embodiment has a configuration substantially similar to that of the first embodiment described above, and has an edge enhancement processing unit 16 at the subsequent stage of the noise removal processing unit 13. In addition, the filter coefficient setting unit 14 for setting the filter coefficient of the edge enhancement processing unit 16 is provided instead of the filter coefficient setting unit 14 for setting the filter coefficient for noise removal processing.

  The edge enhancement processing unit 16 is means for performing predetermined edge enhancement processing on each pixel constituting the output image of the noise removal processing unit 13 based on the filter coefficient obtained by the filter coefficient setting unit 17. As an example of the edge enhancement processing, the luminance value of the pixel to be subjected to the edge enhancement processing and the luminance value of the surrounding pixels (for example, the four pixels on the top, bottom, left, and right) are referred to, and a predetermined weighting calculation is performed, whereby the image It is conceivable to apply a high-pass filter process spatially. By such edge enhancement processing, it is possible to make the outline of the output image obscured by the noise removal processing stand out and suppress the blur of the output image.

  The filter coefficient setting unit 17 has a luminance value for each pixel constituting the input image (a luminance value before correction) and a luminance value for each pixel constituting the output image of the luminance conversion processing unit 11 (a luminance value after correction). ), And a filter coefficient for edge enhancement processing is set according to the calculation result.

  As described above, the image processing apparatus 1B according to the present embodiment sets the filter coefficient for the noise removal processing based on the difference value between the luminance value before correction and the luminance value after correction (that is, the correction amount by the luminance conversion processing). Instead, the filter coefficient for edge enhancement processing is set.

  More specifically, in the image processing circuit 1B of the present embodiment, as schematically shown in FIG. 4, the filter coefficient setting unit 17 strengthens the edge enhancement process as the correction amount by the luminance conversion process is smaller. In addition, the filter coefficient of the edge enhancement processing is set with a linear or curved correlation. In FIG. 4, the target portions X1 to X4 of the luminance dynamic range correction processing correspond to the target portions Y1 to Y4 of the edge enhancement processing, respectively.

  Thus, by adopting a configuration in which the edge enhancement process is performed by reflecting the correction amount of the luminance dynamic range correction process, a portion where the correction amount (the difference value) by the luminance conversion process is small (unnecessarily strong noise removal process is performed). For parts that may be applied), edge enhancement processing is applied more strongly, while for parts with a large amount of correction, edge enhancement processing is weakened and noise removal is prioritized over blurring of the output image. Is possible. Therefore, when the input image is subjected to luminance conversion processing (luminance dynamic range correction processing) to generate a desired output image, noise is appropriately removed without impairing the sharpness of the output image, and the image quality and visibility of the output image are reduced. Can be increased. Therefore, in the image processing apparatus 1B according to the present embodiment, when a desired output image is generated by performing luminance conversion processing (luminance dynamic range correction processing) on the input image, noise is not impaired without impairing the sharpness of the output image. It is possible to appropriately remove the image quality and visibility of the output image.

  In the above embodiment, the case where the present invention is applied to an image processing apparatus that performs luminance conversion processing on each frame image obtained by moving image capturing has been described as an example. The target is not limited to this, and the input image is subjected to luminance conversion processing, such as an image processing device that performs luminance conversion processing on a received image of a television broadcast, a playback image of a DVD, or a still image such as a photograph. The present invention can be applied to an image processing circuit that generates the output image, a semiconductor device formed by integrating the output processing circuit, and an image processing apparatus using the image processing circuit.

  The configuration of the present invention can be variously modified within the scope of the present invention in addition to the above embodiment.

  For example, in the above-described embodiment, the configuration in which the noise removal processing unit 13 is provided in the subsequent stage of the luminance conversion processing unit 11 has been described as an example. However, the configuration of the present invention is not limited to this, and FIG. As shown in FIG. 5 and FIG. 6, the noise removal processing unit 13 may be provided before the luminance conversion processing unit 11.

  The present invention relates to an image processing circuit that performs luminance conversion processing on an input image to generate a desired output image, a semiconductor device integrated with the image processing circuit, and an image processing device using the image processing circuit. This is a useful technique for improving the performance.

These are block diagrams which show 1st Embodiment of the image processing apparatus which concerns on this invention. These are schematic diagrams showing the correlation between the luminance correction amount and the intensity of the noise removal processing. These are block diagrams which show 2nd Embodiment of the image processing apparatus which concerns on this invention. These are schematic diagrams showing the correlation between the luminance correction amount and the strength of the edge enhancement processing. These are block diagrams which show 3rd Embodiment of the image processing apparatus which concerns on this invention. These are block diagrams which show 4th Embodiment of the image processing apparatus which concerns on this invention.

Explanation of symbols

1A, 1B Image processing apparatus 10 Imaging unit (imaging means)
DESCRIPTION OF SYMBOLS 11 Brightness conversion process part 12 Brightness conversion coefficient calculation part 13 Noise removal process part 14 Filter control part 15 Output part 16 Edge emphasis process part 17 Filter control part

Claims (7)

A luminance conversion processing unit for performing luminance conversion processing on each pixel constituting the input image;
A noise removal processing unit for performing noise removal processing on each pixel constituting the output image of the luminance conversion processing unit;
A difference value between a luminance value for each pixel constituting the input image and a luminance value for each pixel constituting the output image of the luminance conversion processing unit is calculated, and the noise removal processing is performed according to the calculation result. A filter coefficient setting unit for setting a filter coefficient;
An image processing circuit comprising:   The image processing circuit according to claim 1, wherein the filter coefficient setting unit sets the filter coefficient so that the noise removal process is strengthened as the difference value increases. A luminance conversion processing unit for performing luminance conversion processing on each pixel constituting the input image;
A noise removal processing unit for performing noise removal processing on each pixel constituting the output image of the luminance conversion processing unit;
An edge enhancement processing unit that performs edge enhancement processing on each pixel constituting the output image of the noise removal processing unit;
A difference value between a luminance value for each pixel constituting the input image and a luminance value for each pixel constituting the output image of the luminance conversion processing unit is calculated, and the edge enhancement processing is performed according to the calculation result. A filter coefficient setting unit for setting a filter coefficient;
An image processing circuit comprising:   The image processing circuit according to claim 3, wherein the filter coefficient setting unit sets the filter coefficient so that the edge enhancement processing is strengthened as the difference value is small.   5. A semiconductor device in which the image processing circuit according to claim 1 is integrated.   6. The semiconductor device according to claim 5, and an imaging unit that forms an optical image of a subject and images the subject, wherein the input image is an image obtained by the imaging. An image processing apparatus.   6. The semiconductor device according to claim 5, and broadcast receiving means or media playback means, wherein the input image is a received image obtained by the broadcast reception means or obtained by the media playback means. An image processing apparatus characterized by being a reproduced image.

Images