JP2011081034A - Image processing apparatus and image display device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image processing apparatus and image display device, capable of performing optimum dynamic range compression processing in accordance with various factors during display. <P>SOLUTION: The image processing apparatus 1 for correcting and processing image data is composed as follows. That is, the image processing apparatus 1 includes: a luminance correction threshold calculation unit 10 calculating a correction threshold for correcting a pixel vale of each pixel constituting image data based on display condition information indicating display condition when image data is reproduced and displayed; an image brightness value smoothing unit 12 performing smoothing processing for image data; an brightness correction coefficient calculation unit 16 calculating a correction coefficient for converting a pixel value of a correction object pixel in which a pixel value is less than a correction threshold to the same value as the correction threshold out of respective pixels constituting image data in which smoothing processing is performed; and a brightness correction coefficient multiplication unit 20 generating output image data by multiplying a pixel value of the correction object pixel, and the correction coefficient corresponding to the correction object pixel. <P>COPYRIGHT: (C)2011,JPO&INPIT

Description

  The present invention relates to an image processing apparatus and an image display apparatus for correcting input image data.

  Conventionally, a digital camera that performs dynamic range compression processing on a captured image is known. As a technology related to this, for example, Patent Literature 1 discloses the following technology.

  That is, Patent Document 1 discloses an imaging device for photographing a subject, processing means capable of processing a photographed image in a first image mode and a second image mode, and a photographed image. A first dynamic range compressing unit that compresses the dynamic range with high quality; a second dynamic range compressing unit that compresses the dynamic range at high speed; and the first dynamic range compressing unit in the first image mode. There is disclosed an imaging apparatus comprising: a compression processing selection unit that selects and selects the second dynamic range compression unit in the second image mode.

  According to the technique disclosed in Patent Document 1, dynamic range compression processing is performed on a bright portion of a subject at the time of recording a photographed image to improve the quality of a dark portion, and a subject having a large difference in brightness can be obtained. It can be recorded appropriately.

  Further, according to the technique disclosed in Patent Document 1, when a preview image is displayed on a display, in order to reduce the difference between the preview image and a captured image recorded by the dynamic range compression processing. In addition, the photographed image subjected to the predetermined dynamic range compression processing is displayed on the display as a preview image.

JP 2007-180718 A

  By the way, in order to properly record and display image data, dynamic range compression processing is performed in view of the very large difference between the dynamic range of human vision and the dynamic range of image sensors and displays. Must be done.

  Furthermore, the dynamic range of human vision changes according to external light reflection on the display surface, for example. In addition, the brightness of the backlight of the display and the illuminance of the environment where the display is placed naturally affect the dynamic range of human vision.

  Therefore, the visibility of the image displayed on the display is greatly reduced depending on various factors at the time of display (it is difficult to visually recognize the gradation difference between each pixel constituting the image). Become). In other words, it may not be possible to obtain the effect of the dynamic range compression process.

  Specifically, for example, when the luminance of the image displayed on the display is lower than the reflection luminance on the display surface (panel surface), the gradation of the image appears to be crushed, so the visibility is greatly reduced.

  That is, when performing dynamic range compression processing on an image to be displayed, it is necessary to perform optimum dynamic range compression processing according to various factors during display.

  Note that the technique disclosed in Patent Document 1 does not consider such a problem.

  The present invention has been made in view of the above circumstances, and an object thereof is to provide an image processing apparatus and an image display apparatus capable of performing an optimum dynamic range compression process according to various factors during display. And

In order to achieve the above object, an image processing apparatus according to a first aspect of the present invention is an image processing apparatus that corrects input image data, and sets a brightness correction threshold value based on the reflected brightness of a display. A correction threshold value calculation unit for calculating and correcting a luminance value so that a pixel having a luminance value less than the luminance correction threshold value calculated by the correction threshold value calculation unit among the pixels of the image data is equal to or greater than the luminance correction threshold value. And a correction unit.

In order to achieve the above object, an image display device according to a second aspect of the present invention includes:
An image display device for correcting and displaying input image data,
A correction threshold value calculation unit for calculating a correction threshold value for correcting a pixel value of each pixel constituting the image data based on display condition information indicating a display condition when reproducing and displaying the image data;
A smoothing unit that performs a smoothing process on the image data;
Among the pixels constituting the image data smoothed by the smoothing unit, the pixel value of the correction target pixel whose pixel value is less than the correction threshold is converted to the same value as the correction threshold. A correction coefficient calculation unit for calculating a correction coefficient;
A correction unit that multiplies the pixel value of the correction target pixel by a correction coefficient corresponding to the correction target pixel to generate output image data;
A display unit for displaying the output image data generated by the correction unit;
It is characterized by comprising.

  ADVANTAGE OF THE INVENTION According to this invention, the image processing apparatus and image display apparatus which can perform the optimal dynamic range compression process according to the various factors at the time of a display can be provided.

1 is a diagram illustrating a configuration example of an image processing apparatus according to an embodiment of the present invention. The figure which shows the flowchart of the dynamic range compression process by the image processing apparatus which concerns on one Embodiment of this invention. The figure which shows an example of input image data. The figure which shows an example of clip image data. The figure which shows an example of smoothed image data. The figure which shows an example of the brightness correction coefficient imaged. The figure which shows an example of a brightness | luminance correction coefficient characteristic. The figure which shows an example of output image data.

  Hereinafter, an image processing apparatus and an image display apparatus according to an embodiment of the present invention will be described.

  FIG. 1 is a diagram illustrating a configuration example of an image display apparatus including an image processing apparatus according to the present embodiment. As shown in FIG. 1, the image display apparatus comprehensively controls the image processing apparatus 1 according to the present embodiment, an image data input unit 100 that inputs image data to the image processing apparatus 1, and the entire image display apparatus. And a display unit 120 that displays the image data output from the image processing apparatus 1.

  The image processing apparatus 1 includes an input image buffer 2, a backlight 4, an illuminance sensor 6, a panel reflectance storage unit 8, a luminance correction threshold value calculation unit 10, an image luminance value smoothing unit 12, and smoothing The image buffer 14, the luminance correction coefficient calculation unit 16, the luminance correction coefficient buffer 18, the luminance correction coefficient multiplication unit 20, and the output image buffer 22 are included.

  The input image buffer 2 stores the image data input from the image data input unit 100.

  The backlight 4 is a light emitting member that irradiates the display unit 120 formed of, for example, a transmissive or transflective liquid crystal display from the back side thereof. The backlight 4 generates luminance data indicating the emission luminance and outputs the luminance data to the luminance correction threshold value calculation unit 10. Note that a liquid crystal display is often mounted on a mobile device represented by a mobile phone or the like, and the liquid crystal display usually requires a backlight. Hereinafter, the liquid crystal display is simply referred to as a display.

  The illuminance sensor 6 detects the brightness of the environment around the image processing apparatus 1 (or an image display apparatus including the image processing apparatus), and outputs the detection result to the luminance correction threshold value calculation unit 10 as illuminance data. To do.

  The panel reflectivity storage unit 8 stores panel reflectivity data indicating the panel reflectivity of a display that displays the image data processed by the image processing apparatus 1.

  The luminance correction threshold value calculation unit 10 is based on the luminance data output from the backlight 4, the illuminance data output from the illuminance sensor 6, and the panel reflectivity data output from the panel reflectivity storage unit 8. The brightness correction threshold value is calculated. Specifically, the luminance correction threshold value calculation unit 10 calculates the reflection luminance of the panel surface in the display based on the illuminance data and the panel reflectance data, and the reflection luminance of the panel surface and the luminance data of the backlight 4 Based on the above, a luminance correction threshold value is calculated. The method for calculating the brightness correction threshold will be described in detail later.

  The image luminance value smoothing unit 12 performs clipping processing and smoothing processing described later on the image data.

  The smoothed image buffer 14 stores the image data output from the image luminance value smoothing unit 12.

  The luminance correction coefficient calculation unit 16 sequentially extracts pixels constituting the image data stored in the smoothed image buffer 14, calculates a luminance correction coefficient described later, and outputs the luminance correction coefficient to the luminance correction coefficient buffer 18. The luminance correction coefficient is a pixel value of a pixel constituting the image data (in this example, meaning a luminance value; hereinafter the same) when the pixel value is less than the luminance correction threshold. This is a multiplication coefficient for conversion to the same value as the luminance correction threshold. When the pixel value of the pixel extracted from the smoothed image buffer 14 is equal to or greater than the luminance correction threshold, the luminance correction coefficient for the pixel is set to 1.

  The luminance correction coefficient buffer 18 stores the luminance correction coefficient output from the luminance correction coefficient calculation unit 16.

  The luminance correction coefficient multiplication unit 20 sequentially extracts pixels constituting the image data stored in the input image buffer 2, and sequentially extracts the luminance correction coefficients of the pixels corresponding to the pixels from the luminance correction coefficient buffer 18, The pixel is multiplied by the pixel value and the luminance correction coefficient and output to the output image buffer 22.

  Hereinafter, dynamic range compression processing by the image processing apparatus according to the present embodiment will be described. FIG. 2 is a diagram illustrating a flowchart of the dynamic range compression processing by the image processing apparatus according to the present embodiment. As shown in the figure, this processing is divided into “processing A relating to luminance correction coefficient calculation (step S1 to step S5)” and “processing B relating to output image data generation (step S6 to step S7)”. Broadly divided.

  First, the luminance correction threshold value calculation unit 10 calculates a luminance correction threshold value based on the reflection luminance of the panel surface in the display calculated by multiplying the illuminance data and the panel reflectance data and the luminance data of the backlight 4. (Step S1).

Specifically, for the display, for example, the reflection luminance of the panel surface is 14 [cd], the luminance of the backlight 4 is 300 [cd], the gamma value is 2.2, and the gradation expression capability is 8 bits (maximum). Assuming that the gradation 255), the brightness correction threshold is

Can be calculated as Note that the value of the brightness correction threshold (upper limit) ThMAx is also a pixel value corresponding to the reflected brightness.

Here, (Formula 1) is generalized as follows:
ThMax = maximum gradation × (reflection luminance ÷ backlight luminance) ^ (1 ÷ gamma value) (Equation 1) ′
It is expressed.

Subsequently, the image luminance value smoothing unit 12 extracts the pixel value of the pixels constituting the image data (input image data shown in FIG. 3A) from the input image buffer 2, and sets the pixel value to the luminance correction threshold value (upper limit) ThMAx. Based on this, clip processing is performed according to the following expression (the pixel value larger than the value of the brightness correction threshold (upper limit) ThMax is limited to the value of the brightness correction threshold (upper limit) ThMax), and the clip image data shown in FIG. 3B is generated. To do.

  Here, in [i, j] represents the pixel value of each pixel constituting the input image data, and Clip [i, j] represents the pixel value of each pixel constituting the clip image data.

  This clip process is not an essential process. However, by performing this clipping process, when there is a bright pixel (a pixel with a high luminance) in the vicinity of a pixel that needs to be corrected for luminance, a “pixel that needs to be corrected for luminance after smoothing processing described later is performed. It is possible to prevent the pixel value “” from being inadvertently increased.

Further, the image luminance value smoothing unit 12 smoothes the clip appropriate image data by the following expression, generates the smoothed image data shown in FIG. 3C, and outputs the smoothed image data to the smoothed image buffer 14 (step S2).

  Here, Blur [i, j] is a pixel value of each pixel constituting the smoothed image data, and f (x, y) is a Gaussian function.

  In this example, f (x, y) is a normal distribution function. However, a function having another shape may be used as long as smoothing processing is possible.

  By the way, the control unit 110 determines whether or not the processing in step S1 and step S2 has been performed on all the pixels constituting the image data input to the image processing apparatus 1 (step S3). When step S3 is branched to NO, the process returns to step S2.

When step S3 is branched to YES, the luminance correction coefficient calculation unit 16 determines the pixel value (Blur [i, j]) of each pixel constituting the smoothed image data, the luminance correction threshold (upper limit) ThMax, and the luminance. Based on the correction threshold value (lower limit) ThMin, the luminance correction coefficient a [i, j] of each pixel is calculated by the following equation (step S4) and output to the luminance correction coefficient buffer 18.

  Note that the value of the brightness correction threshold (lower limit) ThMin is a threshold for suppressing the expansion of the noise component, and is determined in advance according to the quality of the input image data. Specifically, when the image processing apparatus or the image display apparatus according to the present embodiment is applied to, for example, a preview display in a camera, a luminance correction threshold (lower limit) so that noise mixed in the camera system (imaging system) can be seen darely. It is conceivable to set ThMin. For example, when applied to a compressed image such as a moving image clip, it is also conceivable to increase the luminance correction threshold (lower limit) ThMin so that the coding noise is not visible. That is, the value of the brightness correction threshold (lower limit) ThMin needs to be set as appropriate depending on the application, compression rate, and the like. Furthermore, regarding the compression rate, in view of the fact that various compression distortions are mixed when applied to an image that has been subjected to secondary processing, the current situation is that the luminance correction threshold (lower limit) ThMin is expressed by one equation. However, the calculation is actually performed by, for example, a method of adjusting by visual observation or the like.

  The brightness correction coefficient a [i, j] calculated in this way is used as a coefficient for calculating output image data in the subsequent processing. When converted into an image, it is represented, for example, as shown in FIG.

  Here, the luminance correction coefficient a [i, j] calculated by the above (Formula 5) has the characteristic (correction coefficient characteristic) shown in FIG. In step S6 to be described later, based on the correction coefficient characteristics shown in FIG. 5, the pixel value of each pixel constituting the input image data is converted to calculate the pixel value of each pixel constituting the output image data.

  By the way, the control unit 110 determines whether or not the processing in step S4 has been performed for all the pixels constituting the smoothed image data (step S5). When step S5 is branched to NO, the process returns to step S4.

When step S5 is branched to YES, the luminance correction coefficient multiplying unit 20 sequentially extracts the pixel values of the pixels constituting the input image data from 2 from the input image buffer, and uses the luminance correction coefficient corresponding to the pixels as the luminance. They are taken out from the correction coefficient buffer 18 and multiplied as shown in the following equation to generate the pixel value out [i, j] of each pixel constituting the output image data and output it to the output image buffer 22 (step S6). .

  Then, the control unit 110 determines whether or not the processing in step S6 has been performed for all the pixels constituting the input image data (step S7). When step S7 is branched to NO, the process returns to step S6. On the other hand, when step S7 is branched to YES, the image processing is terminated. At this time, pixel values of all the pixels constituting the output image data (see FIG. 6) are calculated and stored in the output image buffer 22.

  As described above, for the sake of convenience, the dynamic range compression processing by the image processing apparatus according to the present embodiment has been described assuming monochrome image data, but it is needless to say that it can also be applied to color image data. Hereinafter, an example in which the above-described series of dynamic range compression processing is applied to color image data will be described. In order to avoid duplication of explanation, only differences will be explained.

As is well known, one pixel in color display is composed of three RGB pixels. Therefore, when calculating the pixel value of each pixel constituting color image data, the pixel value of each pixel is obtained by a method such as taking the maximum value of each RGB pixel or taking the average value of each RGB pixel. May be calculated. In other processes, using the pixel values calculated in this way,
What is necessary is just to apply the process mentioned above to each pixel of RGB.

Specifically, the pixel values of the RGB pixels constituting the pixel value in [i, j] of each pixel constituting the input image data are set to in [i, j] .R, in [i, j] .G. , In [i, j] .B, (Expression 2) is replaced with the following expression.

Further, (Equation 6) described above is replaced with the following equation.

  As described above, the method of calculating the pixel value by taking the maximum value of each RGB pixel is a more effective method when the display image is a color close to the primary color.

  As described above, according to the present embodiment, it is possible to provide an image processing apparatus and an image display apparatus that can perform an optimal dynamic range compression process according to various factors during display. Specifically, for example, the following effects can be obtained.

-Effective dynamic range compression processing was realized by changing the compression characteristics of dynamic range compression processing according to the optical dynamic range of the display that changes according to the brightness of the viewing environment.

Correction processing is performed so that the luminance of the image displayed on the display exceeds the reflection luminance of the panel surface of the display. That is, an optimum dynamic range compression process according to the viewing environment is realized by extracting pixels in the image data that are equal to or lower than the reflection luminance and correcting the brightness so as to be equal to or higher than the reflection luminance.

・ Compared to the conventional technology that makes the final purpose of dynamic range compression processing match the gradation expression capability of the display, gradation is not easily lost even in an environment with a large external light factor, and a good image display is possible. It becomes. This can be said to be a particularly special effect for mobile devices represented by mobile phones and the like.

・ Because the dynamic range compression processing is performed according to the viewing environment, the image data that is poor in visibility because it is darker than the reflection brightness of the display panel when displayed using the conventional technology is optimally brightened. Can be converted and displayed.

・ Image data is corrected based not only on the dynamic range of the display that displays the image data but also on the dynamic range of the viewer, which varies with external light. Therefore, optimal display based on both the display and the viewing environment is performed. An image can be generated.

The above-described processing can be performed using the circuit for performing γ correction provided in the conventional image processing apparatus as it is. Therefore, according to the present embodiment, the structure is not complicated.

  The present invention has been described based on one embodiment. However, the present invention is not limited to the above-described embodiment, and various modifications and applications are possible within the scope of the gist of the present invention. It is.

  Further, the above-described embodiments include inventions at various stages, and various inventions can be extracted by appropriately combining a plurality of disclosed constituent elements. For example, even if some constituent requirements are deleted from all the constituent requirements shown in the embodiment, the problem described in the column of the problem to be solved by the invention can be solved, and the effect described in the column of the effect of the invention can be achieved. In the case of being obtained, a configuration from which this configuration requirement is deleted can be extracted as an invention.

    DESCRIPTION OF SYMBOLS 1 ... Image processing apparatus, 2 ... Input image buffer, 4 ... Backlight, 6 ... Illuminance sensor, 8 ... Panel reflectance memory | storage part, 10 ... Luminance correction threshold value calculation part, 12 ... Image luminance value smoothing part, 14 ... Smoothing 16... Luminance correction coefficient calculation unit 18... Luminance correction coefficient buffer 20... Luminance correction coefficient multiplication unit 22... Output image buffer 100... Image data input unit 110.

Claims (6)

An image processing apparatus for correcting input image data,
A correction threshold value calculation unit for calculating a luminance correction threshold value based on the reflected luminance of the display;
A correction unit that corrects a luminance value to be equal to or higher than the luminance correction threshold for a pixel having a luminance value less than the luminance correction threshold calculated by the correction threshold calculation unit among the pixels of the image data. An image processing apparatus.   The correction threshold value calculation unit has a value equal to or greater than the luminance correction threshold value for a pixel having a luminance value less than the luminance correction threshold value calculated by the correction threshold value calculation unit and greater than or equal to a predetermined threshold value among the pixels of the image data. The image processing apparatus according to claim 1, wherein the brightness value is corrected so that An image processing apparatus for correcting input image data,
A correction threshold value calculation unit for calculating a correction threshold value for correcting a pixel value of each pixel constituting the image data based on display condition information indicating a display condition when reproducing and displaying the image data;
A correction coefficient calculation unit that calculates a correction coefficient for converting a pixel value of a correction target pixel whose pixel value is less than the correction threshold value to a value equal to or higher than the correction threshold value among the pixels constituting the image data;
A correction unit that multiplies the pixel value of the correction target pixel by a correction coefficient corresponding to the correction target pixel to generate output image data;
An image processing apparatus comprising: The output image data generated by the correction unit is output to a display unit including a display panel and a backlight for illuminating the display panel,
The image processing apparatus includes:
A luminance data acquisition unit that acquires luminance data indicating the luminance of the backlight included in the display unit;
An illuminance sensor that detects illuminance of the environment in which the display unit is placed and generates illuminance data; a panel reflectance storage unit that holds panel reflectance data indicating the panel reflectance of the display panel;
With
The image processing apparatus according to claim 3, wherein the display condition information is the luminance data, the illuminance data, and the panel reflectance data. A clip processing unit that performs a process of limiting a pixel value based on the correction threshold for each pixel constituting the input image data;
The image processing apparatus according to claim 3, wherein the smoothing unit performs a smoothing process on the image data processed by the clip processing unit. An image display device for correcting and displaying input image data,
A correction threshold value calculation unit for calculating a correction threshold value for correcting a pixel value of each pixel constituting the image data based on display condition information indicating a display condition when reproducing and displaying the image data;
A smoothing unit that performs a smoothing process on the image data;
Among the pixels constituting the image data smoothed by the smoothing unit, the pixel value of the correction target pixel whose pixel value is less than the correction threshold is converted into the same value as the correction threshold. A correction coefficient calculation unit for calculating a correction coefficient;
A correction unit that multiplies the pixel value of the correction target pixel by a correction coefficient corresponding to the correction target pixel to generate output image data;
A display unit for displaying the output image data generated by the correction unit;
An image display device comprising:

Images