JP2019161491A - Display device, display method, program, and storage medium - Google Patents

Abstract

To provide a display device that allows a user to easily grasp a change in display luminance of an image due to control even when the display luminance of the image is controlled and displayed.SOLUTION: A display device according to the present invention includes display means for displaying a composite image including an input image, a first image, and a second image, and control means for performing luminance conversion for reducing display luminance of the composite image displayed by the display means in a predetermined case, and the second image is an image displayed at a reference luminance that is a luminance for the user to confirm the correspondence between a gradation value of the image and the display luminance, and the first image is an image that is displayed with the display luminance that has changed from the reference luminance by the luminance conversion.SELECTED DRAWING: Figure 1

Description

  The present invention relates to a display device, a display method, a program, and a storage medium that display an image based on an input image.

  In video production, when shooting a subject, an image may be displayed by adjusting the imaging conditions of the imaging device so that the reference brightness is associated with a predetermined gradation value. Here, the reference brightness is brightness corresponding to, for example, white gradation or gray gradation, and is brightness for the user to confirm the correspondence between the gradation value of the image and the display luminance. When editing image data obtained by the imaging device, color adjustment and gamma adjustment of the entire image are performed so that the area indicated by the predetermined gradation value does not change significantly from the reference brightness. Sometimes. Therefore, the display brightness of the area associated with the predetermined gradation value in the image data is an important index in the editing process.

  On the other hand, the imaging device generates an image (HDR image) having a dynamic range (High Dynamic Range, HDR) wider than the dynamic range (Standard Dynamic Range, SDR) used for displaying an image on the display device. Is possible. SDR is, for example, BT. The dynamic range defined in 709. Here, the dynamic range indicates a display luminance range corresponding to a display device or an image.

  In addition, the power consumption that can be used by the display device may be limited by the range in which the power supply can be output. For example, when the display device displays an HDR image in the same dynamic range as the image, the upper limit of the display brightness is higher than when the SDR image is displayed, and the required power consumption becomes higher than the limit value. was there.

  On the other hand, the display device of Patent Document 1 calculates power consumption based on an image, and when the power consumption exceeds a reference value, restricts the power consumption to a predetermined range by limiting the displayable dynamic range. . Thereby, the dynamic range of the image displayed on the display device may be different from the dynamic range of the image.

JP 2006-119465 A

  However, when the display device performs the control to limit the dynamic range as described above, the display luminance of the region corresponding to the reference brightness in the HDR image may change according to the control. When a user performs an editing operation based on an image displayed on a display device, if the display brightness of an area corresponding to a reference brightness such as gray or white changes, the necessity of color or gamma adjustment is mistaken. There was a problem.

Further, in a display device that can display an HDR image with a wider dynamic range than a conventional SDR, the displayable dynamic range may not correspond to the dynamic range of the HDR image. In such a case, a part of the dynamic range of the HDR image is clipped (displayed), or the entire dynamic range of the HDR image is compressed so that the display device can display the dynamic range. An image may be displayed. Therefore, even in such a case, the same problem as in the case where the dynamic range of the image displayed on the display device is different from the dynamic range of the image occurs.

  In view of the above-described problems, an object of the present invention is to provide a display device that allows a user to easily grasp a change in display luminance due to control even when the display luminance of an image is controlled and displayed. And

The first aspect of the present invention is:
Display means for displaying a composite image including the input image, the first image, and the second image;
Control means for performing brightness conversion for reducing display brightness of the composite image displayed by the display means in a predetermined case;
With
The second image is an image displayed at a reference luminance that is a luminance for the user to confirm the correspondence between the gradation value of the image and the display luminance.
The first image is an image that is displayed with a display brightness changed from the reference brightness in accordance with the brightness conversion.
This is a display device.

The second aspect of the present invention is:
A display step of displaying a composite image including the input image, the first image, and the second image;
A control step of performing luminance conversion for reducing display luminance of the composite image displayed by the display step;
Including
The second image is an image displayed at a reference luminance that is a luminance for the user to confirm the correspondence between the gradation value of the image and the display luminance.
The first image is an image displayed at a display brightness changed from the reference brightness in accordance with the brightness conversion.
This is a display method characterized by this.

  ADVANTAGE OF THE INVENTION According to this invention, even when a display apparatus controls and displays the display brightness of an image, the user can grasp | ascertain easily the change of the display brightness by control.

1 is a block diagram of an image display device according to Embodiment 1. FIG. It is a figure explaining the backlight part which concerns on Embodiment 1. FIG. It is a figure which shows OSD which concerns on Embodiment 1,2. It is a figure which shows the relationship between the gradation value which concerns on Embodiment 1, and display luminance. It is a conceptual diagram explaining the 1st image and 2nd image which concern on Embodiment 1. FIG. It is a figure which shows the input image which concerns on Embodiment 1. FIG. 3 is an operation flowchart of the image display apparatus according to the first embodiment. It is a figure explaining loading control concerning Embodiment 1. FIG. 6 is a diagram illustrating an image displayed on a display unit according to Embodiment 1. FIG. It is a figure which shows the area of the maximum gradation value which concerns on the modification 1, and the relationship of L_gain. 6 is a block diagram of an image display apparatus according to Embodiment 2. FIG. It is a figure explaining the clip process and compression process which concern on Embodiment 2. FIG. 10 is an operation flowchart of the image display apparatus according to the second embodiment.

  Hereinafter, exemplary embodiments of the present invention will be described in detail with reference to the drawings. The technical scope of the present invention is determined by the claims, and is not limited by the following individual embodiments. In addition, all combinations of features described in the embodiments are not necessarily essential to the present invention.

<Embodiment 1>
Embodiment 1 according to the present invention will be described below with reference to the drawings. In the present embodiment, the image display device displays an image having a display brightness (first image) and a reference brightness image (second image) in consideration of the reference brightness and loading control on the screen. However, it is possible to intuitively grasp the amount of change in display luminance due to loading control. Here, the loading control is control (display luminance conversion control) for narrowing the entire display luminance range of the image display device.

  Specifically, if the input image is a dark image as a whole, both the first image and the second image are displayed at the reference brightness, and therefore the user confirms that the input image is not subjected to loading control. Intuitive. Therefore, for example, when color adjustment of an input image is performed using a display device, it can be seen that the color adjustment performed while checking the display screen is reflected on the input image itself in the same manner as the display screen. On the other hand, if the input image is a bright image as a whole, the first image is displayed darker than the second image, so that the user can intuitively understand that the loading control is applied to the input image. Therefore, the user can adjust the gradation value for the input image by considering that the display brightness corresponding to the gradation value and the gamma characteristic of the input image is higher than the display brightness of the displayed image. In the case of performing, it is possible to prevent the gradation value from being too high. Specifically, it is possible to prevent overexposure of pixels due to adjustment of gradation values.

[Configuration of image display device]
The image display apparatus 10 according to the present embodiment includes a display unit 11, a backlight control unit 103, a loading control unit 104, an input unit 105, a first generation unit 106, a second generation unit 107, an image composition unit 108, and an image quality correction unit. 109. The display unit 11 includes a display panel 101 and a backlight unit 102. In this embodiment, the input image (input image data) is 10-bit 0-1023 category (gradation) data, and an image related to HDR related to display luminance or reflectance is input. The display luminance range of the image display device (display luminance range that can be displayed by the display unit 11) is assumed to be 0 to 1000 cd / m ² . Further, it is assumed that the display luminance reduction rate by the loading control is 0.5 to 1.0. Here, the rate of decrease adjusts the display luminance range of the image display device. If the rate of decrease is 1.0, an image can be displayed in the specified display luminance range, and if the rate of decrease is 0.5, the specified display luminance range. The image is displayed in the half range. That is, if the reduction rate is 1.0, loading control is not applied.

  The display panel 101 is a transmissive display panel having a plurality of elements capable of controlling the light transmittance according to a composite image (composite image data) input from an image quality correction unit 109 described later. For example, it is assumed that the display panel 101 is a liquid crystal panel using a liquid crystal element. Note that the display panel 101 may include a plurality of transmissive elements configured with Micro Electro Mechanical Systems (MEMS).

  The backlight unit 102 is a lighting device that includes a plurality of light sources and irradiates the display panel 101 with light. In the present embodiment, the backlight unit 102 uses an LED (Light Emitting Diode) as a light source. The light source of the backlight unit 102 corresponds to an area including a plurality of pixels of the input image, and controls light emission (brightness) in units divided into the areas.

In this embodiment, as shown in FIG. 2A, each light source corresponds to an area in which a display screen (displayed image) is divided into 10 × 6. The light source corresponding to each region is designated by coordinates (x, y) = (i, j) as shown in FIG. 2A (i, j are 1 ≦ i ≦ 10, 1 ≦, respectively). j ≦ 6).

  In addition, each light source (x, y) of the present embodiment emits light for each region described above based on a control parameter BP (x, y) output from a loading control unit 104 described later. The setting range of BP (x, y) is 0-100. That is, the emission luminance to be controlled when the upper limit of the emission luminance of each light source is 100 is BP (x, y). Further, it is assumed that the correspondence relationship of the upper limit value of the display luminance of the image with respect to BP (x, y) is a linear characteristic as shown in FIG. That is, the light emission luminance of each light source is proportional to the upper limit value of the display luminance of the region corresponding to the light source.

  The backlight control unit 103 receives an input image from the image composition unit 108 or a composite image in which the first image and the second image are superimposed on the input image. Whether an input image or a composite image is input can be determined from an image flag described later input from the image composition unit 108. The backlight control unit 103 calculates an estimated light emission luminance EP (x, y) estimated for each light source (x, y) included in the backlight unit 102. The backlight control unit 103 calculates an estimated light emission luminance EP (x, y) of each light source according to the input image. The estimated emission brightness EP (x, y) has a setting range of 0 to 100, as with the control parameter BP (x, y). In addition, it is assumed that the correspondence relationship between the estimated light emission luminance EP (x, y) and the upper limit value of the image display luminance is a linear characteristic.

  As a method of calculating the estimated light emission luminance EP (x, y), first, the backlight control unit 103 divides an image into a plurality of regions, and detects, for example, APL (Average Tone Level; Average Picture Level) for each region. . When the acquired APL is low, the backlight control unit 103 sets the estimated light emission luminance EP (x, y) of the light source (x, y) corresponding to the region to be low. When the acquired APL is high, the backlight control unit 103 sets the estimated light emission luminance EP (x, y) of the light source (x, y) corresponding to the region to be high. Thus, when the light emission luminance of the light source is changed according to the APL for each region, the contrast of the image displayed on the display unit 11 can be increased. Note that the backlight control unit 103 outputs an image flag indicating whether the input image is an input image or a composite image, and the set estimated emission luminance EP (x, y) to the loading control unit 104.

The estimated light emission luminance EP (x, y) and the image flag of each light source are input from the backlight control unit 103 to the loading control unit 104. The loading control unit 104 compares the average value P_Ave of the estimated emission luminance EP (x, y) of each light source with a predetermined threshold Th (0 to 100) set in advance, and based on the comparison result, the loading gain ( L_gain) is calculated. L_gain can be calculated using Equation 1 below.

Here, the loading gain (L_gain) is a reduction rate of the maximum luminance (a reduction rate of the display luminance range) in display luminance by loading control (display luminance conversion control). For example, when L_gain is 1.0, there is no reduction in the maximum luminance, and when L_gain is 0.5, it means that the maximum luminance is reduced to 50%. That is, the display luminance range is narrowed according to the value of L_gain.

  The threshold Th may be set according to an upper limit (maximum value) of power that can be supplied to the image display device. In the present embodiment, it is assumed that 50 is set as the threshold Th. Therefore, in this embodiment, since the threshold value Th = 50, L_gain is determined as 0.5 to 1.0. By the above-described processing, for example, the maximum luminance is limited as the area corresponding to the maximum gradation value 1023 in the image is larger. As a result, the power consumption can be kept within a predetermined range.

  When the image flag indicates a composite image, the loading control unit 104 calculates a control parameter BP (x, y) that controls the light emission luminance of each light source of the backlight unit 102. Here, the control parameter BP (x, y) can be calculated from the estimated emission luminance EP (x, y) and L_gain by the following equation: BP (x, y) = EP (x, y) × L_gain. The loading control unit 104 outputs L_gain to the second generation unit 107 when the image flag indicates an input image, and outputs the control parameter BP (x, y) to the backlight unit 102 when the image flag indicates a composite image. To do. That is, the loading control unit 104 controls the light emission luminance of the backlight unit 102 and the display luminance of the displayed image by the control parameter BP (x, y).

The input unit 105 displays, for example, an OSD (On-Screen Display) as shown in FIG. 3A on the screen, and a reference brightness (hereinafter referred to as reference luminance) is set (input) by the user. The Here, the reference luminance indicates the display luminance of the reference image for visually confirming whether or not the user has caused a decrease in the luminance of the screen. In the present embodiment, it is assumed that 100 cd / m ² is set as the reference luminance.

Note that the reference luminance is not limited to the setting by the user, but may be automatically set by the input unit 105 according to the input image, for example. For example, when the gamma characteristic of the input image is acquired and the gamma characteristic is HLG, a luminance value of 200 cd / m ² when an image with a reflectance of 100% is displayed on the screen in HLG is automatically set. Good. The reference luminance is set from a luminance range that can be displayed even when loading control is applied (0 to 500 cd / m ² in this embodiment).

The first generation unit 106 generates an image with a gradation value (N gradation value) at which a white patch of 100 cd / m ² is displayed in a state where no loading control is applied. Here, the gamma characteristic indicating the relationship between the gradation value of the image and the display luminance includes a PQ (Perceptual Quantizer) method and an HLG (Hybrid Log Gamma) method. In the present embodiment, to simplify the description, assuming that the gamma characteristic of the display panel 101 is 2.2 as shown in FIG. 4, the gradation value N is calculated based on the following Equation 2. This is because the upper limit of the display luminance range of the image display apparatus 10 is 1000 cd / m ² and the upper limit of the gradation value of the input image is 1023. Note that the gamma characteristic may be, for example, a PQ method for an image related to display luminance, or may be an HLG method for an image related to reflectance.

That is, the 1st production | generation part 106 produces | generates the white patch comprised by each 359 gradations as R value, G value, and B value as a 1st image. In the present embodiment, the first image is a rectangular image pattern as shown in FIG. 5A and 5B conceptually show the gradation values in the respective composite images, and are not actually displayed images. By generating the first image in this way, the display brightness is corrected to the display luminance corresponding to the loading gain L_gain by the above-described loading control and displayed on the screen. The first generation unit 106 outputs the generated first image to the second generation unit 107 and the image composition unit 108. The first image is not limited to a rectangle, but may be an arbitrary shape such as a circle, a polygon, or a star.

  The second generation unit 107 generates a second image based on the gradation value (N gradation value) corresponding to the reference luminance input from the first generation unit 106 and L_gain input from the loading control unit 104. .

The second generation unit 107 performs control so that the second image is always displayed at the reference luminance (100 cd / m ² ) even when there is loading control. Specifically, the second generation unit 107 generates an image by performing gradation conversion processing on the N gradation values so as to cancel out the luminance reduction due to the loading control. Hereinafter, the gradation value after gradation conversion processing applied to the N gradation value is referred to as an M gradation value. The M gradation value can be calculated from the N gradation value and L_gain by using Equation 3 below.

  The second generation unit 107 generates a white patch in which each of the R value, the G value, and the B value is an M gradation value as the second image, and outputs the generated second image to the image composition unit 108. The second image is a rectangular image pattern as shown in FIG.

  First, an input image is input to the image composition unit 108. Here, if the second image corresponding to the input image is not input from the second generation unit 107, the image composition unit 108 outputs the input image to the backlight control unit 103. At this time, the image composition unit 108 writes information indicating that the image is an input image in the image flag and outputs the information to the backlight control unit 103.

  If a second image corresponding to the input image is input from the second generation unit 107, the image composition unit 108 displays the second image, the first image input from the first generation unit 106, and the input image. A composite image is generated by combining. For example, the composite image is generated so that the first image and the second image are adjacent to each other at the end of the input image as shown in FIG. The generated composite image is output to the backlight control unit 103 and the image quality correction unit 109. At this time, the image composition unit 108 describes information indicating that the image is a composite image in an image flag and outputs the information to the backlight control unit 103.

  The image quality correction unit 109 corrects the influence of light leakage (the so-called halo phenomenon) caused by the difference in light emission amount for each region of the backlight unit 102 by image processing. The image processing of the image quality correction unit 109 is a well-known technique such as leveling processing, for example, and is not an essential configuration in the present embodiment, so detailed description thereof is omitted.

[Processing flow of image display device]
Hereinafter, an operation flow of the image display apparatus 10 when the two input images illustrated in FIGS. 6A and 6B are input will be described. Here, the operation flow will be described with reference to the flowchart of FIG. 6A is an image in which 25% of the entire image area is a white patch (1023 gradations), and FIG. 6B is an image in which the entire area (100%) is a white patch (1023 gradations). Suppose that it is an image. It should be noted that portions other than the white patch in FIG.

(S1001)
An input image is input to the image composition unit 108. At this time, since the second image corresponding to the input image is not input to the image composition unit 108, the image composition unit 108 outputs the acquired input image to the backlight control unit 103. The input image may be input from an arbitrary device such as a PC, a portable terminal, a digital camera, or a server.

(S1002)
The backlight control unit 103 calculates an estimated light emission luminance EP (x, y) of each light source (x, y) from the input image that has been input. Here, x and y indicate the position of each light source in the backlight unit 102 as described above. Estimated emission luminance EP (x, y) indicates normalized emission luminance with the upper limit emission luminance of the light source as 100. The backlight control unit 103 calculates an average gradation value (APL) of pixels in the area of the input image corresponding to each light source, and sets EP (x, y) according to the value. For example, if the region corresponds to the white patch (1023 gradations) in FIGS. 6A and 6B, EP (x, y) = 100 is set, and the black region (0 in FIG. 6A) is set. EP (x, y) = 0 is set for (gradation). In such setting of the estimated emission luminance EP (x, y) from the average gradation value (APL) of the pixels in the region, the estimated emission luminance EP (x, y) is set in consideration of the gamma characteristic. Good. For example, in the case of the present embodiment, the estimated light emission luminance EP (x, y) may be calculated using the following Equation 4. The set estimated light emission brightness EP (x, y) is output to the loading control unit 104.

  In the case of the input image shown in FIG. 6A, 25% of the total light sources are estimated emission luminance EP (x, y) = 100, and 75% are estimated emission luminance EP (x, y) = 0. is there. On the other hand, in the case of the input image shown in FIG. 6B, the estimated emission luminance EP (x, y) of all the light sources is 100. Further, the estimated emission luminance EP (x, y) corresponding to the composite image may be adjusted so as not to take a larger value than the estimated emission luminance EP (x, y) corresponding to the input image. Specifically, when calculating the estimated emission luminance EP (x, y) of the input image, the estimated emission luminance EP (x, y) of the light source in the region corresponding to the first image is set to 100, and similarly to the second image. The estimated light emission luminance EP (x, y) of the light source in the region corresponding to may be fixed at 100.

(S1003)
The loading control unit 104 calculates the display luminance decrease rate L_gain from the input estimated light emission luminance EP (x, y) and the threshold Th. The calculated L_gain is output to the second generation unit 107. The calculation method is as shown in Equation 1 above, and the value of L_gain is calculated depending on whether or not the average value P_Ave of the estimated light emission luminances EP (x, y) of all the light sources is larger than the threshold Th. In this embodiment, the average of the estimated light emission luminance EP (x, y) in the range of 1 ≦ x ≦ 10 and 1 ≦ y ≦ 6 is P_Ave. Note that P_Ave does not have to be calculated from the average value of the estimated light emission luminances EP (x, y) of all the light sources, for example, the average value or the center of the estimated light emission luminances EP (x, y) of a predetermined number of light sources. It may be a value. Hereinafter, a method for calculating L_gain will be described for each of the cases where the input image is shown in FIGS. 6 (A) and 6 (B).

First, in the case of FIG. 6A, 25% takes 100 and 75% takes 0 out of the estimated light emission luminance EP (x, y) of all light sources, so P_Ave = 100 × 0.25 + 0 × 0.75 = 25 is calculated. Here, in this embodiment, since the threshold value Th = 50, P_Ave is a value equal to or less than the threshold value Th = 50. Therefore, L_gain = 1.0 can be calculated by Equation 1. That is, if L_gain = 1.0 in the input image of FIG. 6A, loading control is not performed, so 25% of the entire image shown in FIG. 8A is displayed at 1000 cd / m ² .

On the other hand, in the case of FIG. 6B, since the estimated emission luminance EP (x, y) of all the light sources is 100, P_Ave = 100 is calculated. Since P_Ave has a larger value than the threshold Th = 50, L_gain = Th / P_Ave = 50/100 = 0.5 can be calculated using Equation 1. Here, when loading control is performed on the input image of FIG. 6B with L_gain = 0.5, the entire image shown in FIG. 8B is uniformly displayed at a display luminance of 500 cd / m ² .

(S1004)
A reference luminance is input to the input unit 105 from the user. For example, when the OSD as shown in FIG. 3A is displayed, 0 to 300 can be input in units of 50 cd / m ² . The input reference luminance is output to the first generation unit 106. In the present embodiment, description will be made assuming that 100 cd / m ² is input as the reference luminance.

(S1005)
The first generation unit 106 first calculates a gradation value (N gradation value) corresponding to the input reference luminance. Here, the correspondence between the display luminance and the gradation value indicates the correspondence defined by the above-described PQ method, HGL method, or the like. In the present embodiment, the correspondence between the display luminance and the gradation value is defined by the gamma 2.2 method. Therefore, when the reference luminance is Lum1 and the gradation value is N, the correspondence is shown in Expression 5.

Here, in order to calculate the gradation value N from the reference luminances Lum1 and L_gain, the following equation 5 ′ obtained by modifying equation 5 may be used. Using this equation 5 ′, in the case of the input image shown in FIGS. 6A and 6B, Lum1 = 100 cd / m ² , and therefore N = 359 can be calculated.

  The first generation unit 106 generates a white patch image having the calculated N gradation values. That is, the first image is generated so that the R value, G value, and B value of each pixel take N gradation values. The size of the first image to be generated may be a size according to an instruction from the user, or a size obtained by reducing the size of the input image at a predetermined reduction rate. In addition, the image need not be a complete white patch, and an image in which the difference between the gradation value of each pixel and the N gradation value is within a predetermined threshold may be generated. You may produce | generate the thing which carried out the gradation conversion so that an average gradation value may take N. The first generation unit 106 outputs the generated first image to the image composition unit 108 and the calculated N gradation value to the second generation unit 107.

(S1006)
The second generation unit 107 calculates the gradation value (M gradation value) of the second image generated from the input L_gain and the N gradation value. Here, the M gradation value of the second image indicates the gradation value of the image that can be displayed with the reference luminance even when the loading control is performed. As a method for calculating the M gradation value from L_gain and the N gradation value, the above Equation 3 may be used. Note that it is not necessary to use the N gradation value, and if the reference luminances Lum1 and L_gain are used, the M gradation value can be calculated using Equation 6.

Here, in the case of the input image shown in FIG. 6A, since Lum1 = 100 cd / m ² and L_gain = 1.0, it is possible to calculate M = 359 as with the N gradation value. On the other hand, in the case of the input image shown in FIG. 6B, since Lum1 = 100 cd / m ² and L_gain = 0.5, M = 492 can be calculated.

  The second generation unit 107 generates an image of the white patch having the calculated M gradation value. That is, the second generation unit 107 generates the second image so that the R value, the G value, and the B value of each pixel take M gradation values. By calculating the M gradation value in consideration of L_gain in this way, the second image is displayed at the reference luminance even when the loading control is performed on the second image having the M gradation value in a later process. be able to. That is, the second image is an image that can be displayed with the reference luminance set in the converted display luminance range even if the display luminance range of the input image that is originally defined is converted by the loading control. The size of the second image to be generated is preferably the same as the size of the first image (the size that the user looks substantially the same), but the size of the user's input image is reduced by a predetermined reduction ratio. It may be size. In addition, the image need not be a complete white patch, and an image in which the difference between the gradation value of each pixel and the M gradation value is within a predetermined threshold value may be generated. An image obtained by gradation conversion so that the average gradation value of an image prepared in advance takes M may be generated. The second generation unit 107 outputs the generated second image to the image composition unit 108.

(S1007)
The input image, the first image, and the second image are input to the image composition unit 108. The image composition unit 108 generates a composite image so as to include (superimpose) the input image, the first image, and the second image. Although the position where the first image and the second image are arranged is arbitrary, it may be arranged in the vertical direction or the horizontal direction at the edge of the screen so that the user does not get in the way when viewing the input image.

  In the present embodiment, when the input image of FIG. 6A is input, the image composition unit 108 is the first image of the white patch whose gradation value is 359 as shown in FIG. 5A. The second image is arranged at the lower left of the screen. On the other hand, when the input image of FIG. 6B is input, the image composition unit 108, as shown in FIG. 5B, the first image having a gradation value of 359 and the second image having a gradation value of 492. The images are arranged side by side at the bottom left of the screen. The generated composite image is output to the backlight control unit 103 and the image quality correction unit 109.

  Hereinafter, S1008 to S1010 are processing steps for controlling the backlight unit 102, and S1011 to S1012 are processing steps for controlling the display panel 101. For this reason, S1008-S1010 and S1011-S1012 are good to be performed in parallel.

(S1008)
The backlight control unit 103 sets (acquires) the estimated light emission luminance EP (x, y) of each light source corresponding to each region of the composite image input from the image composition unit 108. The calculation method of the estimated light emission luminance EP (x, y) is the same as that in S1002 described above. Note that the estimated light emission luminance EP (x, y) of the light source corresponding to a region other than the region including the first image and the second image is not assumed to change, and the value set in S1002 may be used. Further, the estimated emission luminance EP (x, y) for all light sources may be set to the value set in S1002 without performing this step. The set estimated light emission brightness EP (x, y) is output to the loading control unit 104.

(S1009)
The loading control unit 104 calculates L_gain from the input estimated light emission luminance EP (x, y). The calculation method is the same as S1003. The loading control unit 104 multiplies the calculated L_gain and the estimated light emission luminance EP (x, y) to determine a control parameter BP (x, y) that is a value actually set for each light source of the backlight unit 102. That is, BP (x, y) = EP (x, y) × L_gain is established. Note that L_gain may be the value calculated in S1003 without being calculated in this step. By such processing, the display luminance range is narrowed, and the power consumption of the image display apparatus 10 can be suppressed. The determined control parameter BP (x, y) is output to the backlight unit 102.

(S1010)
The backlight unit 102 emits light at a light emission luminance corresponding to the input control parameter BP (x, y). That is, the backlight control unit 103 estimates the light emission luminance of each light source, and the backlight unit 102 emits light with the light emission luminance controlled by the loading control unit 104 based on the estimated light emission luminance.

(S1011)
The image quality correction unit 109 corrects the influence of light leakage caused by the difference in light emission amount for each region (light source) of the backlight unit 102 on the input composite image by image processing. The corrected composite image is output to the display panel 101.

(S1012)
The display panel 101 adjusts the transmittance of the panel based on the input composite image. The backlight unit 102 irradiates light on the adjusted panel, thereby displaying a composite image on the panel. If the input image of FIG. 6A is input, as shown in FIG. 9A, the input image that is not subjected to loading control, the first image of the reference luminance (100 cd / m ² ), and the first image Two images are displayed. On the other hand, in the case where the input image of FIG. 6B is input, as shown in FIG. 9B, the input image subjected to the loading control and the first image subjected to the loading control with respect to the image of the reference luminance are used. An image and a second image having a reference luminance are displayed. In FIG. 9B, since loading control is performed with L_gain = 0.5, the display luminance of the input image = 1000 × 0.5 = 500 cd / m ² .

  The operation flow of the image display apparatus 10 has been described above. Note that both the processing of S1002 and S1008 are performed by the backlight control unit 103, but may be performed by different functional units. Similarly, both the processing of S1003 and S1009 is performed by the loading control unit 104, but may be performed by different control units.

In addition, the first image and the second image may be superimposed on the input image, or the first image and the second image may be displayed in a black belt area where the input image is not displayed, such as a letter box. Good. In the present embodiment, the display unit 11 is a transmissive display using the transmissive display panel 101 and the backlight unit 102, but is not limited to this configuration. The display unit 11 may be a self-luminous display such as an OLED display using OLED (Organic Light Emitting Diode) as an element. That is, the display unit 11 may be a display that displays an image by a plurality of light sources self-emitting. Further, the display unit 11 may be a projection device that projects light onto the projection object by transmitting the light emitted from the backlight unit 102 through the display panel 101. In this embodiment, it can be said that the backlight control unit 103 and the loading control unit 104 are a control unit and a light source control unit in the image display apparatus 10, and the first generation unit 106 and the second generation unit 107 are the generation unit. I can say that.

  Note that the external device of the image display device 10 may store the characteristics of the loading control of the image display device 10 in advance and calculate L_gain according to the input image. In this case, the external device may generate the first image from the preset reference luminance and generate the second image from the reference luminance and L_gain. Furthermore, the external device may combine the input image, the first image, and the second image to generate a combined image and input it to the image display device 10. The image display apparatus to which the composite image has been input only needs to perform the control of the backlight unit 102 and the control of the display panel 101 in S1008 to S1012 in the flowchart of FIG.

[effect]
As described above, the user can intuitively grasp the change in the display luminance due to the loading control by comparing the brightness of the first image with the display luminance corresponding to the loading control and the second image with the reference luminance.

(Modification 1)
In the first embodiment, L_gain is calculated from the average value of the threshold Th and the estimated light emission luminance EP (x, y). However, L_gain may be calculated according to the characteristics of the loading control of the image display device shown in FIG. That is, the loading control unit 104 may determine L_gain according to the size of the area of the maximum gradation value (1023 gradations) in the input image. In FIG. 10, the horizontal axis represents the area (number of pixels) of the maximum gradation value (1023 gradations) of the pixels included in the input image, and the vertical axis represents L_gain. In the loading control of this modification, L_gain is set to be smaller as the ratio of the maximum gradation value in the image is larger. Note that the loading control of this modification is not limited to the above-described method depending on the maximum gradation value, and may be control using the number of pixels equal to or higher than a predetermined gradation value or APL, for example.

(Modification 2)
In the first embodiment, the reference luminance is set to a display luminance that can be displayed even when loading control is performed. On the other hand, in this modification, the reference luminance may be set from a luminance range (501 to 1000 cd / m ² in this embodiment) that may not be displayed by loading control. In this case, the ratio between the display brightness of the first image and the display brightness of the second image may be displayed in accordance with the degree of brightness reduction by loading control. For example, when the reference luminance is set to 1000 cd / m ² , L_gain = 0.5 when an input image of a white patch having an area of 100% shown in FIG. 6B is input. At this time, the second image must be displayed at a display luminance of 1000 cd / m ² , but even when 1023 which is the upper limit of the gradation value is set, the display luminance of the second image is 500 cd / million by the loading control. Displayed in m ² . In this case, the first generation unit 106 displays the first image after performing gradation conversion so that the display luminance is 0.5 times. That is, the image display apparatus 10 takes the display brightness of the first image as 250 cd / m ² and the display brightness of the second image as 500 cd / m ² (display brightness of the first image: display brightness of the second image = L_gain: 1.0 (set the tone value of the first image). By the above-described processing, the ratio of the display brightness of the first image and the display brightness of the second image is displayed in accordance with the degree of brightness reduction by loading control, so that the user can visually observe the brightness reduction by loading control. I can grasp.

<Embodiment 2>
In the first embodiment, a technique has been described in which a user can intuitively (visually) understand a change in display luminance due to loading control. In the second embodiment, even when the original brightness of the input image is not displayed by the gradation conversion process on the input image, the user can intuitively (visually) understand the change in display luminance due to the process. Will be described.

  The image display apparatus has a concern about an increase in power consumption and a decrease in reliability due to heat generation of components. For this reason, the original display luminance range of the image display apparatus may be narrower than the dynamic range (data luminance range) in which an image can be expressed. That is, the display luminance that originally corresponds to the gradation value of the input image may be outside the display luminance range of the image display device. As a method of displaying such an image by the image display device, there is a method of converting data corresponding to a luminance higher than the maximum luminance in the display luminance range of the image display device to the maximum luminance (clip processing). In addition, there is a method of compressing the entire data luminance range of an image (compression process). In the second embodiment, an image display apparatus that allows the user to intuitively understand the change in display luminance caused by the compression processing will be described.

  Specifically, in the present embodiment, when the data luminance range of the input image is displayed without being compressed, the first image and the second image are displayed with the same reference luminance on the image display device. Since the display brightness of the two images is the same, the user can determine that the input image is not compressed. Therefore, even when image processing is performed using an image display device, the user can intuitively recognize that the displayed image and the data of the image are consistent. On the other hand, when the data luminance range of the input image is compressed and displayed, the second image is displayed at the reference luminance on the image display device, and the first image is displayed darker than the reference luminance. Based on the difference in display brightness between the two images, the user can determine that the input image is being compressed. For this reason, when performing image processing using an image display device, the user can determine that the displayed image and the data of the image have lower luminance than the displayed image. Adjustment to raise can be suppressed.

Hereinafter, of the data luminance ranges, the luminance range corresponding one-to-one with the display luminance of the image display device is referred to as a display range (indicated luminance range). That is, for example, when the maximum value of the data the luminance is converted from 2000 cd / ^{m 2} to 1000 cd / ^{m 2} by a clip processing, display range is 1000 cd / ^{m 2.} On the other hand, when the maximum value of the data the luminance is converted from 2000 cd / ^{m 2} to 1000 cd / ^{m 2} by a compression process, the display range is 2000 cd / ^{m 2.} Therefore, it can be said that the display range is other than the data luminance range that is uniformly converted to the same data luminance by the clipping process.

[Configuration of image display device]
FIG. 11 is a block diagram of the image display device 20 according to the present embodiment. A second input unit 201 and a gradation conversion unit 202 are added to the configuration of the image display apparatus according to the first embodiment. That is, the image display device 20 includes the display unit 11, the backlight control unit 103, the loading control unit 104, the input unit 105, the first generation unit 203, the second generation unit 107, the image composition unit 108, the image quality correction unit 109, A two-input unit 201 and a gradation conversion unit 202 are provided. Further, the first generation unit 203 is different in processing from the function unit of the same name in the first embodiment. In addition, in order to perform the same process as the function part of the same name of Embodiment 1, in another function part, the same code | symbol is attached | subjected and description in this embodiment is abbreviate | omitted.

The image display apparatus 20 according to the present embodiment receives an HDR input image, and the input image is 0.
It shall have a data luminance range of ˜2000 cd / m ² . The display luminance range of the image display device 20 (the display luminance range that can be displayed by the display unit 11) is assumed to be 0 to 1000 cd / m ² as in the first embodiment. In the following, an example in which loading control does not occur will be described. However, even if loading control occurs, an image can be displayed in consideration of loading control and compression processing.

In the second input unit 201, the above-described display range is set by the user using an OSD as shown in FIGS. Note that the second input unit 201 may allow the user to set the maximum value of the display range in a range that is greater than or equal to the maximum value of the display luminance range and less than or equal to the maximum value of the data luminance range of the input image. In the present embodiment may be able to select between ^{1000cd / m 2 ~2000cd / m 2} . This is because the range up to 1000 cd / m ² is the luminance range that the image display device 20 can originally display. Further, the maximum value of the display range is not limited to being set by the user. For example, it may be always set to the maximum value of the data luminance range of the input image, or may be set to the maximum value of the display luminance range. Also good. The set display range is output to the gradation conversion unit 202 and the first generation unit 203. In this embodiment, it is assumed that the minimum value of the display range is uniformly 0 cd / m ² .

  The gradation conversion unit 202 performs gradation conversion on the input image based on the display range. The operation of the gradation conversion unit 202 will be described with reference to FIG. The gradation conversion unit 202 first determines what processing is to be performed on the input image.

For example, in the present embodiment, the following three patterns (1) to (3) are assumed for the determination. (1) 1000cd / ^{m 2} display is within the luminance range 0~1000cd / ^{m 2} of the image display apparatus 20 when it is set as the maximum value of the display range, it is determined that the clipping process.
(2) when the maximum value 2000 cd / ^{m 2} data luminance range 0~2000cd / ^{m 2} of the input image is set as the maximum value of the display range is determined to carry out the compression process.
(3) When 1500 cd / ^{m 2} that exists between the two values is set as the maximum value of the display range, after processing of the 1500 cd / ^{m 2} larger data luminance 1500 cd / ^{m 2,} overall 1000 cd / It is determined that the compression process is performed so as to fall within m ² .

The gradation conversion unit 202 performs gradation conversion processing on the input image according to the determination result. In the case of (1) described above, the gradation conversion unit 202 converts data luminance higher than 1000 cd / m ² in the input image into 1000 cd / m ² . In the case of (2) described above, the gradation conversion unit 202 compresses the entire dynamic range of the input image by 0.5 times. That is, the gradation conversion unit 202 multiplies the data brightness of the input image by 0.5. In short, in the cases (1) and (2) above, the relationship between the input data luminance and the data luminance after gradation conversion shown in FIG. 12 is established. In the case of (3) described above, after the clipping process in which the data luminance greater than 1500 cd / m ² in the input image is set to 1500 cd / m ² , the entire dynamic range of the clipped input image is compressed 2/3 times. Note that the gradation conversion unit 202 outputs the magnification when the compression processing is performed in the above (2) and (3) to the first generation unit 203 as C_gain. That is, C_gain = 0.5 is output to the first generation unit 203 in the case of (2), and C_gain = 2/3 is output in the case of (3). Here, it can be said that C_gain is a ratio between the maximum value in the display luminance range of the image display device 20 and the maximum value in the display range. Note that the input image that has undergone gradation conversion is output to the image composition unit 108. If the input image is not subjected to compression processing, C_gain = 1.0 may be output to the first generation unit 203. In addition, when the display range is not instructed by the user, the input image may be subjected to compression processing corresponding to (2) without performing the above determination.

For gradation conversion by compression processing, for example, an LUT (Look Up Table) indicating correspondence between data luminance and gradation value in an input image is prepared in advance, and the original data luminance and C_
This is possible by selecting a gradation value corresponding to a value obtained by multiplying gain from the LUT. For gradation conversion by clip processing, the same LUT may be used, for example, processing for converting a gradation value corresponding to a value higher than a predetermined data luminance into a gradation value corresponding to the predetermined data luminance. .

The first generation unit 203 first calculates an N gradation value from the reference luminance as in the first embodiment. In the present embodiment, if the reference luminance is 100 cd / m ² as in the first embodiment, the N gradation value is 359 as well. Next, based on the calculated N gradation value and the inputted C_gain, a gradation value (N_1 gradation) reflecting the compression processing is calculated for the first image based on the following Expression 7.

  The first generation unit 203 generates a white patch in which the R value, the G value, and the B value are generated with N_1 gradations = 262, respectively, as the first image reflecting the luminance reduction due to the compression process. As in the first embodiment, the first generation unit 203 outputs the first image to the image composition unit 108 and the N gradation value to the second generation unit 107. Other functional units are the same as those in the first embodiment.

[Processing flow of image display device]
In the following, the image display device 20 is set as an example where the maximum value of the display range is set to 1000 cd / m ² as shown in FIG. 3B or 2000 cd / m ² as shown in FIG. The operation flow will be described. Hereinafter, the operation flow will be described with reference to FIG. 13, but the processing in S1002 to S1004 and S1006 to S1012 is the same as that in the first embodiment, and thus the description thereof is omitted.

(S2001)
The gradation conversion unit 202 acquires an input image. Note that the data luminance range of the input image is acquired from meta information or the like included in the acquired input image.

(S2002)
The second input unit acquires a display range. In this processing flow, 1000 cd / m ^{2 as} shown in FIG. 3B or 2000 cd / m ² as shown in FIG. 3C is set as the maximum value of the display range as described above. The set display range is output to the first generation unit 203 and the gradation conversion unit 202. In other words, in the case of FIG. 3 (B), the display range of the 0~1000cd / ^{m 2,} in the case of FIG. 3 (C), the display range of the 0~2000cd / ^{m 2} is set.

(S2003)
The gradation conversion unit 202 performs gradation conversion of the input image based on the display range. Specifically, the data luminance value of the input image is converted. If the maximum value of the display range of 1000 cd / ^{m 2,} as shown in FIG. 3 (B) is set, with respect to pixels having 1000 cd / ^{m 2} larger data luminance, the data luminance 1000 cd / ^{m 2} Clip processing is performed. When the maximum value of the display range of 2000 cd / m ² is set as shown in FIG. 3C, compression processing is performed to uniformly multiply the data luminance by 0.5. The processed input image is the image composition unit 108.
Is output. Further, when the input image is subjected to compression processing, the gradation conversion unit 202 outputs the compression magnification to the first generation unit 203 as C_gain. In the case of FIG. 3C, C_gain = 0.5 is output to the first generation unit 203. When the compression process is not performed, C_gain = 1.0 may be output to the first generation unit 203.

(S2004)
The first generation unit 203 first calculates an N gradation value from the reference luminance as in the first embodiment. The calculated N gradation value is output to the second generation unit 107. Subsequently, the first generation unit 203 calculates an N_1 gradation value from the N gradation value. Here, the N_1 gradation value is a gradation value corresponding to the case where the same compression process as the compression process performed on the input image is performed on the image having the N gradation value. Since the calculation method of the N_1 gradation value has been described above, the description is omitted. The first generation unit 203 generates a white patch image having the calculated N_1 gradation value and outputs the white patch image to the image composition unit 108. Note that the size of the first image may be the same as that generated in the first embodiment.

  With the above processing flow, the gradation conversion unit 202 controls the display brightness of the first image and the input image. From the above-described processing, the display brightness of the first image and the input image is reduced according to the ratio (C_gain) between the maximum value of the input image in the display brightness range of the image display device 20 and the maximum value in the display range. When there is no input of the display range, the display brightness of the first image and the input image depends on the ratio between the maximum value in the display brightness range of the image display device 20 and the maximum value in the data brightness range of the input image. Will be reduced.

  In the present embodiment, the image display apparatus 20 that can perform the loading control process as in the first embodiment has been described. However, the loading control is not necessarily performed, and the loading control unit 104 used for the loading control is used. Is not required. For example, in this embodiment, L_gain may always be 1.0. In the present embodiment, the backlight control unit 103, the loading control unit 104, and the gradation conversion unit 202 can be said to be control means in the image display device 20.

(effect)
As described above, for example, when the maximum display range value 2000 cd / m ² shown in FIG. 3C is input, the first image is displayed with the luminance (50 cd / m ² ) corresponding to the compression process, and the second image is displayed. Is always displayed at the reference luminance (100 cd / m ² ) regardless of the compression processing. In addition, even when loading control is performed on an input image as in the first embodiment, it is possible to display the first image and the second image with display luminance that consider both loading control and compression processing. Thereby, in addition to loading control, the user can grasp | ascertain intuitively the variation | change_quantity of the reference | standard brightness | luminance by compression processing.

  In addition, each hardware part of each said embodiment and each modification may be individual hardware, and may not be so. The functions of two or more functional units may be realized by common hardware. Each of a plurality of functions of one functional unit may be realized by individual hardware. Two or more functions of one functional unit may be realized by common hardware. Each functional unit may or may not be realized by hardware such as an ASIC, FPGA, or DSP. For example, the apparatus may include a processor and a memory (storage medium) in which a control program is stored. The functions of at least some of the functional units included in the apparatus may be realized by the processor reading and executing the control program from the memory.

(Other embodiments)
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

10: Image display device 101: Display panel 102: Backlight unit 104: Loading control unit

Claims (20)

Display means for displaying a composite image including the input image, the first image, and the second image;
Control means for performing brightness conversion for reducing display brightness of the composite image displayed by the display means;
With
The second image is an image displayed at a reference luminance that is a luminance for the user to confirm the correspondence between the gradation value of the image and the display luminance.
The first image is an image displayed at a display brightness changed from the reference brightness in accordance with the brightness conversion.
A display device characterized by that. Generating means for generating the first image and the second image;
Further comprising
The generating means includes
Generating the first image whose gradation value does not change by the luminance conversion;
The second image is generated by changing a gradation value so that display luminance does not change by the luminance conversion.
The display device according to claim 1. The generating means generates the first image having the same gradation value of each pixel and the second image having the same gradation value of each pixel;
The display device according to claim 2. The display means has a plurality of light sources that emit light,
The control means includes light source control means for performing the brightness conversion by controlling light emission brightness of the plurality of light sources.
The display device according to claim 1, wherein the display device is a display device. The display luminance range that can be displayed by the display means is a first luminance range,
The light source control means performs the luminance conversion for converting the display luminance of the first luminance range to the display luminance of a second luminance range that is a range included in the first luminance range.
The display device according to claim 4. The reference luminance is a display luminance included in the second luminance range.
The display device according to claim 5. Each of the light sources corresponds to a region including a plurality of pixels of the input image,
The light source control means of the control means obtains the estimated light emission luminance of the light source corresponding to the region from the gradation values of the plurality of pixels included in the region, and averages the estimated light emission luminance of the plurality of light sources. Is greater than a predetermined threshold value, the luminance conversion for reducing the display luminance of the composite image is performed.
The display device according to claim 4, wherein the display device is a display device. The light source control means of the control means performs the brightness conversion for further reducing the display brightness of the composite image as the average value of the estimated light emission brightness of the plurality of light sources is larger.
The display device according to claim 7. The predetermined threshold is a value corresponding to an upper limit value of power that can be supplied to the display device.
The display device according to claim 7 or 8, wherein The display device according to any one of claims 4 to 9, wherein the display unit further includes a display panel that transmits light emitted from the plurality of light sources and displays an image. The display unit further includes a projection unit that transmits light emitted from the plurality of light sources and projects an image onto a projection target.
The display device according to claim 4, wherein the display device is a display device. The plurality of light sources display images by self-emission,
The display device according to claim 4, wherein the display device is a display device. The apparatus further comprises input means for inputting the reference luminance from a user.
The display device according to claim 1, wherein the display device is a display device. The first image and the second image are the same size,
The first image and the second image in the composite image are arranged in a vertical direction or a horizontal direction,
The display device according to claim 1, wherein the display device is a display device. The control means has gradation conversion means for reducing the display brightness of the image by reducing the gradation value of the image,
The display luminance range that can be displayed by the display means is a first luminance range,
The gradation converting means, when the gradation value of the input image corresponds to a third luminance range larger than the first luminance range,
Reducing the display brightness of the input image and the display brightness of the first image according to the ratio of the maximum value in the first brightness range and the maximum value in the third brightness range;
The display device according to claim 1, wherein the display device is a display device. A second input means for inputting an instruction luminance range from the user;
The control means has gradation conversion means for reducing the display brightness of the image by reducing the gradation value of the image,
The display luminance range that can be displayed by the display means is a first luminance range,
The gradation converting means, when the indicated luminance range is larger than the first luminance range,
Reducing the display brightness of the input image and the display brightness of the first image in accordance with the ratio of the maximum value in the first brightness range and the maximum value in the indicated brightness range;
The display device according to claim 1, wherein the display device is a display device. The gradation converting means, when the indicated luminance range is larger than the first luminance range,
Reducing the display brightness greater than the maximum value of the indicated brightness range in the input image to the maximum value of the indicated brightness range;
Further, the display brightness of the input image and the display brightness of the first image are reduced according to the ratio between the maximum value in the first brightness range and the maximum value in the indicated brightness range.
The display device according to claim 16. A display step of displaying a composite image including the input image, the first image, and the second image;
A control step of performing luminance conversion for reducing display luminance of the composite image displayed by the display step;
Including
The second image is an image displayed at a reference luminance that is a luminance for the user to confirm the correspondence between the gradation value of the image and the display luminance.
The first image is an image displayed at a display brightness changed from the reference brightness in accordance with the brightness conversion.
A display method characterized by that.   The program for making a computer perform each process of the display method of Claim 18.   A computer-readable storage medium storing the program according to claim 19.

Images