JP2019080156A - Image processing method and image processing apparatus - Google Patents

Abstract

To provide a technology that enables obtaining a high-quality display image even when clip processing and OOTF processing are executed.SOLUTION: An image processing method according to the present invention includes a first acquisition step of acquiring first information on a first range which is a luminance range of input image data, a second acquisition step of acquiring second information on a second range which is a luminance range of a display target, a first conversion step of generating first converted image data by executing first conversion processing of converting the luminance range of the input image data into a luminance range equal to the display luminance range on the basis of the second information, and a second conversion step of generating second converted image data by executing second conversion processing of converting the luminance of the first converted image data on the basis of the first information and the second information, and in the second conversion step, the second conversion processing is executed such that the luminance of the first converted image data is reduced by a large reduction amount as the maximum luminance in the second range decreases.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing method and an image processing apparatus.

  In recent years, with the improvement of the light receiving performance of an imaging device, BT. Photographed image data having a dynamic range wider than the dynamic range (brightness range) of 709 has come to be generated. A wide dynamic range is also called "HDR (High Dynamic Range)" or the like, and image data having HDR is also called "HDR image data" or the like. Meanwhile, BT. The dynamic range of 709 is also called "SDR (Standard Dynamic Range)" or the like, and the image data having SDR is also called "SDR image data" or the like. As a data format of HDR image data, for example, Cineon Log defined based on characteristics of a film having a wide dynamic range has been proposed.

  Here, it is assumed that photographed image data is SDR image data and an image based on the SDR image data is displayed. In this case, generally, since the gamma value of the imaging device is about 0.5 and the gamma value of the display device is about 2.4, the incident light (subject light) of the imaging device and the output of the display device The correspondence with the light (screen light) is non-linear. Specifically, the above correspondence relationship has a downward convex characteristic in the graph in which the horizontal axis indicates the brightness of the subject light and the vertical axis indicates the brightness of the screen light. The function of light conversion such as conversion from subject light to screen light is also called "OOTF (Opto-Optical Transfer Function)" or the like.

Various schemes of HDR have been proposed, and OOTF processing is performed in various schemes of HDR. The OOTF processing is image processing using the OOTF, and is also called "system gamma processing" or the like. For example, ITU-R (Radio communication Sector of ITU) BT. In 2100, the HLG (Hybrid Log-Gamma) method is defined as the HDR method. The HLG-based OOTF process is a gamma conversion process using a gamma value γ obtained by Equation 1 below. The gamma value γ is also called “system gamma” or the like. In Equation 1, “Lw” is the upper limit of the display luminance (luminance of the screen; luminance of the screen light) of the display device. Therefore, in the HLG method, gamma conversion processing is performed as the OOTF processing to reduce data luminance (luminance of image data) by a large reduction amount as the upper limit display luminance (upper limit of display luminance) Lw increases. Data brightness can be said to be "brightness assumed in image data".

γ = 1.2 + 0.42 LOG ₁₀ (Lw / 1000) (Equation 1)

In the display device, there is a concern that power consumption is increased, reliability is decreased due to heat generation of components, and the like. Therefore, the display brightness range (brightness range from the lower limit to the upper limit of the display brightness) may be narrower than the dynamic range (data brightness range) of the image data. A display method in the case where the display luminance range is narrower than the dynamic range of the image data is disclosed, for example, in Patent Document 1. In the display method disclosed in Patent Document 1, the dynamic range of the image data is compressed to the display luminance range using the compression parameter corresponding to the instruction from the user. As another method, there is a method of converting data luminance higher than the upper limit display luminance Lw into the upper limit display luminance Lw without converting data luminance corresponding to the display luminance range (clip processing). There is also a method of compressing the entire dynamic range of image data (compression processing). When the clip processing is performed, the image is displayed at a display luminance higher than the display luminance when the compression processing is performed.

JP 2004-220438 A

  However, in the prior art, when clip processing and OOTF processing are performed, a high quality display image (image displayed on the screen) may not be obtained. Specifically, as described above, when the clip processing is performed, the image is displayed at a display luminance higher than the display luminance when the compression processing is performed. Since the upper limit display luminance Lw does not depend on image processing such as clip processing and compression processing, the same OOTF processing as the OOTF processing when compression processing is performed is performed when clip processing is performed. Therefore, when clip processing is performed, an image may be displayed with display luminance higher than necessary. Then, an image area corresponding to the data luminance range from the low luminance side to the intermediate luminance may be displayed with a display luminance higher than necessary, and the contrast of the display image may be reduced.

  An object of the present invention is to provide a technology that makes it possible to obtain a high-quality display image even when clip processing and OOTF processing are performed.

The first aspect of the present invention is
A first acquisition step of acquiring first information on a first range which is a luminance range of input image data;
A second acquisition step of acquiring second information related to a second range which is a luminance range to be displayed;
A first conversion step of generating first converted image data by performing a first conversion process of converting a luminance range of the input image data into a luminance range equal to a display luminance range based on the second information;
A second conversion step of generating second converted image data by performing a second conversion process of converting the luminance of the first converted image data based on the first information and the second information;
Have
In the second conversion step, the second conversion processing is performed such that the luminance of the first converted image data is reduced by a large reduction amount as the maximum luminance of the second range decreases. It is a processing method.

The second aspect of the present invention is
A first acquisition step of acquiring first information on a first range which is a luminance range of input image data;
A second acquisition step of acquiring second information related to a second range which is a luminance range to be displayed;
A first conversion step of generating first converted image data by performing conversion processing of converting the luminance of the input image data based on the first information and the second information;
Have
The image processing method is characterized in that, in the first conversion step, the conversion processing is performed such that the luminance of the input image data is reduced by a large reduction amount as the maximum luminance of the second range decreases. .

The third aspect of the present invention is
First acquisition means for acquiring first information on a first range which is a luminance range of input image data;
A second acquisition unit that acquires second information related to a second range that is a luminance range to be displayed;
First conversion means for generating first converted image data by performing a first conversion process of converting a luminance range of the input image data into a luminance range equal to a display luminance range based on the second information;
Second conversion means for generating second converted image data by performing a second conversion process of converting the luminance of the first converted image data based on the first information and the second information;
Have
The image is characterized in that the second conversion processing is performed such that the luminance of the first converted image data is reduced by a large reduction amount as the maximum luminance of the second range decreases. It is a processing device.

The fourth aspect of the present invention is
First acquisition means for acquiring first information on a first range which is a luminance range of input image data;
A second acquisition unit that acquires second information related to a second range that is a luminance range to be displayed;
Conversion means for generating converted image data by performing conversion processing for converting the luminance of the input image data based on the first information and the second information;
Have
The conversion means is an image processing apparatus characterized in that the conversion processing is performed such that the luminance of the input image data is reduced by a large reduction amount as the maximum luminance of the second range is lower.

  A fifth aspect of the present invention is a program for causing a computer to execute the steps of the image processing method described above.

  According to the present invention, it is possible to obtain a high-quality display image even when clip processing and OOTF processing are performed.

An example of light conversion in the HLG method according to the first embodiment Example of Configuration of Display Device According to Embodiment 1 An example of gradation characteristics of input image data according to the first embodiment Example of gradation characteristics of linear image data according to the first embodiment Example of gradation characteristics of range converted image data according to the first embodiment Example of Display Characteristics According to Embodiment 1 An example of the correspondence between the luminance HRange and PeakLum_2 according to the first embodiment Example of Correspondence between Brightness H Range and Gamma Value γ According to Embodiment 1 An example of the conversion characteristic of the OOTF process according to the first embodiment An example of the effect of the first embodiment An example of light conversion in the PQ method according to the second embodiment Configuration Example of Display System According to Second Embodiment

Example 1
Hereinafter, Example 1 of the present invention will be described. The method of HDR (High Dynamic Range) is not particularly limited, but in the present embodiment, ITU-R (Radio communication Sector of ITU) BT. HL specified by 2100
An example of the G (Hybrid Log-Gamma) method will be described. HDR is a BT. The dynamic range is wider than the dynamic range (brightness range) of 709.

  FIG. 1 shows an example of light conversion in the HLG method. As shown in FIG. 1, in the HLG method, scene brightness (brightness of an object) is converted into a gradation value (Opto-Electronic Transfer Function (OETF) processing). Next, the tone values are returned to the scene brightness (Inverse OETF processing). Then, the scene luminance is converted into display luminance (luminance of the screen) (Opto-Optical Transfer Function (OOTF) processing). The OETF processing is performed by the imaging device, and the Inverse OETF processing and the OOTF processing are performed by the display device.

  FIG. 2 is a block diagram showing a configuration example of the display device 100 according to the present embodiment. The display device 100 includes an image processing unit 101, a display unit 102, a dynamic range acquisition unit 103, and a display range acquisition unit 104. The image processing unit 101 includes a linear conversion unit 105, a range conversion unit 106, and an OOTF conversion unit 107.

  The image processing unit 101 performs image processing on input image data (image data input to the display device 100), and the display unit 102 displays an image based on the image data after the image processing on a screen. As the display unit 102, for example, a combination of a light emitting unit and a transmissive display panel that transmits light emitted from the light emitting unit based on image data can be used. A liquid crystal panel, a micro electro mechanical system (MEMS) shutter type display panel, or the like can be used as the transmissive display panel. The light emitting unit can be said to be a "backlight unit". As the display unit 102, a self-luminous display panel such as an organic EL (Electro Luminescence) display panel or a plasma display panel can also be used. Although the upper limit display luminance (upper limit of display luminance) is not particularly limited, in the present embodiment, the upper limit display luminance is 500 cd / m 2. Although the data format of the input image data is not particularly limited, in the present embodiment, the input image data has the gradation characteristic of the HLG method. Specifically, in the present embodiment, the input image data is image data obtained by the OETF process of FIG.

  In the present embodiment, the input image data has the gradation characteristics of FIG. The horizontal axis in FIG. 3 indicates the luminance (data luminance) of the image data, and the vertical axis in FIG. 3 indicates the gradation values (R value, G value, B value, Y value, etc.) of the image data. In the tone characteristic of FIG. 3, the tone value increases logarithmically with the increase in data luminance. Data brightness can be said to be "brightness assumed in image data". The data brightness in FIG. 3 corresponds to the reflectance when light is reflected by the object. The luminance of a light receiving object illuminated by ambient light is expressed with a reflectance of about 0 to 100%, and the luminance above that (the luminance of a light emitting object such as a lighting device, the sun, etc.) is expressed with a reflectance higher than 100% It is often done. In this embodiment, the dynamic range of input image data is a brightness range (data brightness range) of 0% or more of data brightness (reflectance) and 1000% or less of data brightness, and the gradation value of input image data is 10 bits. The value of (0 to 1023). Note that a value different from the reflectance may be used as the data luminance.

The dynamic range acquisition unit 103 acquires dynamic range information on the dynamic range of input image data. In the present embodiment, the dynamic range acquisition unit 103 acquires dynamic range information according to the display setting of the display device 100. Specifically, a plurality of pieces of dynamic range information respectively corresponding to a plurality of display settings are predetermined, and the dynamic range acquisition unit 103 acquires dynamic range information corresponding to the current display setting from the plurality of dynamic range information. Do. For example, BT. In the case of the display setting corresponding to 709, the dynamic range acquisition unit 103 acquires dynamic range information indicating a dynamic range of data brightness 0% or more and data brightness 100% or less. In the case of the display setting corresponding to HDR, the dynamic range acquisition unit 103 acquires dynamic range information indicating a dynamic range wider than the dynamic range of data brightness 0% or more and data brightness 100% or less. Display settings are performed automatically or manually. As described above, in the present embodiment, the dynamic range of input image data is a data brightness range of data brightness 0% or more and data brightness 1000% or less. Then, the dynamic range acquisition unit 103 acquires dynamic range information indicating a dynamic range of data brightness 0% or more and data brightness 1000% or less.

  In addition, the acquisition method of dynamic range information is not specifically limited. For example, dynamic range information may be acquired by analyzing input image data. When dynamic range information is included in metadata of input image data, dynamic range information may be acquired from metadata of input image data.

  The display range acquisition unit 104 acquires display range information regarding a display range which is a data luminance range to be displayed. The display range can be said to be "data luminance range for performing display with gradation". In the present embodiment, the display range acquisition unit 104 acquires information designated by the user as display range information. Specifically, the display range acquisition unit 104 acquires (generates) display range information indicating the specified display range in accordance with a user operation specifying the display range. Although the display range is not particularly limited, in the present embodiment, the display range is part or all of the dynamic range of the input image data. In addition, the acquisition method of display range information is not specifically limited. For example, display range information may be automatically acquired according to input image data, a use environment of the display device 100, and the like.

  The linear conversion unit 105 converts the input image data into linear image data in which the data brightness increases linearly with an increase in tone value. That is, the linear conversion unit 105 converts the gradation characteristic of the input image data into the gradation characteristic (FIG. 4) in which the data luminance increases linearly with the increase of the gradation value. The conversion from input image data to linear image data is realized by image processing (tone conversion processing; linear conversion processing) for converting tone values of the image data. The linear transformation process corresponds to the Inverse OETF process of FIG. The display device 100 (image processing unit 101) may not have the linear conversion unit 105.

  The range conversion unit 106 converts the linear image data generated by the linear conversion unit 105 into range conversion image data based on the display range information acquired by the display range acquisition unit 104. Specifically, the range conversion unit 106 performs image processing (tone conversion processing; range conversion processing) for converting the dynamic range of the linear image data (the dynamic range of the input image data) into the dynamic range equal to the display luminance range. . Thereby, range conversion image data is generated. The display luminance range is a luminance range from the lower limit to the upper limit of the display luminance. As shown in FIG. 2, the range conversion unit 106 may acquire dynamic range information from the dynamic range acquisition unit 103 and use it.

  The range conversion process will be described in detail with reference to FIG. FIG. 5 shows an example of the gradation characteristic of the range conversion image data.

In the present embodiment, the range conversion unit 106 converts the gradation value InputLevel_1 of the linear image data into the gradation value OutputLevel_1 of the range conversion image data, using Expression 2 below. However, when the gradation value OutputLevel_1 larger than 1023 is obtained (calculated) by Expression 2, 1023 is set as the gradation value OutputLevel_1. In Equation 2, “DRange” is the maximum luminance (maximum data luminance) of the dynamic range of the input image data, and “HRange” is the maximum data luminance of the display range.

OutputLevel_1 =
(DRange ÷ HRange) × InputLevel_1 (Equation 2)

  If the display range is a data brightness range of data brightness 0% or more and data brightness 500% or less by range conversion processing, range converted image data having the gradation characteristic 501 of FIG. 5 is generated. In the gradation characteristic 501, the gradation value linearly increases from the lower limit 0 to the upper limit 1023 with respect to an increase in data luminance from the minimum luminance (minimum data luminance) 0% to the maximum data luminance 500% in the display range. The upper limit 1023 of the gradation value is associated with the data luminance higher than the maximum data luminance 500% of the display range. On the other hand, when the display range is a data brightness range of data brightness 0% or more and data brightness 1000% or less, range conversion image data having the gradation characteristic 502 of FIG. 5 is generated. In the gradation characteristic 502, the gradation value linearly increases from the lower limit 0 to the upper limit 1023 with respect to the increase in data luminance from the minimum data luminance 0% to the maximum data luminance 1000% in the display range.

  FIG. 6 shows an example of display characteristics when the image processing of the OOTF converter 107 is omitted. The horizontal axis of FIG. 6 indicates data brightness, and the vertical axis of FIG. 6 indicates display brightness.

When the display range is a data brightness range of data brightness 0% or more and data brightness 500% or less, an image is displayed with the display characteristic 601 of FIG. 6 according to the gradation characteristic 501 of FIG. In the display characteristic 601, the display brightness increases linearly from the lower limit 0 cd / ^{m 2} to an upper limit 500 cd / ^{m 2} with an increase in the data the luminance from the minimum data luminance of 0% of the display range up to 500% maximum data luminance. That is, in the display range (0% to 500%), display with gradation is performed. The data brightness higher than the maximum data brightness 500% of the display range is displayed at the upper limit display brightness 500 cd / m ² . That is, for the data luminance range higher than the maximum data luminance 500% of the display range, display with no gradation is performed.

On the other hand, when the display range is a data luminance range of data luminance 0% or more and data luminance 1000% or less, an image is displayed with the display characteristic 602 of FIG. 6 according to the gradation characteristic 502 of FIG. In the display characteristic 602, the display brightness increases linearly from the lower limit 0 cd / ^{m 2} to an upper limit 500 cd / ^{m 2} with an increase in the data the luminance from the minimum data luminance of 0% of the display range up to 1000% maximum data luminance. In the case of the display characteristic 602, since display without giving gradation is not performed, it is easier to confirm the gradation of the entire dynamic range of the input image data than in the case of the display characteristic 601. However, in the case of the display characteristic 601, the tonality of the data luminance range in which the display with gradation is performed is easier to confirm than in the case of the display characteristic 602.

In the present embodiment, it is assumed that the data brightness X% corresponds to the absolute brightness Xcd / m ² . It is understood from FIG. 6 that the dynamic range (0% to 1000%) of the linear image data is converted to the dynamic range (0 cd / m ^{2 to} 500 cd / m ² ) equal to the display luminance range by the range conversion process. The range conversion processing when the maximum data luminance of the display range is lower than the maximum data luminance of the input image data, that is, the range conversion processing for realizing display characteristics such as the display characteristic 601 can also be referred to as “clip processing”. On the other hand, range conversion processing in the case where the maximum data luminance of the display range is equal to the maximum data luminance of the input image data, that is, range conversion processing for realizing the display characteristic 602 can be said to be “compression processing”. It can also be understood from FIG. 6 that, when clip processing is performed, an image is displayed with display luminance higher than display luminance when compression processing is performed.

The OOTF conversion unit 107 generates display image data by performing image processing (tone conversion processing) for converting data luminance of the range conversion image data generated by the range conversion unit 106. In the present embodiment, the OOTF conversion unit 107 converts range converted image data into display image data based on the dynamic range information acquired by the dynamic range acquisition unit 103 and the display range information acquired by the display range acquisition unit 104. Convert. The tone conversion processing of the OOTF converter 107 corresponds to the OOTF processing of FIG. The display image data is input to the display unit 102, and the display unit 102 displays an image based on the display image data on the screen. The OOTF processing may be tone conversion processing for individually converting each of the R value, the G value, and the B value, or may be tone conversion processing for converting the Y value.

  The process of the OOTF converter 107 will be described in detail.

First, the OOTF converter 107 obtains the luminance PeakLum_2 using the following Equation 3 (FIG. 6). In Equation 3, “PeakLum_1” is the actual upper limit display luminance, which is 500 cd / m ² in this embodiment. The luminance PeakLum_2 can also be said to be the "ideal upper limit display luminance". According to Equation 3, the luminance PeakLum_2 is the luminance obtained by multiplying the ratio of the maximum data luminance DRange of the dynamic range of the input image data to the maximum data luminance HRange of the display range by the upper limit display luminance PeakLum_1. The numerical value of the upper limit display luminance PeakLum_1 may or may not be described in advance in a program or the like. For example, the display device 100 may have a storage unit that stores display luminance information on the upper limit display luminance PeakLum_1. Then, the OOTF conversion unit 107 may acquire display luminance information from the storage unit.

PeakLum_2 = DRange ÷ HRange × PeakLum_1
... (Equation 3)

  FIG. 7 shows an example of the correspondence between the maximum data luminance HRange and the luminance PeakLum_2. FIG. 7 shows an example where the maximum data luminance DRange is fixed. The upper limit display luminance PeakLum_1 is fixed. It can be seen from FIG. 7 that the lower the maximum data luminance HRange, the higher the luminance PeakLum_2. According to Equation 3, as the ratio of the maximum data luminance DRange to the maximum data luminance HRange is larger, a higher luminance PeakLum_2 is obtained.

Next, the OOTF converter 107 obtains the gamma value γ using Equation 4 below. The HLG-type OOTF process is a gamma conversion process, and the equation 4 is approximately equal to the equation defined to obtain the gamma value used in the HLG-type OOTF process. Conventionally, the upper limit display luminance PeakLum_1 is used as the luminance Lw, but in the present embodiment, the luminance PeakLum_2 is used as the luminance Lw.

γ = 1.2 + 0.42 LOG ₁₀ (Lw / 1000) (Equation 4)

  FIG. 8 shows an example of the correspondence between the maximum data luminance HRange and the gamma value γ. FIG. 8 shows an example where the maximum data luminance DRange is fixed. It can be seen from FIG. 8 that the lower the maximum data luminance HRange, the larger the gamma value γ. According to Equation 4, a larger gamma value γ is obtained as the luminance Lw = PeakLum_2 is higher. As described above, as the ratio of the maximum data luminance DRange to the maximum data luminance HRange is larger, higher luminance PeakLum_2 is obtained. Therefore, the larger the ratio of the maximum data luminance DRange to the maximum data luminance HRange, the larger the gamma value γ can be obtained.

Then, the OOTF conversion unit 107 performs gamma conversion processing using the gamma value γ as the OOTF processing, as shown in the following Equation 5. In Expression 5, “InputLevel_2” is a tone value of range conversion image data, and “OutputLevel_2” is a tone value of display image data.

OutputData_2 =
(InputData_2 ÷ 1023) ^γ × 1023 ... (Equation 5)

  FIG. 9 shows an example of the conversion characteristic of the OOTF process. The horizontal axis in FIG. 9 indicates the input tone value (tone value InputLevel_2) in the OOTF process, and the vertical axis in FIG. 9 indicates the output tone value (tone value OutputLevel_2) in the OOTF process. It can be seen from FIG. 9 that the conversion characteristic of the OOTF process is a downward convex characteristic. Further, it can also be seen that the conversion characteristic of the OOTF process is a characteristic which sinks down significantly as the gamma value γ increases. Therefore, in the OOTF process, the data luminance of the range conversion image data is reduced by a large reduction amount as the gamma value γ is larger.

  As described above, the higher the luminance Lw = PeakLum_2, the larger the gamma value γ. Therefore, in the OOTF process, as the luminance Lw = PeakLum_2 is higher, the data luminance of the range conversion image data is reduced by a large reduction amount. In addition, as described above, the larger the ratio of the maximum data luminance DRange to the maximum data luminance HRange, the larger the gamma value γ. Therefore, in the OOTF process, as the ratio of the maximum data luminance DRange to the maximum data luminance HRange is larger, the data luminance of the range conversion image data is reduced by a large reduction amount. Then, as described above, when the maximum data luminance DRange of the dynamic range of the input image data is fixed, the gamma value γ is larger as the maximum data luminance HRange of the display range is lower. Therefore, when the maximum data luminance DRange is fixed, the data luminance of the range conversion image data is reduced by a large reduction amount as the maximum data luminance HRange is lower.

  In the present embodiment, in the OOTF process, the reduction amount of the data brightness of the range conversion image data is controlled based on the maximum data brightness HRange of the display range and the like. As a result, even when clip processing and OOTF processing are performed, a high quality display image can be obtained. For example, it can be suppressed that an image is displayed at a display luminance higher than necessary. Then, it is possible to suppress that the image area corresponding to the data luminance range from the low luminance side to the intermediate luminance is displayed with the display luminance higher than necessary, and it is possible to suppress the reduction in the contrast of the display image. Although the data luminance range from the low luminance side to the intermediate luminance is not particularly limited, in the present embodiment, the data luminance range from the low luminance side to the intermediate luminance is a gradation range of 0 or more and 64 or less. Suppose that it corresponds (FIG. 9).

  The effect of this embodiment will be described in detail with reference to FIG. FIG. 10 shows an example of the correspondence between the display range and the display luminance. The horizontal axis of FIG. 10 indicates the maximum data brightness HRange of the display range, and the vertical axis of FIG. 10 indicates the display brightness. FIG. 10 shows an example where the maximum data luminance DRange of the dynamic range of the input image data is fixed (0% to 1000%). The correspondence relationships 1001 to 1003 in FIG. 10 correspond to the data brightness 18% of the input image data. The correspondence relationship 1001 indicates the correspondence relationship when the OOTF process is omitted. The correspondence relationship 1002 indicates a correspondence relationship (conventional example) when the OOTF process is performed using the upper limit display luminance PeakLum_1 as the luminance Lw. The correspondence relationship 1003 indicates the correspondence relationship (the present embodiment) when the OOTF process is performed using the luminance PeakLum_2 as the luminance Lw.

The correspondence relationship 1001 when the OOTF process is omitted will be described. As described above, when the OOTF process is omitted and the clip process is performed, the image is displayed with display luminance higher than the display luminance when the OOTF process is omitted and the compression process is performed. Then, the display luminance decreases with the increase of the maximum data luminance HRange of the display range. When the maximum data brightness HRange = 500%, the data brightness 18% of the input image data is displayed at the display brightness 18 cd / m ² .

  The correspondence relationship 1002 (conventional example) in the case where the OOTF process of the luminance Lw = PeakLum_1 is performed will be described. Since the luminance Lw = PeakLum_1 is fixed independently of the maximum data luminance HRange of the display range, the OOTF process of the luminance Lw = PeakLum_1 is also fixed independently of the maximum data luminance HRange. Therefore, when clip processing and OOTF processing of luminance Lw = PeakLum_1 are performed, an image may be displayed with display luminance higher than necessary. Then, an image area corresponding to the data luminance range from the low luminance side to the intermediate luminance may be displayed with a display luminance higher than necessary, and the contrast of the display image may be reduced. For example, in the correspondence relationship 1002, the display brightness in the case of the maximum data brightness HRange = 500% is much higher than the display brightness in the case of the maximum data brightness HRange = 1000%. Therefore, when the maximum data luminance HRange = 500%, the image is displayed with display luminance higher than necessary.

  A correspondence relationship 1003 (this embodiment) when the OOTF process of the luminance Lw = PeakLum_2 is performed will be described. In this embodiment, since the luminance Lw = PeakLum_2 changes depending on the maximum data luminance HRange of the display range, the OOTF processing of the luminance Lw = PeakLum_2 also changes depending on the maximum data luminance HRange. As a result, even when the clipping process and the OOTF process of the luminance Lw = PeakLum_2 are performed, it is possible to suppress the display of the image with the display luminance higher than necessary. Then, it is possible to suppress that the image area corresponding to the data luminance range from the low luminance side to the intermediate luminance is displayed with the display luminance higher than necessary, and it is possible to suppress the reduction in the contrast of the display image. For example, in the correspondence relationship 1003, the display brightness in the case of the maximum data brightness HRange = 500% is equivalent to the display brightness in the case of the maximum data brightness HRange = 1000%. Therefore, in the case of the maximum data luminance HRange = 500%, the image is not displayed at a display luminance higher than necessary, and the image is displayed at a suitable display luminance.

  As described above, according to the present embodiment, in the OOTF process, the reduction amount of data brightness is controlled based on the maximum data brightness HRange of the display range and the like. As a result, even when clip processing and OOTF processing are performed, a high quality display image can be obtained.

  In the present embodiment, an example has been described in which the HDR method is the HLG method and the OOTF processing based on the luminance PeakLum_2 corresponding to the maximum data luminance DRange of the input image data is performed. However, the present invention is not limited thereto. For example, the luminance PeakLum_2 may not be determined. The OOTF processing may be performed based on the display luminance Lb when the OOTF processing is omitted, which corresponds to the data luminance La arbitrarily set to be included in the display range. Specifically, the data brightness of the range-converted image data may be reduced by a large reduction amount as the display brightness Lb is higher. In that case, for example, the gamma value γ in Equation 4 may be obtained using luminance Lw = Lb × DRange ÷ HRange. FIG. 6 shows an example of the data brightness La and the display brightness Lb when the display range is a data brightness range of data brightness 0% or more and data brightness 500% or less.

The data brightness Lb set arbitrarily can be said to be "data brightness corresponding to the tone value set arbitrarily." The gradation value of input image data or linear image data is set as the gradation value. When the gradation value InputLevel_3 of the linear image data is set, the gradation value OutputLevel_3 of the range-converted image data is obtained using Expression 2. Then, the luminance Lb is calculated using Equation 6 below.

Lb = OutputLevel_3 ÷ 1023 × PeakLum_1 (6)

  The display unit 102 may display parameters (such as gamma value γ) used in the OOTF process on the screen. For example, the display unit 102 may display an OSD (On Screen Display) image indicating the gamma value γ on the screen. This allows the user to easily grasp the parameters used in the OOTF process. Similarly, the display unit 102 may display the display range on the screen. Thereby, the user can easily grasp the display range. The display unit 102 may display the conversion characteristic of the OOTF process on the screen. This allows the user to easily grasp the conversion characteristics of the OOTF process. If a plurality of pieces of information (gamma value γ, display range, conversion characteristics of OOTF processing, and the like) are displayed on the screen, the user can easily grasp the correspondence between the plurality of pieces of information.

  The display device 100 may output the parameters used in the OOTF process to the outside. For example, the display device 100 may output the gamma value γ to the outside as file data. Thus, the parameters used in the OOTF process can be used in the post process, and the display on the display device 100 can be easily reproduced in the post process.

Example 2
Hereinafter, Example 2 of the present invention will be described. Although the example of the HLG method has been described in the first embodiment, the ITU-R BT. An example of a PQ (Perceptual Quantizer) method defined in 2100 will be described. In the following, points different from the first embodiment (structure, processing, and the like) will be described in detail, and description of the same points as the first embodiment will be omitted.

  FIG. 11 shows an example of light conversion in the PQ method. As shown in FIG. 11, in the PQ method, scene luminance is converted to display luminance (OOTF processing). Next, the display luminance is converted into gradation values (Inverse EOTF (Electro-Optical Transfer Function) processing). Then, the gradation value is returned to the display luminance (EOTF processing). The OOTF processing and the Inverse EOTF processing are performed by the imaging device, and the EOTF processing is performed by the display device. That is, in the present embodiment, the OOTF processing is performed by the imaging device.

  FIG. 12 is a block diagram showing a configuration example of a display system 200 according to the present embodiment. In FIG. 12, the same functional units as those of the first embodiment (FIG. 2) are denoted by the same reference numerals as those of the first embodiment. The display system 200 includes an imaging device 210 and a display device 220. The imaging device 210 includes an imaging unit 211, an OOTF conversion unit 212, and a gamma conversion unit 213. The display device 220 includes an image processing unit 221, a display unit 102, a dynamic range acquisition unit 103, and a display range acquisition unit 104. The image processing unit 221 includes a linear conversion unit 105 and a range conversion unit 106.

  The imaging unit 211 converts object light into captured image data (electrical signal) (imaging). As the imaging unit 211, for example, an image sensor such as a charge-coupled device (CCD) or a complementary metal-oxide semiconductor (CMOS) sensor can be used. The photographed image data can also be referred to as “input image data of the imaging device 210”.

The OOTF converter 212 acquires, from the display device 220, dynamic range information, display range information, and display luminance information. Specifically, the OOTF converter 212 acquires, from the display device 220, the maximum data luminance DRange, HRange and the upper limit display luminance PeakLum_1. The dynamic range of input image data of the display device 220 is equal to the dynamic range of photographed image data. Therefore, the dynamic range information on the dynamic range of the input image data of the display device 220 is also the information on the dynamic range of the photographed image data.

  Then, the OOTF conversion unit 212 performs the same OOTF processing as the OOTF processing of the OOTF conversion unit 107 according to the first embodiment on the captured image data generated by the imaging unit 211. Thereby, light conversion image data is generated. The OOTF processing of this embodiment is image processing (tone conversion processing) for converting data luminance of photographed image data.

  Note that the gamma value γ may be generated by the display device 220, and the OOTF conversion unit 212 may acquire the gamma value γ from the display device 220. The dynamic range information, the display range information, the display luminance information, the gamma value γ, and the like may be wirelessly transmitted from the display device 220 to the imaging device 210, or transmitted from the display device 220 to the imaging device 210 by wire. May be As a transmission standard, for example, HDMI (Hish-Definition Multimedia Interfac), USB (Universal Serial Bus), or the like can be used.

  The gamma conversion unit 213 performs gamma conversion processing on the light conversion image data generated by the OOTF conversion unit 212 to generate output image data. In this embodiment, PQ image data having tone characteristics of the PQ method is generated as output image data. The gamma conversion process of the gamma conversion unit 213 corresponds to the Inverse EOTF process of FIG. Inverse EOTF processing is not limited to gamma conversion processing. For example, Inverse EOTF processing may be tone conversion processing different from gamma conversion processing.

  Then, the gamma conversion unit 213 outputs the PQ image data to the display device 220. The PQ image data may be wirelessly transmitted from the display device 220 to the imaging device 210, or may be transmitted by wire from the display device 220 to the imaging device 210. As a transmission standard, for example, HDMI, USB, etc. can be used.

  The image processing unit 221 converts the PQ image data generated by the gamma conversion unit 213 into display image data by the linear conversion processing of the linear conversion unit 105 and the range conversion processing of the range conversion unit 106. Then, the image processing unit 221 outputs the display image data to the display unit 102. In the present embodiment, the linear conversion process corresponds to the EOTF process of FIG.

  As described above, according to this embodiment, the same OOTF processing as the OOTF processing of the first embodiment is performed in an image processing apparatus (specifically, an imaging apparatus) different from the display device. Therefore, when the OOTF process is performed by an image processing apparatus different from the display device, the same effect as that of the first embodiment can be obtained. In this embodiment, the clip processing can be performed after the OOTF processing, but even in such a case, a high quality display image can be obtained by the same operation as the operation of the first embodiment.

  The various tone conversion processes (linear conversion process, range conversion process, OOTF process, etc.) described in the first and second embodiments may be processes using mathematical expressions, or processes using tables. It may be.

Each functional unit of the first and second embodiments may or may not be individual hardware. The functions of two or more functional units may be realized by common hardware. Each of the plurality of functions of one functional unit may be realized by separate hardware. Two or more functions of one functional unit may be realized by common hardware. Also, each functional unit may or may not be realized by hardware. For example, the device may have a processor and a memory in which a control program is stored. The functions of at least some of the functional units of the device may be realized by the processor reading and executing the control program from the memory.

  The first and second embodiments are merely examples, and configurations obtained by appropriately changing or changing the configurations of the first and second embodiments within the scope of the present invention are also included in the present invention. The configuration obtained by appropriately combining the configurations of Embodiments 1 and 2 is also included in the present invention.

<Other Embodiments>
The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or apparatus via a network or storage medium, and one or more processors in a computer of the system or apparatus read and execute the program. Can also be realized. It can also be implemented by a circuit (eg, an ASIC) that implements one or more functions.

100, 220: Display device 101, 221: Image processing unit 102: Display unit 103: Dynamic range acquisition unit 104: Display range acquisition unit 105: Linear conversion unit 106: Range conversion unit 107, 212: OOTF conversion unit 200: Display system 210: imaging device 211: imaging unit 213: gamma conversion unit

Claims (19)

A first acquisition step of acquiring first information on a first range which is a luminance range of input image data;
A second acquisition step of acquiring second information related to a second range which is a luminance range to be displayed;
A first conversion step of generating first converted image data by performing a first conversion process of converting a luminance range of the input image data into a luminance range equal to a display luminance range based on the second information;
A second conversion step of generating second converted image data by performing a second conversion process of converting the luminance of the first converted image data based on the first information and the second information;
Have
In the second conversion step, the second conversion processing is performed such that the luminance of the first converted image data is reduced by a large reduction amount as the maximum luminance of the second range decreases. Processing method. In the second conversion step, the second conversion is performed such that the luminance of the first converted image data is reduced by a large reduction amount as the ratio of the maximum luminance of the first range to the maximum luminance of the second range increases. The image processing method according to claim 1, wherein the processing is performed. In the second conversion step, the second conversion processing is performed such that the luminance of the first converted image data is reduced by a large reduction amount as the luminance obtained by multiplying the ratio by the upper limit of the display luminance is higher. The image processing method according to claim 2, characterized in that: In the second conversion step,
From the luminance Lw, which is the luminance obtained by multiplying the ratio by the upper limit of the display luminance, a gamma value γ according to the following equation is determined:
γ = 1.2 + 0.42 LOG ₁₀ (Lw / 1000)
The image processing method according to claim 3, wherein a gamma conversion process using the gamma value γ is performed as the second conversion process. The image processing method according to claim 3, further comprising: a third acquisition step of acquiring third information related to the upper limit of the display luminance. In the second conversion step, the higher the display luminance when the second conversion processing corresponding to the luminance set to be included in the second range is omitted, the larger the reduction amount of the first converted image data. The image processing method according to claim 1, wherein the second conversion process is performed so that the luminance is reduced. The second conversion processing is image processing using parameters based on the first information and the second information,
The image processing method according to any one of claims 1 to 6, further comprising: a first display step of displaying the parameter on a display unit. The second conversion processing is image processing using parameters based on the first information and the second information,
The image processing method according to any one of claims 1 to 7, further comprising: an output step of outputting the parameter. The image processing method according to any one of claims 1 to 8, wherein the second range is a part of the first range. The method further includes a third conversion step of converting the input image data into third converted image data in which the luminance linearly increases with an increase in gradation value,
10. The image processing according to any one of claims 1 to 9, wherein the first conversion processing is image processing for converting a luminance range of the third converted image data into a luminance range equal to the display luminance range. Image processing method. In the first converted image data, the gradation value linearly increases from the lower limit to the upper limit with respect to the increase in luminance from the minimum luminance to the maximum luminance in the second range, and the luminance is higher than the maximum luminance in the display range. 11. The image processing method according to claim 10, wherein the upper limit of the gradation value is associated. The image processing method according to any one of claims 1 to 11, wherein, in the second acquisition step, information specified by a user is acquired as the second information. The image processing method according to any one of claims 1 to 12, further comprising: a second display step of displaying an image based on the second converted image data on display means. The image processing method according to any one of claims 1 to 13, wherein the input image data is image data having gradation characteristics of HLG (Hybrid Log-Gamma) method. A first acquisition step of acquiring first information on a first range which is a luminance range of input image data;
A second acquisition step of acquiring second information related to a second range which is a luminance range to be displayed;
A first conversion step of generating first converted image data by performing conversion processing of converting the luminance of the input image data based on the first information and the second information;
Have
3. The image processing method according to claim 1, wherein, in the first conversion step, the conversion processing is performed such that the luminance of the input image data is reduced by a large reduction amount as the maximum luminance of the second range decreases. The image processing method according to claim 15, further comprising a second conversion step of converting the first converted image data into second converted image data having gradation characteristics of PQ (Perceptual Quantizer) method. . First acquisition means for acquiring first information on a first range which is a luminance range of input image data;
A second acquisition unit that acquires second information related to a second range that is a luminance range to be displayed;
First conversion means for generating first converted image data by performing a first conversion process of converting a luminance range of the input image data into a luminance range equal to a display luminance range based on the second information;
Second conversion means for generating second converted image data by performing a second conversion process of converting the luminance of the first converted image data based on the first information and the second information;
Have
The image is characterized in that the second conversion processing is performed such that the luminance of the first converted image data is reduced by a large reduction amount as the maximum luminance of the second range decreases. Processing unit. First acquisition means for acquiring first information on a first range which is a luminance range of input image data;
A second acquisition unit that acquires second information related to a second range that is a luminance range to be displayed;
Conversion means for generating converted image data by performing conversion processing for converting the luminance of the input image data based on the first information and the second information;
Have
The image processing apparatus, wherein the conversion means performs the conversion processing such that the luminance of the input image data is reduced by a large reduction amount as the maximum luminance of the second range is lower.   The program for making a computer perform each step of the image processing method of any one of Claims 1-16.

Images