JP2019004230A - Image processing device and method, and imaging apparatus - Google Patents

Abstract

To appropriately correct brightness of a subject near proper exposure in an image of a high dynamic range.SOLUTION: An image processing device comprises: first conversion means for converting image data into first image data of a first dynamic range; second conversion means for converting image data into second image data of a second dynamic range that is wider than the first dynamic range; first calculation means for calculating first luminance of a subject region in the first image data and second luminance of a predetermined region including the subject region; second calculation means for calculating third luminance of a subject region in the second image data and fourth luminance of a predetermined region; generation means which generates correction data for calculating target luminance of the subject region in the second image data based on the first, second and fourth luminance and converting the third luminance into the target luminance; and third conversion means which converts image data into third image data of the second dynamic range.SELECTED DRAWING: Figure 5

Description

  The present invention relates to an image processing apparatus and method, and an imaging apparatus, and more particularly to an image processing apparatus and method and an imaging apparatus capable of developing an HDR (High Dynamic Range) image.

  In recent years, due to the evolution of LED elements of the display, etc., black can be tightened and the high luminance can be further enhanced, and the high dynamic range (hereinafter referred to as “HDR”) information of the image is displayed as it is without being compressed. It has become possible to do. On the other hand, the conventional SDR display has a narrower dynamic range than the HDR display (hereinafter referred to as “SDR”), and it is expected that the SDR display and the HDR display will be used for a while. Is done. Therefore, it is necessary for the imaging apparatus to have both a conventional development method as the content for the SDR display and an HDR development method as the content for the HDR display.

  The HDR display can realistically reproduce landscapes such as the gradation of blue sky clouds and neon nightscapes, which had previously decreased in contrast with the SDR display, while the brighter the brighter the part, the darker the main subject becomes. It also has the feature of being perceived. For example, FIG. 14A is a diagram showing a display example using a conventional SDR display, and it is assumed that the building as the main subject is properly exposed, for example, 18 nits, and the sky is 100 nits. On the other hand, FIG. 14B is a diagram showing a display example on the HDR display. The building as the main subject has 18 nits equivalent to the SDR display, but the sky is 400 nits. Then, although the building which is the main subject in FIGS. 14A and 14B is physically displayed with the same luminance, the building in FIG. 14B feels darker in human vision. May end up.

  This is considered to be caused by the simultaneous contrast of spatial contact in the color contrast that is human visual characteristics. In addition to the lightness contrast shown in FIG. 14, the simultaneous contrast includes a saturation contrast in which a color surrounded by a highly saturated color appears to have a decreased saturation. Since this phenomenon is a phenomenon in which the contrast is more emphasized, there is a scene that is preferable as a realistic HDR image quality, but it is not necessarily preferable when the brightness of the main subject, particularly the face of a person, becomes dark. There is concern that there are unlimited scenes.

  Patent Document 1 discloses a correction method that maintains the contrast by compressing based on the overall luminance when the dynamic range of the output device is narrower than the dynamic range of the imaging device.

Japanese Patent No. 4687320

  However, the prior art disclosed in Patent Document 1 is basically a compression method from image data with a wide dynamic range to narrow image data, and only mentions the basic concept, and human visual characteristics. Is not taken into account.

  The present invention has been made in view of the above problems, and an object of the present invention is to suitably correct the brightness of a subject in the vicinity of proper exposure in an image with a high dynamic range.

  In order to achieve the above object, an image processing apparatus according to the present invention includes first conversion means for converting image data obtained by photographing into first image data having a first dynamic range, and the image data. The second conversion means for converting the second dynamic range to the second image data of the second dynamic range, which is wider than the first dynamic range, the first luminance of the subject area in the first image data, First calculation means for calculating a second luminance of a predetermined area including the subject area, a third luminance of the subject area in the second image data, and a fourth luminance of the predetermined area. Based on the second calculation means, the first luminance, the second luminance, and the fourth luminance, the target luminance of the subject area in the second image data is calculated, and the third luminance is calculated. Is converted to the target brightness That has a generation means for generating correction data by using the correction data, the third conversion means for converting the image data into third image data of the second dynamic range, the.

  According to the present invention, it is possible to suitably correct the brightness of a subject in the vicinity of proper exposure in an image with a high dynamic range.

Sectional drawing which shows schematic structure of a common interchangeable lens type digital single-lens reflex camera. FIG. 2 is a block diagram illustrating a configuration example of an electric circuit of a camera body and an interchangeable lens according to an embodiment of the present invention. The figure which showed the area | region of the SDR and HDR scene and the main subject. 7 is a flowchart showing development processing in the first and second embodiments. The block diagram which shows the function structure of the signal processing circuit in 1st and 2nd embodiment, and the flow of data. The figure which shows an example which divided | segmented the image in 1st Embodiment into the some block. The figure which shows an example of the detection result of a person's face. The figure which shows the EOTF characteristic of PQ (Perceptual Quantization) standardized by SMPTE ST2084. The figure which showed the characteristic of the gradation correction data in 1st Embodiment. The figure which showed the example of the high-intensity area | region in the HDR scene. The figure which showed the characteristic of the gradation correction data in 2nd Embodiment. The block diagram which shows the function structure of the signal processing circuit in 3rd Embodiment, and the flow of data. The figure which showed the example of a setting of viewing conditions by a SDR display. The figure which shows the image of the difference in appearance by a SDR display and a HDR display.

  DESCRIPTION OF EMBODIMENTS Hereinafter, embodiments for carrying out the present invention will be described in detail with reference to the accompanying drawings. However, the dimensions, shapes, relative arrangements, and the like of the components exemplified in the present embodiment should be changed as appropriate according to the configuration of the apparatus to which the present invention is applied and various conditions. It is not limited to. In the following embodiments, a digital camera will be described as an example of an imaging apparatus according to the present invention. However, the present invention is not limited to a device mainly intended for shooting such as a digital camera. For example, the present invention can be applied to any device that has a built-in or externally connected imaging device, such as a mobile phone, personal computer (laptop type, desktop type, tablet type, etc.), game machine, and the like. Therefore, the “imaging device” in this specification includes any electronic device having an imaging function.

  FIG. 1 is a cross-sectional view mainly showing the arrangement of optical members, sensors and the like in an interchangeable lens type digital single-lens reflex camera according to an embodiment of the present invention. The camera in the present embodiment is a so-called digital single-lens reflex camera capable of exchanging lenses, and has a camera body 1 and an interchangeable lens 2. The interchangeable lens 2 performs information communication with the camera body 1 as necessary through a contact point of a lens mount provided in the camera body 1.

  In the camera body 1, the image sensor 10 is, for example, a CMOS image sensor, a CCD image sensor, or the like, and a plurality of pixels (storage type photoelectric conversion elements) are arranged in the light receiving unit. In the imaging device 10, in addition to the pixels arranged in the light receiving unit, an amplification circuit for pixel signals, a peripheral circuit for signal processing, and the like are formed. A mechanical shutter 11 provided near the front of the image sensor 10 controls the exposure timing and exposure time of the image sensor 10.

  The semi-transparent main mirror 3 and the first reflecting mirror 7 disposed on the back surface of the main mirror 3 jump up to the upper side at the time of photographing and retreat from the photographing optical path. The second reflecting mirror 8 further reflects the light beam reflected by the first reflecting mirror 7 and makes it incident on a focus detection sensor (AF sensor) 9. The AF sensor 9 may be an image sensor having a smaller number of pixels than the image sensor 10, for example. The first reflection mirror 7, the second reflection mirror 8, and the AF sensor 9 are configured to perform focus detection by a phase difference detection method at an arbitrary position in the shooting screen.

  The photometric sensor (AE sensor) 6 receives the image of the photographing screen reflected by the pentaprism 4 and the third reflecting mirror 5. The AE sensor 6 can divide the light receiving unit into a plurality of arbitrary areas and output luminance information of the subject for each divided area.

  The finder optical system is constituted by the pentaprism 4, and the subject image reflected by the pentaprism 4 can be observed from an eyepiece (not shown). Of the light beam reflected by the main mirror 3 and diffused by the focus plate 12, a part outside the optical axis is incident on the AE sensor 6. Note that, during live view display and moving image recording, the main mirror 3 is always in a state of jumping upward, so that exposure control and focus adjustment control are performed using image information on the imaging surface.

  FIG. 2 is a block diagram showing a configuration example of an electric circuit of the camera body 1 and the interchangeable lens 2 shown in FIG. The control unit 21 is, for example, a one-chip microcomputer in which an ALU (Arithmetic and Logic Unit) 21c, a ROM 21d, a RAM 21e, an A / D converter 21a, a timer 21b, a serial communication port (SPI) 21f, and the like are incorporated. The control unit 21 controls the operations of the camera body 1 and the interchangeable lens 2 by executing a program stored in the ROM 21d, for example. The specific operation of the control unit 21 will be described later.

  The output signals of the AF sensor 9 and the AE sensor 6 are connected to the input terminal of the A / D converter 21a of the control unit 21, and the control unit 21 performs AF processing and AE processing based on the A / D signal.

  The signal processing circuit 25 controls the image sensor 10 in accordance with an instruction from the control unit 21 and applies A / D conversion and signal processing to a signal output from the image sensor 10 to obtain an image signal. Further, the signal processing circuit 25 performs necessary image processing such as compression / combination when recording the obtained image signal. The memory 28 is a DRAM or the like, and is used as a work memory when the signal processing circuit 25 performs various signal processing, or as a VRAM when displaying an image on the display 27 described later.

  The display 27 is assumed to be connected to an external display in accordance with a standard such as a rear liquid crystal display or HDMI (registered trademark), and displays information such as camera setting values, messages, menu screens, and captured images. . The display 27 is controlled by an instruction from the control unit 21. The storage unit 26 is a non-volatile memory such as a flash memory, for example, and a captured image signal is input from the signal processing circuit 25.

  The motor 22 performs up / down of the main mirror 3 and the first reflecting mirror 7 and charges the mechanical shutter 11 under the control of the control unit 21. The operation unit 23 is an input device group such as a switch used by the user to operate the camera. The operation unit 23 is a release switch for instructing the start of shooting preparation operation and the start of shooting, and a shooting mode selection for selecting a shooting mode. Switches, direction keys, enter keys, etc. are included. The contact part 29 is a contact for communicating with the interchangeable lens 2 and is connected to an input / output signal of the serial communication port of the control part 21. The shutter drive unit 24 is connected to the output terminal of the control unit 21 and drives the mechanical shutter 11.

  The interchangeable lens 2 is provided with a contact portion 50 that makes a pair with the contact portion 29. A lens control unit 51, which is a one-chip microcomputer similar to the control unit 21, is connected to the contact unit 50, and communication with the control unit 21 is possible. For example, the lens control unit 51 executes a program stored in the ROM, and controls the operation of the interchangeable lens 2 based on an instruction from the control unit 21. Further, the controller 21 is notified of information such as the state of the interchangeable lens 2. The focus lens driving unit 52 is connected to the output terminal of the lens control unit 51 and drives the focus lens. The zoom drive unit 53 changes the angle of view of the interchangeable lens according to the control of the lens control unit 51. The aperture driving unit 54 adjusts the aperture amount of the aperture according to the control of the lens control unit 51.

  When the interchangeable lens 2 is attached to the camera body 1, the lens control unit 51 and the control unit 21 of the camera body can communicate data via the contact parts 29 and 50. Further, electric power for driving the motor and actuator in the interchangeable lens 2 is also supplied through the contact portions 29 and 50. Lens-specific optical information necessary for the control unit 21 of the camera body 1 to perform focus detection and exposure calculation, information on the subject distance based on the distance encoder, and the like from the interchangeable lens 2 to the control unit of the camera body 1 21 is output by data communication. In addition, focus adjustment information and aperture information obtained as a result of focus detection and exposure calculation performed by the control unit 21 of the camera body 1 are output from the control unit 21 to the interchangeable lens 2 by data communication. The lens control unit 51 controls the focus lens, the aperture, and the like according to the obtained focus adjustment information and aperture information.

Now, when displaying the image obtained by the above-mentioned imaging device, in order to perceive that the luminance of the halftone as the main subject is the same even if the maximum luminance changes between the SDR display and the HDR display. The condition described below is a condition. First, the average of 3 scene a1 (predetermined region, the entire image) of SDR shown in (a) the average luminance _{AveSB SDR} of the average luminance _{AveOB SDR} of the main object a2, HDR scenes b1 shown in FIG. 3 (b) The brightness is AveSB _HDR and the average brightness of the main subject b2 is AveOB _HDR . At this time, the sensation of brightness is proportional to the logarithm of luminance as shown in the following equation (1) according to the Weber-Fechner law.

Here, the average luminance is used as the luminance. However, if the maximum luminance is used as the luminance, for example, a point light source such as a night view has a high luminance for a small area. This is because such an adverse effect is expected to occur. That is, there is a detrimental effect that the high contrast, which is the effect of HDR, is lost on the contrary, because the correction is such that it is strongly pulled by the brightness of the point light source and the gain is greatly increased.
In the above equation (1), AveOB _HDR is the average luminance of the HDR subject area, that is, the target luminance.

It can ask for. Specifically, it is desirable to set the luminance of SDR in consideration of display characteristics at the time of viewing a general SDR display, and the brightness of HDR in consideration of display characteristics at the time of viewing of an HDR display. This point will be described in detail later.

<First Embodiment>
Hereinafter, specific operations in the first embodiment of the present invention will be described. When the control unit 21 becomes operable, for example, when a power switch included in the operation unit 23 of FIG. 1 is turned on, the control unit 21 first communicates with the lens control unit 51 of the interchangeable lens 2 to perform focus detection and photometry. Initialization processing such as obtaining information on various lenses necessary for the operation.

  Next, the state of the shutter switch included in the operation unit 23 is monitored, and when SW1 is turned on during the operation of the shutter switch (for example, half-pressed), the operation of AF (autofocus) processing and AE (automatic exposure) processing starts. Instruct. Thereafter, when SW2 is turned on upon completion of the operation of the shutter switch (for example, fully pressed), a start of a series of processes including an exposure process, a development process, and a recording process is instructed. In the exposure process, light from the subject (subject image) forms an image on the image sensor 10 through the imaging optical system of the interchangeable lens 2. The image sensor 10 is, for example, a single plate color image sensor including a general primary color filter. The primary color filter is composed of three types of color filters each having a transmission main wavelength band in the vicinity of 650 nm, 550 nm, and 450 nm, and each color plane corresponds to each band of R (red), G (green), and B (blue). Take a picture of. In a single-plate color image sensor, this color filter is spatially arranged for each pixel, and each pixel can obtain an intensity in a single color plane. Therefore, a color mosaic image (captured RAW image) is output from the image sensor 10.

  In the next development process, the photographic RAW image output from the image sensor 10 is developed. Here, with reference to the flowchart in FIG. 4 and the block diagram in FIG. 5, the development processing in the first embodiment for the captured RAW image 101 will be described. Note that the development processing described below is performed by the signal processing circuit 25 unless otherwise specified. However, part of the development processing described below can be performed in cooperation with the control unit 21.

  First, the SDR development process in S10 and the HDR development process in S11 will be described. In the white balance unit 102, white balance processing is performed to correct a pixel signal detected as white to a signal representing white. Specifically, the RGB data of each pixel constituting the captured RAW image data is plotted in a predetermined color space such as an xy color space. As a result, R, G, and B of data plotted in the vicinity of a black body radiation locus having a high possibility of a light source color in the color space are integrated, and the white balance coefficient G / R of the R and B components from the integrated value. And G / B are obtained. White balance processing is performed using the white balance coefficient generated by the above processing.

  The color interpolation unit 103 interpolates the color mosaic image to generate a color image in which R, G, and B color information are aligned in all pixels. A basic color image is generated from the generated color image through the matrix conversion unit 104, the first gamma conversion unit 105, and the second gamma conversion unit 106. The first gamma conversion unit 105 assumes development for an SDR display, and basically has a reverse characteristic of a general monitor gamma as a gamma characteristic. On the other hand, the second gamma conversion unit 106 is assumed to be developed for HDR display, and the gamma characteristic was developed by PQ (Perceptual Quantization) standardized by SMPTE ST2084 or ARIB STD-B67, for example. It is the reverse characteristic (photoelectric conversion characteristic, OETF: Optical-Electro Transfer Function) of the electro-optical conversion characteristic (EOTF) of HLG (Hybrid Log Gamma). That is, the monitor gamma characteristic corresponding to the gamma characteristic used in the first gamma conversion unit 105 has a gentler slope on the high luminance side than the monitor gamma characteristic corresponding to the gamma characteristic used in the second gamma conversion unit 106. .

  Thereafter, the first color luminance adjustment unit 107 and the second color luminance adjustment unit 108 perform color luminance adjustment processing for improving the appearance of the image on the color image subjected to gamma conversion. For example, image correction such as detecting a sunset scene and enhancing saturation is performed according to the scene. When the color luminance adjustment processing by the first color luminance adjustment unit 107 and the second color luminance adjustment unit 108 is completed, the development processing is completed, and the SDR developed image 109 and the HDR developed image 110 are generated. In FIG. 4, it is described that HDR development is performed in S <b> 11 after SDR development in S <b> 10, but the processing order may be reversed or may be performed in parallel as described above.

Next, the SDR luminance calculation unit 111 calculates the average luminance AveSB _SDR of the scene in the SDR developed image 109 (S12). The average luminance of the scene is obtained as follows, for example. First, as shown in FIG. 6, the image is divided into a plurality of blocks, and RGB values are calculated for each block. Then, the average value of the RGB values of all the blocks is obtained. At this time, the RGB values of all the blocks may be simply averaged, or the weight spread spatially in a Gaussian shape with the peak at the center of the angle of view. May be superposed on each block, and then the average value (weighted average value) of all blocks may be taken. Thereafter, the average value of the calculated RGB values is converted into a luminance value. As a reference environment for an SDR still image, the monitor gamma is generally treated as a power of 2.4 for the dark portion and the bright portion for the bright portion. Alternatively, it may be a simple power of 2.2. Expressions (3) and (4) show an example of sRGB (D65).

Also, when R, G, and B are maximum values, Y is the maximum luminance, which is converted to an average luminance value of the scene at the time of SDR viewing, assuming that this is 100 nits.

Next, the average luminance AveOB _SDR of the subject area of the main subject of the _SDR is calculated in the same manner (S13). There are several possible definitions of the main subject. For example, as shown in FIG. 7, a case where a person is the main subject can be considered. As a representative method of identifying a person, there is a method of detecting a face, and face detection is performed using a known method such as face identification based on Haar-Like feature values. The average luminance of the face area thus identified (for example, within the dotted line area in FIG. 7) is calculated as the average luminance value of the subject area at the time of SDR viewing by the same calculation method as the average luminance of the scene described above. .

On the other hand, the HDR luminance calculation unit 112 calculates the average luminance AveSB _HDR of the scene in the HDR developed image 110 in the same manner as the SDR scene (S14). The previous stage of conversion to the average luminance value of the scene at the time of HDR viewing, that is, the average value of RGB values is calculated by the same method as the SDR. After that, referring to the HDR characteristic 120, the average of the scene at the time of viewing Convert to luminance value. When the PQ curve is used in the second gamma conversion unit 106, the absolute luminance L is acquired using the EOTF conversion equation represented by the following equation (5) standardized by SMTPE ST2084.

Here, N is an input signal level, and m ₁ , m ₂ , c ₁ , c ₂ , and c ₃ are parameters. FIG. 8 is a diagram showing the EOTF characteristics of PQ.

Also, the average luminance CurrentAveOB _HDR of the subject area in the HDR developed image 110 is calculated by the same method as the SDR (S15). An arithmetic average is generally used as a method for calculating the average luminance. However, since there is an influence of being pulled by a minimum peak luminance, it is desirable that the geometric average is more compatible with human perception.

Then, in the correction data calculation unit 113, the average luminance _AveSB SDR of the calculated SDR scene, the average brightness _AveOB SDR of SDR of the subject region, the average luminance _{AveSB HDR} of HDR scenes, the the target HDR object area The average luminance AveOB _HDR is calculated using the above equation (2). After that, for example, a one-dimensional LUT having characteristics as shown in FIG. 9 is used as tone correction data from the average luminance Current AVEOB _{HDR of} the HDR subject region and the average luminance AveOB _HDR of the target HDR subject region. Look Up Table) is generated (S16).

  The gradation correction unit 114 performs gradation correction by LUT conversion on the HDR developed image 110 using the gradation correction data (LUT) calculated by the correction data calculation unit 113 (S17). In the conversion, not only the brightness component and the color difference component such as YUV but the color space based on the three primary colors, such as RGB, can be used to correct not only the brightness but also the saturation. Since correction is performed at the same time, natural correction is possible. With the above processing, the development processing in the first embodiment is completed.

  Thereafter, the HDR developed image corrected by the compression unit 115 is compressed, converted into image data suitable for recording by the recording unit 116, and output as a final HDR developed image. Although not shown, when an SDR developed image is output for an SDR display, the SDR developed image 109 output from the first color luminance adjustment unit 107 is input to the compression unit 115, and is recorded by the recording unit 116. What is necessary is just to convert into image data suitable for recording.

  As described above, according to the first embodiment, the brightness of the HDR main subject is preferably corrected based on the average brightness and main subject brightness of the SDR scene and the average brightness and main subject brightness of the HDR scene. be able to.

<Second Embodiment>
Next, a second embodiment of the present invention will be described. In the first embodiment described above, tone correction is performed based on the average brightness of the HDR subject area and the average brightness of the target HDR subject area. On the other hand, the second embodiment described below is different from the first embodiment in that a region where the luminance does not change is added at the time of gradation correction so as to maintain a high luminance contrast as much as possible.

  Note that the flow of the development processing in the second embodiment is basically the same as the flowchart shown in FIG. 4, and the details of the signal processing circuit 25 that performs the development processing are the same as those in FIG. The difference from the first embodiment is that when the HDR luminance calculation unit 112 in FIG. 5 calculates the average luminance of the HDR scene, the high level of the HDR scene as shown in the area within the dotted frame in FIG. This is a point for calculating the average luminance. As the definition of high luminance, for example, 100 nits, which is 100% reflectance in SDR, is set as a threshold, and luminance equal to or higher than this threshold is set as high luminance. In addition, what is necessary is just to set this threshold value suitably. Specifically, the same processing as the calculation of the average luminance of the HDR scene of the first embodiment may be performed, but of the absolute luminance for each block calculated by the equation (5), in the second embodiment, Further, the average luminance is acquired only from the blocks that exceed a preset threshold value (for example, 100 nits) of the high luminance region.

As the fixed point the average luminance AveHi _HDR a high brightness region of the HDR obtained as described above does not correct the high-brightness side than this, it generates a tone correction characteristic shown in Figure 11, the tone correction section Apply at 114. The concern in the second embodiment, when the average brightness _AveHi luminance difference is small scene _HDR high luminance region of the target average luminance _{AveOB HDR} and HDR of the subject region of HDR, due to one-dimensional LUT shown in FIG. 11 It can be mentioned that the conversion reduces the contrast. Therefore, although it takes a long calculation time, for example, instead of a one-dimensional LUT, a method of correcting gradation while maintaining local contrast by a local tone mapping method represented by Retinex or the like may be used.

  As described above, according to the second embodiment, it is possible to raise the main subject to the target luminance while maintaining the high-brightness contrast enlarged by HDR as compared with the first embodiment.

<Third Embodiment>
Next, a third embodiment of the present invention will be described. In the first embodiment and the second embodiment, 100 nits is assumed as the maximum brightness of the SDR display. However, there is a SDR display that supports bright brightness such as 500 nits, and the actual use is assumed to be 100 nits. It may feel dark. On the other hand, in the third embodiment, the highest brightness of the SDR display is set as an external setting, and the appropriate exposure in the HDR display is set based on the highest brightness of the SDR display. And different.

  FIG. 12 is a block diagram illustrating a functional configuration of the signal processing circuit according to the third embodiment. Compared to FIG. 5 described in the first embodiment, an SDR characteristic 118 is added. Although details will be described later, in the third embodiment, gamma conversion for HDR display is performed by the third gamma conversion unit 121 using the gradation correction data generated by the correction data calculation unit 113. In the third embodiment, the brightness, color temperature, gamma, etc. of the white point of the SDR are set as shown in FIG. 13 via the operation unit 23 of FIG. 2, and the result is held as the SDR characteristic 118 of FIG. deep. Then, in the SDR luminance calculation unit 111, the gamma value used in Expression (3) is referred to the user-set gamma. Similarly, the user-set white point color temperature is referred to the RGB to XYZ conversion matrix coefficient used in Equation (4), and the user-set brightness value is referred to the calculation from the calculated Y value to the absolute brightness. By doing so, it is possible to calculate the SDR luminance according to the actual assumed use. It is necessary to reflect the highest luminance associated with 100% which is the SDR peak signal value set by the user to the HDR luminance calculation unit 112 as well. For example, by calculating gradation correction data in the correction data calculation unit 113 so that the absolute luminance corresponding to 18% that is the appropriate exposure is maintained even in HDR, the sensory brightness of the subject in the vicinity of the appropriate exposure of SDR and HDR is calculated. Can be made equivalent.

  As a method of calculating the absolute luminance of HDR, in the first and second embodiments described above, the standard was implemented using the standard in SMTPE ST2084, but it was implemented using the HLB (Hybrid Log-Gamma) according to the ARIB standard STD-B67. It doesn't matter. In that case, conversion as shown in the following equation (6)

The luminance E may be calculated using the inverse transformation of. In Equation (6), E ′ is an input signal level, and a, b, and c are coefficients.

  The third gamma conversion unit 121 uses a characteristic obtained by adding (synthesizing) the gradation correction data generated by the correction data calculation unit 113 to the gamma characteristic (conversion data) used by the second gamma conversion unit 106. The image output from the matrix conversion unit 104 is gamma converted. In this way, by reflecting the generated gradation correction data in the third gamma conversion unit 121, it is possible to realize the target brightness with one gamma, and it is possible to suppress degradation such as quantization error.

  The image data obtained in this manner is subjected to the same color luminance adjustment processing as that of the second color luminance adjustment unit 108 by the third color luminance adjustment unit 122, compressed by the compression unit 115, and recorded by the recording unit 116. It is converted into suitable image data and output as a final HDR developed image.

  As described above, according to the third embodiment, when the assumed environment of SDR is changed, the luminance of the main subject of HDR is suitably corrected after matching the absolute luminance of appropriate exposure of SDR content and HDR content. can do.

<Other embodiments>
Further, the present invention supplies a program that realizes one or more functions of the above-described embodiment to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus execute the program. It can also be realized by a process of reading and executing. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

  105: first gamma conversion unit, 106: second gamma conversion unit, 107: first color luminance adjustment unit, 108: second color luminance adjustment unit, 111: SDR luminance calculation unit, 112: HDR luminance calculation , 113: correction data calculation unit, 114: gradation correction unit, 118: SDR characteristic, 120: HDR characteristic, 121: third gamma conversion unit, 122: third color luminance adjustment unit

Claims (16)

First conversion means for converting image data obtained by photographing into first image data having a first dynamic range;
Second conversion means for converting the image data into second image data having a second dynamic range wider than the first dynamic range;
First calculation means for calculating a first luminance of a subject area in the first image data and a second luminance of a predetermined area including the subject area;
Second calculating means for calculating a third luminance of the subject area in the second image data and a fourth luminance of the predetermined area;
Based on the first luminance, the second luminance, and the fourth luminance, a target luminance of the subject area in the second image data is calculated, and the third luminance is converted into the target luminance. Generating means for generating correction data;
An image processing apparatus comprising: third conversion means for converting the image data into third image data having the second dynamic range using the correction data. The first conversion means develops the first image by developing using a first photoelectric conversion characteristic corresponding to a first electro-optical conversion characteristic of the first display device having the first dynamic range. Convert to data,
The second conversion means develops the first image by developing using a second photoelectric conversion characteristic corresponding to a second electro-optical conversion characteristic of the second display device having the second dynamic range. The image processing apparatus according to claim 1, wherein the image processing apparatus is converted into data.   The image processing apparatus according to claim 2, further comprising an input unit configured to input information relating to characteristics of the first display device.   4. The image processing apparatus according to claim 2, wherein the first photoelectric conversion characteristic has a gentler slope on the high luminance side than the second photoelectric conversion characteristic. 5.   5. The image processing according to claim 1, wherein the first calculation unit and the second calculation unit calculate the first to fourth luminances by a geometric average, respectively. 6. apparatus.   The first calculation means and the second calculation means calculate the first to fourth luminances by a weighted average using a weight with a peak at the center of each region. The image processing apparatus according to any one of 1 to 4. The second calculation means further calculates a fifth luminance of a region having a luminance equal to or higher than an arbitrary threshold among the predetermined regions,
The image processing apparatus according to claim 1, wherein the generation unit generates the correction data so as not to correct a signal having a luminance higher than the fifth luminance.   8. The third conversion unit according to claim 1, wherein the third conversion unit converts the second image data into the third image data by correcting the second image data using the correction data. An image processing apparatus according to 1.   The third conversion unit converts the image data into the third image data by using conversion data obtained by synthesizing the conversion data used in the second conversion unit and the correction data. The image processing apparatus according to claim 1.   The image processing apparatus according to claim 1, wherein the subject is a human face.   The image processing apparatus according to claim 1, wherein the predetermined area is an entire image area.   The image processing apparatus according to claim 1, wherein the generation unit obtains the target luminance based on Weber-Fechner's law. Imaging means for photographing a subject and outputting image data;
An imaging apparatus comprising: the image processing apparatus according to claim 1. A first conversion step in which a first conversion means converts image data obtained by photographing into first image data having a first dynamic range;
A second conversion step, wherein the second conversion means converts the image data into second image data having a second dynamic range wider than the first dynamic range;
A first calculation step in which a first calculation means calculates a first luminance of a subject area in the first image data and a second luminance of a predetermined area including the subject area;
A second calculation step in which a second calculation means calculates a third luminance of the subject area in the second image data and a fourth luminance of the predetermined area;
The generation means calculates a target brightness of the subject area in the second image data based on the first brightness, the second brightness, and the fourth brightness, and sets the third brightness to the target brightness A generation process for generating correction data to be converted into luminance;
And a third conversion step of converting the image data into third image data of the second dynamic range using the correction data.   The program for functioning a computer as each means of the image processing apparatus of any one of Claims 1 thru | or 12.   A computer-readable storage medium storing the program according to claim 15.