JP7157714B2 - Image processing device and its control method - Google Patents

Description

The present invention relates to an image processing apparatus and its control method, and more particularly to a technology for handling high dynamic range (HDR) signals.

2. Description of the Related Art Due to improvements in the performance of light-emitting elements (for example, LEDs), display devices with a wider dynamic range of display luminance than ever before have been realized. With such a display device, it is possible to more faithfully display an image (HDR image) having colors and details in a high luminance range, which could not be expressed by a conventional display device.

A signal characteristic representing the relationship between the video signal level and display brightness in an HDR image is defined by an EOTF (Electro-Optical Transfer Function). There are two EOTFs: PQ (Perceptual Quantization) standardized by SMPTE ST 2084 and HLG (Hybrid Log Gamma) standardized by ARIB STD-B67. The major difference is that HLG treats the display luminance as a relative value, whereas PQ treats the display luminance as an absolute value up to 10000 nits (or cd/m ² ). Therefore, when shooting in a shooting mode in which the output dynamic range changes, the display peak luminance may change in PQ.

FIG. 1 shows an example of input/output characteristics when the output dynamic range changes with two shooting modes 11 and 12, where the horizontal axis indicates the number of input steps and the vertical axis indicates the output luminance. Comparing the gamma curves of the respective shooting modes, they have common input/output characteristics except for the high luminance region, and peak luminances 13 and 14 are different. In the following, unless otherwise specified, the signal characteristics of HDR images are based on PQ.

By the way, an image signal (SDR signal) assuming a conventional display luminance dynamic range (Standard Dynamic Range: SDR) is premised on viewing in an environment having an EOTF such as γ = 2.2, whereas an HDR image (HDR signal) is premised on viewing in an environment with EOTF such as PQ. For this reason, when the HDR signal is reproduced in a viewing environment assuming the SDR signal, an image different from the producer's intention is displayed. As a method of avoiding such a problem, for example, although the effect of HDR is lost, the gradation value of the HDR signal is converted (compressed) into the gradation value of the SDR signal for viewing in the SDR environment. , tone mapping can be considered. Note that the HDR signal can be given static metadata called maxCLL (maximum Content Light Level), which represents the maximum brightness of the content. By referring to maxCLL, the playback device can specify the maximum luminance of received HDR content.

Patent Documents 1 and 2 disclose tone mapping for converting an HDR signal into an SDR signal. Document 1 describes a configuration for designing tone mapping characteristics using an estimated maximum luminance level using a histogram shape of luminance values of an HDR signal and maxCLL. Further, Document 2 describes a configuration in which tone mapping characteristics are designed in accordance with a geometric region of an image in order to suppress tone jumps of a low-frequency object such as the sky.

JP 2017-184220 A JP 2018-093530 A

In the method of Patent Literature 1, for example, even if the HDR signal is captured with underexposure, the maximum luminance level of the HDR signal is mapped to the maximum value of the EOTF domain of the SDR. Therefore, viewing the SDR signal after mapping in the SDR environment may make the viewer feel brighter than viewing the HDR signal in the HDR environment.

Also, the method of Patent Literature 2 does not perform tone mapping in consideration of the D range of the HDR image, so the output SDR signal may not have the brightness intended by the creator.

The present invention has been made in view of such problems of the prior art. SUMMARY OF THE INVENTION It is an object of the present invention to provide an image processing apparatus capable of converting an HDR signal into an SDR signal so as to suppress changes in brightness between viewing in an HDR environment and viewing in an SDR environment, and a control method thereof. and

The above object is an image processing device that records HDR (High Dynamic Range) signals as a file, and according to the shooting conditions of the HDR signal, among a plurality of different output dynamic ranges preset in the image processing device , Acquisition means for acquiring information indicating the peak luminance of one output dynamic range according to the shooting conditions of the HDR signal, and recording means for recording a first value based on the information acquired by the acquisition means in a file together with the HDR signal. and is achieved by an image processing device characterized by having:

According to the present invention, an image processing apparatus capable of converting an HDR signal into an SDR signal so as to suppress a change in brightness between viewing in an HDR environment and viewing in an SDR environment and a control method thereof are provided. can be done.

Diagram showing gamma curves for shooting modes with different peak brightness Cross-sectional view of a typical interchangeable-lens digital single-lens reflex camera Block diagram showing an example of the configuration of the electrical circuit of the camera and interchangeable lens A diagram for explaining the tone mapping operation in the first embodiment. Flowchart relating to maxDRL calculation operation in the first embodiment The figure which showed the characteristic of PQ (Perceptual Quantization) which is EOTF standardized by SMPTE ST2084. A diagram showing an example of a data file structure used in the embodiment Flowchart relating to tone mapping operation in the first embodiment FIG. 4 is a diagram for explaining the update operation of maxDRL in the second embodiment; Flowchart for update operation of maxDRL in the second embodiment A diagram for explaining an example of retouching and an example of calculation of maxDRL_r in the second embodiment. Flowchart for update operation of tone mapping and maxDRL in the third embodiment A diagram showing an example of a data file structure used in the fifth embodiment

● (first embodiment)
The invention will now be described in detail on the basis of its exemplary embodiments with reference to the accompanying drawings. It should be noted that the described embodiments are merely examples and do not limit the scope of the present invention. For example, an embodiment in which the present invention is applied to a digital camera will be described below. However, a digital camera is only one example of an image processing device to which the present invention can be applied. The present invention can be implemented in any electronic device. Such electronic devices include, but are not limited to, imaging devices such as digital cameras and digital video cameras, as well as personal computers, tablet terminals, mobile phones, game machines, drive recorders, robots, and drones.

● (first embodiment)
FIG. 2 is a cross-sectional view showing an arrangement example of main optical members, sensors, etc. of a digital camera (hereinafter referred to as a camera) according to the embodiment. The camera in this embodiment is a lens-interchangeable digital single-lens reflex camera, and has a camera body 1 and an interchangeable lens 2 . The imaging element 10 in the camera body 1 is, for example, a CMOS image sensor or a CCD image sensor, and has a plurality of pixels (accumulation photoelectric conversion elements) arranged therein. A mechanical shutter 11 provided near the front of the image sensor 10 controls the exposure timing and exposure time of the image sensor 10 .

A semi-transmissive main mirror 3 and a first reflecting mirror 7 arranged behind the main mirror 3 flip up when photographing. The second reflecting mirror 8 further reflects the light beam reflected by the first reflecting mirror 7 and makes it enter a focus detection sensor (AF sensor) 9 . The AF sensor 9 may be, for example, an imaging device with a smaller number of pixels than the imaging device 10 .

The first reflecting mirror 7, the second reflecting mirror 8, and the focus detection sensor 9 are configured to perform focus detection by the phase difference detection method at an arbitrary position within the photographing screen. A photometric sensor (AE sensor) 6 receives the image of the photographic screen reflected by the pentaprism 4 and the third reflecting mirror 5 . The AE sensor 6 divides the light receiving part into a plurality of areas, and can output the brightness information of the subject for each area. There is no limit to the number of divisions. In addition to the pixels arranged in the light-receiving portion, the image sensor includes an amplifier circuit for pixel signals, a peripheral circuit for signal processing, and the like.

The pentaprism 4 constitutes a finder optical system. Although not shown in FIG. 2, the subject image reflected by the pentaprism 4 can be observed through the eyepiece. A portion of the light beam reflected by the main mirror 3 and diffused by the focusing plate 12 is incident on the AE sensor 6 . The interchangeable lens 2 communicates with the camera body 1 through contacts of a lens mount provided on the camera body 1 as necessary. Since the main mirror 3 is flipped up during live view display and video recording, exposure control and focus adjustment control are performed using captured image information.

FIG. 3 is a block diagram showing a configuration example of an electrical circuit of the camera body 1 and its interchangeable lens 2 shown in FIG. In the camera main body 1, the control section 21 is a one-chip microcomputer containing, for example, an ALU (ARITHMETIC and Logic Unit), ROM, RAM, A/D converter, timer, serial communication port (SPI), and the like. The control unit 21 controls operations of the camera body 1 and the interchangeable lens 2 by loading, for example, a program stored in the ROM into the RAM and executing the program. A specific operation of the control unit 21 will be described later.

Output signals from the AF sensor 9 and the AE sensor 6 are connected to A/D converter input terminals of the control section 21 . The signal processing circuit 25 controls the image sensor 10 according to instructions from the control unit 21, applies A/D conversion and signal processing to the signal output from the image sensor 10, and obtains an image signal. Further, the signal processing circuit 25 performs necessary image processing such as compression and synthesis when recording the obtained image signal. The memory 28 is a DRAM or the like, and is used as a work memory when the signal processing circuit 25 performs various signal processing, or as a VRAM when displaying an image on the display 27, which will be described later. A display 27 is a rear liquid crystal display of the camera body 1 or an external display, and displays information such as camera setting values, messages, menu screens, and captured images. The display 27 is assumed to be capable of HDR (High Dynamic Range) display. The display 27 is controlled by instructions from the control section 21 . The storage unit 26 is, for example, a semiconductor memory card, and a captured image signal is input from the signal processing circuit 25 .

The motor 22 raises and lowers the main mirror 3 and the first reflecting mirror 7 and charges the mechanical shutter 11 under the control of the controller 21 . The operation unit 23 is a group of input devices such as switches used by the user to operate the camera. The operation unit 23 includes a release switch for instructing the start of photographing preparation operations and the start of photographing, a photographing mode selection switch for selecting a photographing mode, a direction key, an enter key, and the like. The contact portion 29 is a contact for communicating with the interchangeable lens 2, and the input/output signal line of the serial communication port of the control portion 21 is connected. A shutter drive unit 24 is connected to an output terminal of the control unit 21 to drive the mechanical shutter 11 .

The interchangeable lens 2 is provided with a contact portion 50 paired with the contact portion 29 . A lens control unit 51, which is a one-chip microcomputer similar to the control unit 21, is connected to the contact unit 50, and communication with the control unit 21 is possible. The lens control unit 51 has, for example, a microprocessor, a ROM, and a RAM, and loads a program stored in the ROM into the RAM and executes it, thereby controlling the operation of the interchangeable lens 2 based on instructions from the control unit 21. do. Also, the lens control unit 51 notifies the control unit 21 of information such as the state of the interchangeable lens 2 . The focus lens driving section 52 is connected to the output terminal of the lens control section 51 and drives the focus lens. A zoom drive unit 53 changes the angle of view of the interchangeable lens under the control of the lens control unit 51 . The diaphragm drive unit 54 adjusts the opening amount of the diaphragm under the control of the lens control unit 51 .

When the interchangeable lens 2 is attached to the camera body 1, data communication between the lens control section 51 and the control section 21 of the camera body is enabled via the contacts 29 and 50. FIG. Electric power for driving the motors and actuators in the interchangeable lens 2 is also supplied through the contact portions 29 and 50 . Optical information specific to the lens necessary for the control unit 21 of the camera body to perform focus detection and exposure calculation, information on the subject distance based on the distance encoder, etc. are transmitted from the lens to the control unit 21 of the camera body. Output by Focus adjustment information and aperture information obtained as a result of focus detection and exposure calculation performed by the control unit 21 of the camera body are output from the control unit 21 of the camera body to the lens by data communication, and the lens operates according to the focus adjustment information. The aperture of the focus lens is controlled according to the aperture information.

Specific operations from photographing to development in the first embodiment will be described below with reference to FIGS. 1 and 3 to 8. FIG. When the controller 21 becomes operable by, for example, turning on the power switch included in the operation unit 23 in FIG. Initialization processing such as obtaining necessary information on various lenses is performed. Further, the operation unit 23 accepts various user settings and sets an arbitrary shooting mode. By half-pressing the shutter switch included in the operation unit 23, the control unit 21 starts AF (autofocus) processing and AE (automatic exposure) processing. After that, by fully pressing the shutter switch, the control unit 21 starts the operation of the photographing process.

After that, the RAW image obtained by photographing is developed. Development of a RAW image will be described with reference to FIG. In FIG. 4, 402 to 405 and 408 to 412 describe the processing performed by the control unit 21 as functional blocks, but one or more of them may be implemented as hardware circuits. Each piece of pixel data that constitutes the RAW image data 401 represents the intensity of the color of the color filter provided in the corresponding pixel, and does not have information on other colors. Here, it is assumed that each pixel of the imaging device is provided with a color filter of R (red), G (green), or B (blue).

The white balance unit 402 performs processing for correcting the color cast caused by the light source and reproducing white. Specifically, the white balance unit 402 plots the RGB data of each pixel forming the RAW image data 401 in a predetermined color space such as the xy color space. Then, the white balance unit 402 integrates the R, G, and B data plotted in the vicinity of the black body radiation locus, which is highly likely to be the light source color in the color space, and calculates the white balance of the R and B components from the integrated value. Determine the coefficients G/R and G/B. A white balance unit 402 performs white balance processing by applying white balance coefficients to the image data.

A color interpolation unit 403 generates a color image in which all pixels have R, G, and B color information by noise reduction processing and color interpolation processing. The generated color image passes through a matrix conversion unit 404 and a gamma conversion unit 405 to generate a basic color image. Here, the gamma characteristic in the case of HDR development in the gamma conversion unit 405 is the inverse characteristic of EOTF (Inverse EOTF), for example, the inverse characteristic of PQ (Perceptual Quantization) shown in FIG. 6 (FIG. 6). Note that an OETF (Optical-Electro Transfer Function) combining Inverse EOTF and OOTF (Opto-Optical Transfer Function) characteristics may be used.

The gamma conversion unit 405 performs gamma conversion using a contrast adjustment curve (also called a contrast adjustment gradation characteristic, a gamma curve, or a gamma characteristic). Conventional cameras are equipped with shooting modes in which the output D range changes as shown in FIG. 1, and the user selects which mode to shoot in accordance with the scene. At this time, the difference in D range between the two modes is the difference in peak luminance during HDR development. Similarly, the D range may also differ depending on the ISO sensitivity. Compared to the standard ISO sensitivity of 100 where no gain is applied by the amplifier in the image pickup device, high sensitivity after ISO 200 where analog gain or digital gain is applied has a smaller amount of charge relative to the volume of the photodiode, resulting in a larger D range. . Regarding intermediate ISO sensitivities in steps of 1/3 step, in an imaging device having an inexpensive amplifier with only one gain step, the intermediate ISO sensitivities are realized by increasing or decreasing the gain from the representative ISO sensitivity.

At this time, when the gain is lowered, the saturation signal level is lowered, so the D range becomes smaller. As described above, the peak luminance may differ depending on the shooting conditions, especially the ISO sensitivity. Therefore, gamma curves corresponding to the number of combinations of shooting modes and ISO sensitivities are previously stored in the memory 28 as contrast adjustment curves 407 . Then, the gamma selection unit 408 refers to the shooting conditions (shooting mode and ISO sensitivity) 406 , reads out the contrast adjustment curve 407 corresponding to the combination of shooting mode and ISO sensitivity from the memory 28 , and sends it to the output peak luminance information acquisition unit 409 . Output.

Next, in S501 (FIG. 5), the output peak luminance information obtaining unit 409 obtains the absolute luminance corresponding to each output value of the gamma curve from the obtained contrast adjustment curve 407 . Absolute luminance values can be obtained using EOTFs such as PQ (FIG. 6) and HLG. Then, the output peak luminance information acquisition unit 409 acquires the maximum value of the output values of the contrast adjustment curve 407 or the nits value corresponding thereto. In this specification, the maximum output value of the gamma curve data, that is, the gradation value of the peak luminance value in the output D range or the corresponding nits value is referred to as maxDRL (maximum Dynamic Range Level).

After that, the color image is subjected to processing for improving the appearance of the image by the color luminance adjustment unit 410. For example, image correction such as chroma saturation enhancement is performed by detecting the evening scene according to the scene. When this color brightness adjustment is completed, the compression unit 411 compresses the high-resolution image by a method such as HEVC. Then, in S502, the recording unit 412 describes the maxDRL acquired by the output peak luminance information acquisition unit 409 in the metadata of the file, generates an HDR (High Dynamic Range) signal 413, and outputs it in S503. As a result, maxDRL is recorded in association with the HDR signal 413 .

FIG. 7 shows an example of a file structure for HDR signal recording. Header information such as a thumbnail image 72 is stored in 71, and information related to the main image such as metadata 74 and main image data 75 is stored in 73. FIG. In S502, maxDRL may be recorded in the metadata 74 of FIG. 7, or may be described in another specific area. In practice, a minimum depth of 10 bits is required to represent an HDR signal in PQ, but the conventional JPEG format requires 8 bits, so a new container for still image HDR must be adopted. Here, a container in HEIF (High Efficiency Image File Format) format, which is an image file format defined by MPEG (Moving Picture Experts Group)-H Part 12 (ISO/IEC 23008-12), is used. HEIF has the feature that not only the main image but also thumbnails, multiple temporally related images, and metadata such as EXIF and XMP can be stored in one file. Therefore, HEVC-encoded 10-bit image sequences can also be stored. The aforementioned maxCLL is not defined in the HEIF standard, but in this proposal, not only maxCLL but also maxDRL is stored in Exif MakerNote.

Furthermore, in order to view the HDR signal 413 in an SDR environment, a method of tone mapping to an SDR (Standard Dynamic Range) signal using maxDRL will be described with reference to FIG. Although this processing can be performed by the control unit 21 or the signal processing circuit 25, it is assumed here that the control unit 21 performs the processing.

In S801, the control unit 21 sets the upper limit value Lmax and the lower limit value Lmin when displaying the HDR signal at the maximum value of the EOTF value range for the SDR signal. For example, Lmax can be 2,000 nits and Lmin can be 100 nits, but they are not limited to these. Also, the upper limit value Lmax and the lower limit value Lmin may be settable by the user.

Next, in S802, the control unit 21 acquires maxDRL from the metadata of the file in which the HDR signal is recorded.
In S803, the control unit 21 compares Lmax and maxDRL. Here, if maxDRL is equal to or greater than Lmax, in S804 the control unit 21 performs tone mapping so that Lmax becomes the maximum value of the EOTF domain of SDR.

On the other hand, if maxDRL is less than Lmax, the control unit 21 compares maxDRL with Lmin in S805. If maxDRL is equal to or less than Lmin, in S806 the control unit 21 performs tone mapping so that Lmin becomes the maximum value of the EOTF domain of SDR. If maxDRL is greater than Lmin, in S807 the control unit 21 performs tone mapping such that maxDRL is the maximum value of the EOTF domain of SDR.

Next, in S<b>808 , the control unit 21 displays the SDR signal obtained by tone mapping on the display 27 .
In S809, the control unit 21 displays a GUI on the display 27, for example, to inquire of the user whether to save the tone-mapped SDR signal in the SDR format. For example, if the user instructs to save the signal through the operation unit 23, the control unit 21 saves the SDR signal in the SDR format in the storage unit 26 in S810. When the user instructs not to store the SDR signal, the control unit 21 terminates the process without storing the SDR signal.

As described above, in the present embodiment, when an HDR signal is recorded in a file, maxDRL, which is peak luminance information of the output D range, is described as metadata. As a result, the dynamic range of the HDR signal can be taken into consideration when mapping to the SDR signal, and the brightness when the HDR signal is viewed in the HDR environment and the brightness when the SDR signal is mapped and viewed in the SDR environment difference can be suppressed.

● (Second embodiment)
Next, a second embodiment of the present invention will be described with reference to FIGS. 9-11. In the first embodiment, a method of describing maxDRL, which is the peak luminance information of the output D range, as metadata when recording an HDR signal into a file has been described. In the second embodiment, a method of updating maxDRL when retouching an HDR signal recorded in a file so as to change the peak luminance of the output D range by tone curve correction or the like will be described.

In FIG. 9, 902 to 905 describe the processing performed by the control unit 21 as functional blocks, but one or more of them may be implemented as hardware circuits.
In S<b>1001 ( FIG. 10 ), the output peak luminance information acquisition unit 902 acquires maxDRL described in the metadata of the HDR signal 901 .
In S<b>1002 , the color/luminance adjustment unit 903 displays an image based on the HDR signal 901 and a GUI for image retouching such as tone curve correction on the display device 27 . Also, the color brightness adjustment unit 903 corrects the color and brightness of the image according to the user's instruction through the operation unit 23 .
In S1003, the output peak luminance information calculation unit 904 after color luminance adjustment calculates the peak luminance information maxDRL_r of the output D range in which the image correction by retouching is reflected in the maxDRL acquired by the output peak luminance information acquisition unit 902. . Let x be the gradation value of the input signal before correction, y be the gradation value after correction, and f be the input/output characteristics of the correction.
y=f(x) (Formula 1)

At this time, if the input/output characteristics f are corrected for the tone values in the range of 0≦x≦maxDRL, and the maximum value of the tone values y after correction is ymax, the corrected HDR The gradation value of the signal satisfies 0≦y≦ymax. At this time, the output peak luminance information calculation unit 904 obtains maxDRL_r by the following (Equation 2).
maxDRL_r=ymax (Formula 2)

Examples of retouching and calculation of maxDRL_r are shown in FIGS. 11(a) to 11(c). FIG. 11(a) shows the luminance histogram before retouching, FIG. 11(b) shows the luminance histogram after retouching to increase the luminance, and FIG. 11(c) shows the luminance histogram after retouching to decrease the luminance. . When retouching as shown in FIG. 11(b) is performed, maxDRL_r becomes larger than maxDRL, and conversely, when retouching as shown in FIG. 11(c) is performed, maxDRL_r becomes smaller than maxDRL.
In S1004, the recording unit 905 records maxDRL_r in the metadata 74 of the HDR signal.
In S<b>1005 , the recording unit 905 records the HDR signal file in the storage unit 26 .

In addition, by applying the first embodiment to the retouched HDR signal and performing tone mapping such that maxDRL_r is the maximum value of the EOTF domain of SDR, the retouched HDR signal is also suitable in the SDR environment. You can see it in the light.

● (Third Embodiment)
Next, a third embodiment of the invention will be described with reference to FIG. In the first and second embodiments, procedures for viewing stored HDR signals in an SDR environment have been described. In the present embodiment, when an HDR signal is retouched such as tone curve correction in an SDR environment, a method of performing tone mapping in real time so that the HDR signal during and after retouching can be viewed in an SDR environment will be described.

In S1201 to S1203, output D-range peak luminance information maxDRL_r with retouch information added is calculated by the same procedure as in S1001 to S1003 of the second embodiment. Next, in S1204 to S1210, the HDR signal is tone-mapped to the SDR signal and displayed on the display 27 by the same procedure as in S801 to S808 of the first embodiment. The display here corresponds to viewing in an SDR environment.

Next, in S1211, the control unit 21 displays a GUI on the display 27, for example, to inquire of the user whether to save the retouched HDR signal as a new file. For example, if the user instructs to save via the operation unit 23, the control unit 21 saves the HDR signal with maxDRL_r saved in the metadata 74 in the storage unit 26 in S1212, and advances the process to S1213. If the user instructs not to save the HDR signal, the control unit 21 advances the process to S1213 without saving the HDR signal.
Since S1213 to S1214 are the same as S809 to S810 of the first embodiment, description thereof is omitted.

According to this embodiment, even if an HDR signal is retouched in an SDR environment, maxDRL can be saved and recorded in a file of the HDR signal after retouching. Therefore, when viewing an HDR signal in an SDR environment, by referring to maxDRL, the maximum luminance value of the output D range after retouching can be mapped to the maximum value of the EOTF domain of the SDR signal.

Note that even when a RAW file is developed into an HDR format using development software or the like in an SDR environment, a tone-mapped HDR signal to an SDR signal is displayed according to the procedure of the third embodiment. may

● (Fourth Embodiment)
Next, a fourth embodiment of the invention will be described. In the second embodiment, maxDRL as metadata is updated when the peak luminance is changed due to image retouching. On the other hand, in this embodiment, even if the peak luminance is changed due to image retouching, maxDRL as metadata is not updated. Assume that retouching, eg changing the sharpness intensity changes the peak luminance of the image and updates maxDRL. In this case, if tone mapping to SDR is performed based on maxDRL, the gradation of the image will change depending on the strength of sharpness. However, for example, when the user wants to check the image while changing the strength of sharpness, it is desirable to change only the effect of sharpness without changing the gradation of the image. Therefore, in the present embodiment, even when the peak luminance is changed due to image retouching, maxDRL as metadata is not updated. Note that the mode in which maxDRL is updated as in the second embodiment and the mode in which maxDRL is not updated as in this embodiment may be selectable by a user operation or automatically.

● (Fifth embodiment)
In the above embodiment, the explanation has mainly been given assuming a still image file. In this embodiment, an embodiment assuming a moving image file will be described.
FIG. 13(a) shows the data structure of the MP4 format, which is an example of the moving image file format. The MP4 format data structure includes an area 1301 for recording metadata and an area 1304 for recording a moving image stream 1305 . An area 1301 includes an EXIF area 1302 and a thumbnail area 1303 , and maxDRL is recorded in the EXIF area 1302 . However, for moving image data, if there is a mode change during recording, the peak luminance may change in the middle of continuously recorded moving image data. In this case, the problem is what to record as maxDRL in the EXIF area 1302 . Therefore, in the present embodiment, the peak luminance (the peak luminance of the start frame of the moving image data) based on the setting at the start of moving image shooting is recorded in the EXIF area 1302 as maxDRL. As maxDRL, another value may be used as long as it represents the peak luminance of moving image data. For example, the peak luminance of the end frame of the video data, the maximum value of the peak luminances of all frames, the minimum value of the peak luminances of all frames, the average value of the peak luminances of all frames, and the peak luminance of all frames. A median value or the like can be used.

In some file formats, metadata can be recorded for each frame as shown in FIG. 13(b). 1311, 1312, 1313 and 1314 are the same as 1301, 1302, 1303 and 1304 in FIG. 13(a). A region 1315 is the first frame of the moving image, and a region 1317 is the second frame of the moving image. Furthermore, the file format of FIG. 13(b) includes a metadata area for each frame. An area 1316 is a metadata area for the first frame, and an area 1318 is a metadata area for the second frame. In the case of such a file format, the peak luminance for each frame may be recorded as maxDRL. In this case, the peak luminance for each frame is recorded in metadata areas 1316 and 1318 for each frame. Of course, the above representative peak luminance may be further described in the EXIF area 1312, which is the metadata area of the entire file.

● (other embodiments)
In the above embodiment, maxDRL is described as file metadata, but it is not necessarily limited to handling as file metadata. For example, when a digital camera and an HDR display are directly connected via HDMI (registered trademark) to output an HDR signal, the following configuration is conceivable. For example, a digital camera reads a value corresponding to maxDRL as metadata of an HDR signal to a recording medium for output signals, and inputs the value to an HDR display.

The present invention supplies a program that implements one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and one or more processors in the computer of the system or device reads and executes the program. It can also be realized by processing to It can also be implemented by a circuit (for example, ASIC) that implements one or more functions.

DESCRIPTION OF SYMBOLS 1... Camera main body 2... Interchangeable lens 10... Imaging element 21... Control part 25... Signal processing circuit 51... Lens control part 405... Gamma conversion part 410... Color brightness conversion part

Claims (18)

An image processing device that records an HDR (High Dynamic Range) signal as a file,
Acquiring information indicating a peak luminance of one output dynamic range corresponding to the HDR signal imaging conditions among a plurality of different output dynamic ranges preset in the image processing device according to the HDR signal imaging conditions. a obtaining means for
recording means for recording a first value based on the information acquired by the acquisition means in the file together with the HDR signal;
An image processing device comprising: further having an imaging element for imaging a subject,
The HDR signal is generated from a signal obtained from the imaging element,
The shooting conditions include a shooting mode,
2. The image processing apparatus according to claim 1, wherein said shooting mode is one of a plurality of shooting modes corresponding to said plurality of different output dynamic ranges. 3. The image processing apparatus according to claim 2, wherein the gradation characteristic for contrast adjustment applied to the signal obtained from the image sensor differs according to the shooting mode included in the shooting conditions. The plurality of different output dynamic ranges are determined according to the contrast adjustment gradation characteristics, and the peak luminance is the maximum output value in the contrast adjustment gradation characteristics of the one output dynamic range. 4. The image processing apparatus according to claim 3. 5. An image according to any one of claims 1 to 4, further comprising updating means for updating said first value when said peak luminance changes due to color or luminance corrections to said HDR signal. processing equipment. 5. The image processing according to any one of claims 1 to 4, wherein the first value is not updated even when the peak luminance changes due to color or luminance correction for the HDR signal. Device. Conversion means for converting the HDR signal into a signal of the predetermined format by converting the grayscale values of the HDR signal into grayscale values in a predetermined format having a different dynamic range based on the first value. 7. The image processing apparatus according to any one of claims 1 to 6, further comprising: 8. The image processing apparatus according to claim 7, wherein the different dynamic range is SDR (Standard Dynamic Range). 9. The image processing apparatus according to claim 1, wherein said file is recorded in HEIF (High Efficiency Image File Format). An image processing device that processes HDR (High Dynamic Range) signals,
obtaining means for obtaining a first value recorded in association with the HDR signal , wherein the first value is selected from among a plurality of different output dynamic ranges preset in the image processing apparatus; Shows the peak luminance of one output dynamic range according to the shooting conditions of the HDR signal,
transforming the HDR signal into a signal of the predetermined format by tone mapping the grayscale values of the HDR signal to grayscale values in a predetermined format having a different dynamic range based on the first value; means and
An image processing device comprising: When the first value is smaller than a predetermined second value, the conversion means performs tone mapping so that the first value becomes the maximum gradation value in the predetermined format. 11. The image processing apparatus according to claim 10. When the first value is larger than a predetermined second value, the converting means performs tone mapping so that the second value becomes the maximum gradation value in the predetermined format. 12. The image processing apparatus according to claim 10 or 11. 13. The image processing apparatus according to any one of claims 10 to 12, wherein said predetermined format signal is an SDR (Standard Dynamic Range) signal. 14. The image processing apparatus according to any one of claims 10 to 13, wherein the HDR signal is recorded as a file in HEIF (High Efficiency Image File Format) format. A control method for an image processing device that records an HDR (High Dynamic Range) signal as a file,
an obtaining step of obtaining information indicating a peak luminance of one output dynamic range according to the shooting conditions of the HDR signal among a plurality of different output dynamic ranges preset in the image processing device ;
a recording step of recording a first value based on the information obtained in the obtaining step in the file together with the HDR signal;
A control method for an image processing device, comprising: A control method for an image processing device that records an HDR (High Dynamic Range) signal as a file,
an acquiring step of acquiring a first value recorded in association with the HDR signal , wherein the first value is one of a plurality of different output dynamic ranges preset in the image processing device; Among them, showing the peak luminance of one output dynamic range according to the shooting conditions of the HDR signal,
transforming the HDR signal into a signal of the predetermined format by tone mapping the grayscale values of the HDR signal to grayscale values in a predetermined format having a different dynamic range based on the first value; process and
A control method for an image processing device, comprising: A program for causing a computer to function as each unit included in the image processing apparatus according to any one of claims 1 to 9. A program for causing a computer to function as each unit included in the image processing apparatus according to any one of claims 10 to 14.

Images