JP6355345B2 - Image processing apparatus, image processing method, display apparatus, display method, and program - Google Patents

Description

  The present invention relates to an image processing device, an image processing method, a display device, a display method, and a program.

In recent years, HDR (High Dynamic) which is image data with a wide dynamic range.
Range) Opportunities to handle image data are increasing. Further, JPEG-HDR has been proposed as a format of HDR image data.
JPEG-HDR is a format in which HDR image data is expressed by base image data and difference data. The difference data includes, for example, luminance difference data, color difference data, or both.

The HDR image data is, for example, image data in which each of the Y value, the Cb value, and the Cr value is a gradation value corresponding to 32 bits. The HDR image data may be YCbCr data in which the pixel value is a combination of the Y value, the Cb value, and the Cr value, and the pixel value is a combination of the R value, the G value, and the B value. It may be RGB data.
The base image data is JPEG standard image data obtained by down-sampling (gradation compression) HDR image data. The base image data is, for example, image data in which each of the Y value, Cb value, and Cr value is a gradation value equivalent to 8 bits. The HDR image data may be YCbCr data or RGB data.

  The luminance difference data is data used in the luminance range expansion process for expanding the dynamic range of the image data, and is data representing the difference between the luminance values of the HDR image data and the base image data. For example, the luminance difference data is a ratio between the luminance value (gradation value) of the base image data and the luminance value (gradation value) of the HDR image data in pixel units (or in units of regions including a predetermined number of pixels). This is luminance ratio data representing the luminance ratio. That is, the luminance difference data is the ratio of the luminance value (gradation value) of the HDR image data to the luminance value (gradation value) of the base image data or the reciprocal thereof in pixel units (or an area unit composed of a predetermined number of pixels). Is luminance ratio data representing However, the luminance difference data is obtained by subtracting one from the luminance value (gradation value) of the base image data and the luminance value (gradation value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). Data representing a luminance difference value which is a subtracted difference value may be used. Further, the luminance difference data may be luminance conversion table data representing a correspondence relationship between the input luminance value and the output luminance value in the luminance range expansion process. Note that the luminance range expansion processing can also be said to be processing for reproducing luminance that cannot be expressed in the base image data.

  The color difference data is data used in the color gamut expansion process for expanding the color gamut of the image data, and is data representing the color difference between the HDR image data and the base image data. For example, the color difference data is a color difference value (Cb value, Cr value) of the base image data and a color difference value (Cb value, Cr value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). This is data representing a color difference value which is a difference value obtained by subtracting the other from one. However, the color difference data is a color difference value (Cb value, Cr value) of the base image data and a color difference value (Cb value, Cr value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). Color difference ratio data representing a color ratio that is a ratio of Further, the color difference value and the color ratio may be values calculated using R value, G value, and B value instead of the color difference value. Note that the color gamut expansion processing can be said to be processing for reproducing colors that cannot be expressed by the base image data.

By using JPEG-HDR, it is possible to display both JPEG standard image data and HDR image data that have been used conventionally (Patent Document 1). Conventionally used JPEG image data is, for example, image data in which each of a Y value, a Cb value, and a Cr value is a gradation value equivalent to 8 bits. In order to display JPEG standard image data used in the past, the base image data may be displayed. When it is desired to display the HDR image data, the HDR image data may be restored using the base image data and the difference data, and the restored HDR image data may be displayed. More specifically, HDR image data may be restored using base image data and at least one of luminance difference data and color difference data.

  As described above, the HDR image data is image data in which each of the Y value, the Cb value, and the Cr value is a gradation value equivalent to 32 bits. However, the number of bits of the gradation value that can be displayed by the display device is not necessarily 32 bits. For example, the number of bits of the gradation value that can be displayed on the display device may be 10 bits, which is less than 32 bits. In such a case, the number of bits of the HDR image data is reduced to the number of bits that can be displayed by the display device, and the HDR image data after the reduction of the number of bits is displayed. That is, the dynamic range of the HDR image data is reduced to the dynamic range of the image data that can be displayed on the display device, and the HDR image data after the reduction of the dynamic range is displayed.

When JPEG-HDR becomes widespread, it is conceivable that captured HDR image data is streamed and reproduced at the shooting site. Specifically, it is conceivable that the image capturing device generates JPEG-HDR data by image capturing, and sequentially performs processing for outputting the generated JPEG-HDR data to a display device (display). Then, it is conceivable that the display device sequentially acquires JPEG-HDR data, restores HDR image data from the acquired data, and displays the restored HDR image data. Accordingly, it is possible to check the photographing result in real time using the display device as a display for live view.
Further, it is conceivable that JPEG-HDR data is sent from the image processing apparatus to the display apparatus, not limited to the shooting site. Then, it is conceivable that the display device restores the HDR image data from the acquired JPEG-HDR data and displays the restored HDR image data.

It is conceivable that stream-format HDR image data is output to a display device using an existing SDI cable used at a shooting site or an image production site. In other words, it is conceivable that the HDR image data is stream-transferred to the display device.
By using the SDI cable, it is possible to obtain an advantage that an existing cable can be used and an advantage that a plurality of devices can be connected in a daisy chain.
However, the SDI cable cannot transmit JPEG-HDR files (file format JPEG-HDR data).
In addition, HDR image data can be appropriately displayed on an HDR compatible device (display device that supports HDR image data), but HDR image data cannot be displayed on a non-HDR compatible device (display device that does not support HDR image data). It cannot be displayed properly. Therefore, it is desirable to output JPEG-HDR data to the display device rather than outputting HDR image data to the display device.
In addition, if various difference data are used, various HDR image data can be obtained and various HDR image data can be confirmed.

In recent years, a technique for expanding the dynamic range of a display image (an image displayed on a screen) by controlling the light emission luminance of a backlight of a liquid crystal display device for each region is becoming common.
If not only the dynamic range of the base image data but also the backlight emission brightness can be controlled based on the difference data, the brightness that could not be displayed simply by controlling the transmittance of the liquid crystal panel is reduced. Can be assigned to. As a result, a display image with a wider dynamic range can be obtained. For example, when the number of bits of the gradation value that can be displayed on the display device is 10 bits, a display image having a wider dynamic range than the dynamic range corresponding to 10 bits can be obtained.
From such a viewpoint, it is desirable to output JPEG-HDR data to a display device.

For example, Patent Document 2 discloses a conventional technique related to transmission of JPEG image data. In the technique disclosed in Patent Document 2, only image data is transmitted when 4: 1: 1 image data is transmitted, and not only image data but also enhancement is transmitted when 4: 2: 2 image data is transmitted. Data is also transmitted.
In the technique disclosed in Patent Document 2, 4: 1: 1 compressed image data (first compressed image data) is generated by adding and coding luminance (Y) data and color (C) data. The
In the technique disclosed in Patent Document 2, color (C) data is up-converted. Then, the luminance (Y) data and the up-converted color (C) data are added and subjected to compression encoding, thereby generating 4: 2: 2 compressed image data (second compressed image data). Thereafter, the difference between the first compressed image data and the second compressed image data is acquired as enhancement data.
In the technique disclosed in Patent Document 2, the generated first compressed image data and enhancement image data are transmitted.

If the technique disclosed in Patent Document 2 is used, the transmission target data can be controlled so that the base image data and the difference data are transmitted when the HDR image data is transmitted.
However, when the technique disclosed in Patent Document 2 is used, when restoring HDR image data from base image data and difference data, wrong difference data may be used, and HDR image data may not be restored accurately. The difference data A2 corresponding to the base image data A1 and the difference data B2 corresponding to the base image data B1 may be input to a device that accepts a plurality of data. In such a case, erroneous HDR image data may be restored using the base image data A1 and the difference data B2. In addition, erroneous HDR image data may be restored using the base image data B1 and the difference data A2. If incorrect difference data is used, the quality of the display image is degraded.

JP 2011-193511 A Japanese Patent Laid-Open No. 10-200921

  A first object of the present invention is to provide a technique capable of transmitting HDR image data in a stream format so that an appropriate image display can be performed by either an HDR compatible device or an HDR non-compatible device. . A second object of the present invention is to provide a technique capable of accurately restoring HDR image data using base image data and difference data.

The first aspect of the present invention is:
Based on input image data, base image data having at least one of a dynamic range and a color gamut narrower than the input image data, and difference data used in an enlargement process for enlarging at least one of the dynamic range and the color gamut of the image data And generating means for performing a generating process for generating
An adding means for performing an adding process of adding correspondence information representing a correspondence relationship between the base image data and the difference data to at least one of the base image data and the difference data;
Output means for individually outputting the base image data and the difference data after the additional processing is performed;
Have
Wherein in addition processing, the difference data generated by the generation processing, an image processing apparatus characterized by displaying failure information indicating the inability to display on the display device Ru is added.

The second aspect of the present invention is:
Acquisition means for individually acquiring input base image data that is base image data, and input difference data that is difference data used in enlargement processing for enlarging at least one of the dynamic range and color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. Determining means for determining whether the data corresponds to the base image data;
The input difference data is acquired by the acquisition means, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed as the input A range expansion image data is generated by applying to the base image data, and if the input difference data is not acquired by the acquisition unit, a generation unit that does not generate the range expansion image data ;
When the range expansion image data is generated by the generation unit, an image based on the range expansion image data is displayed. When the range expansion image data is not generated by the generation unit, the input base image data is displayed. Display means for displaying an image based on;
Data required for the processing for generating the range expansion image data is acquired when the range expansion image data is not generated by the generation unit and an image based on the input base image data is displayed by the display unit. First notification means for notifying the user that it has not been performed;
It is a display device characterized by having.
The third aspect of the present invention is:
Acquisition means for individually acquiring input base image data that is base image data, and input difference data that is difference data used in enlargement processing for enlarging at least one of the dynamic range and color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. Determining means for determining whether the data corresponds to the base image data;
The input difference data is acquired by the acquisition means, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed as the input A range expansion image data is generated by applying to the base image data, and if the input difference data is not acquired by the acquisition unit, a generation unit that does not generate the range expansion image data;
When the range expansion image data is generated by the generation unit, an image based on the range expansion image data is displayed. When the range expansion image data is not generated by the generation unit, the input base image data is displayed. Display means for displaying an image based on;
The display means displays an image based on the input base image data when the generation means generates the range expansion image data and the time required to generate the range expansion image data is longer than a threshold value. Do
This is a display device.

The fourth aspect of the present invention is:
Based on input image data, base image data having at least one of a dynamic range and a color gamut narrower than the input image data, and difference data used in an enlargement process for enlarging at least one of the dynamic range and the color gamut of the image data And a generation step for performing a generation process for generating
An addition step of performing an addition process of adding correspondence information representing a correspondence relationship between the base image data and the difference data to at least one of the base image data and the difference data;
An output step of individually outputting the base image data and the luminance difference data after the additional processing is performed;
I have a,
In the adding step, display disable information indicating that display on the display device cannot be performed is added to the difference data generated in the generation process .

According to a fifth aspect of the present invention,
An acquisition step of acquiring input base image data that is base image data, and input difference data that is difference data used in an enlargement process for enlarging at least one of a dynamic range and a color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. A determination step of determining whether the data corresponds to the base image data;
In the determination step, the input difference data is acquired, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed. By applying to the base image data, to generate a range expansion image data, if the input difference data is not acquired in the acquisition step, a generation step that does not generate the range expansion image data ,
When the range expansion image data is generated in the generation step, an image based on the range expansion image data is displayed . When the range expansion image data is not generated in the generation step, the input base image data is displayed. A display step for displaying an image based on ;
When the range expansion image data is not generated in the generation step and an image based on the input base image data is displayed in the display step, the range
A first notification step for notifying a user that data necessary for processing for generating enlarged image data has not been acquired;
It is the display method characterized by having.
The sixth aspect of the present invention is:
An acquisition step of individually acquiring input base image data that is base image data, and input difference data that is difference data used in an enlargement process for enlarging at least one of the dynamic range and color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. A determination step of determining whether the data corresponds to the base image data;
The input difference data is acquired in the acquisition step, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed as the input A range expansion image data is generated by applying to the base image data, and if the input difference data is not acquired by the acquisition unit, a generation step of not generating the range expansion image data;
When the range enlarged image data is generated in the generating step, an image based on the range enlarged image data is displayed. When the range enlarged image data is not generated by the generating unit, the input base image data is displayed. A display step for displaying an image based on;
In the generation step, when the range enlarged image data is generated and the time required for the processing for generating the range enlarged image data is longer than a threshold, the display step displays an image based on the input base image data Do
This is a display method characterized by this.

A seventh aspect of the present invention is a program that causes a computer to execute each step of the above-described image processing method.

An eighth aspect of the present invention is a program that causes a computer to execute each step of the display method described above.

  According to the present invention, stream format HDR image data can be transmitted so that an appropriate image display can be performed by either an HDR compatible device or an HDR non-compatible device. Further, according to the present invention, HDR image data can be accurately restored using base image data and difference data.

The block diagram which shows an example of the display system which concerns on this embodiment The figure which shows an example of the data map of the SDI data which concerns on this embodiment The flowchart which shows an example of the processing flow of the display apparatus which concerns on this embodiment.

  Hereinafter, an image processing device, an image processing method, a display device, and a display method according to embodiments of the present invention will be described.

<System configuration>
FIG. 1 is a block diagram illustrating an example of a display system according to the present embodiment.
As shown in FIG. 1, the display system according to the present embodiment includes an image processing device 100 and a display device 120. The image processing apparatus 100 and the display apparatus 120 are connected to be communicable with each other by wire or wireless. In the example of FIG. 1, the image processing apparatus 100 and the display apparatus 120 are connected to each other using an SDI cable 140 that is an SDI (Serial Digital Interface) standard cable. Further, the image processing apparatus 100 and the display apparatus 120 are connected to each other using an SDI cable 150.
In the present embodiment, it is assumed that the resolution of the display device 120 is 1920 pixels in the horizontal direction × 1080 pixels in the vertical direction (hereinafter, Full-HD). In the present embodiment, the resolution of the image data output from the image processing apparatus 100 is also assumed to be Full-HD.

In the example of FIG. 1, the image processing apparatus 100 and the display apparatus 120 are connected to each other using two cables, but the image processing apparatus 100 and the display apparatus 120 are connected to each other using one cable. It may be.
In the example of FIG. 1, SDI cables (SDI cables 140 and 150) are used as cables used to transmit data, but a cable different from the SDI cable may be used.
Note that the resolution of the display device 120 may be lower or higher than Full-HD. Further, the resolution of the image data output from the image processing apparatus 100 may be lower or higher than Full-HD.
In this embodiment, an example in which the external device of the image processing apparatus 100 is the display device 120 will be described, but the external device of the image processing device 100 is not limited to the display device. For example, the external device may be a storage device that stores data output from the image processing apparatus 100.

The image processing apparatus 100 includes a dynamic range reduction unit 101, a difference generation unit 102, a base JPEG encoder 103, a difference JPEG encoder 104, a JPEG-HDR generation unit 105, a base flag addition unit 106, a difference flag addition unit 107, and a content identification flag addition. Section 108, SDI encoders 109 and 110, SDI output terminals 111 and 112, and the like.
The image processing apparatus 100 may not include the base JPEG encoder 103, the difference JPEG encoder 104, and the JPEG-HDR generation unit 105.

The display device 120 includes SDI input terminals 121 and 122, SDI decoders 123 and 124, a data format determination unit 125, a content identification flag analysis unit 126, an enlargement processing unit 127, a BL control unit 128, an image processing unit 129, a display unit 130, A CPU 131, a memory 132, and the like are included. Each functional unit other than the SDI input terminals 121 and 122 is connected to the CPU 131 using a data bus (not shown).
In this embodiment, an example in which the display device 120 is a transmissive liquid crystal display device and the display unit 130 includes a liquid crystal panel and a backlight will be described. However, the display device 120 is not limited to a transmissive liquid crystal display device. . For example, the display device 120 may be a reflective liquid crystal display device, a plasma display device, or an organic EL display device. The display device 120 is a MEMS (Micro Electro Mechanical) instead of a liquid crystal element.
(System) A MEMS shutter type display using a shutter may be used.

<Image processing device>
Each functional unit of the image processing apparatus 100 will be described according to a data flow.
The image processing apparatus 100 receives HDR image data in which each of the Y value, Cb value, and Cr value is a gradation value equivalent to 32 bits as input image data. Then, the image processing apparatus 100 generates a JPEG-HDR file from the HDR image data or outputs data corresponding to the HDR image data. The HDR image data is, for example, image data generated by using a PC or an imaging device from bracket image data continuously shot while changing the exposure time. A JPEG-HDR file is a data file in which base image data and difference data are stored. The base image data is image data that has a narrower dynamic range and / or color gamut than input image data.

  The difference data is data used in an enlargement process for enlarging at least one of the dynamic range and the color gamut of the image data. The difference data includes, for example, at least one of luminance difference data and color difference data.

  The luminance difference data is data used in the luminance range expansion process for expanding the dynamic range of the image data, and is data representing the difference between the luminance values of the HDR image data and the base image data. For example, the luminance difference data is a ratio between the luminance value (gradation value) of the base image data and the luminance value (gradation value) of the HDR image data in pixel units (or in units of regions including a predetermined number of pixels). This is luminance ratio data representing the luminance ratio. That is, the luminance difference data is the ratio of the luminance value (gradation value) of the HDR image data to the luminance value (gradation value) of the base image data or the reciprocal thereof in pixel units (or an area unit composed of a predetermined number of pixels). Is luminance ratio data representing However, the luminance difference data is obtained by subtracting one from the luminance value (gradation value) of the base image data and the luminance value (gradation value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). Data representing a luminance difference value which is a subtracted difference value may be used. Further, the luminance difference data may be luminance conversion table data (for example, a reverse tone map described later) representing a correspondence relationship between the input luminance value and the output luminance value in the enlargement process. Note that the luminance range expansion processing can also be said to be processing for reproducing luminance that cannot be expressed in the base image data.

The color difference data is data used in the color gamut expansion process for expanding the color gamut of the image data, and is data representing the color difference between the HDR image data and the base image data. This color difference data is also referred to as data used in the enlargement process for expanding the dynamic range of the color component of the image data. For example, the color difference data is a color difference value (Cb value, Cr value) of the base image data and a color difference value (Cb value, Cr value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). This is data representing a color difference value which is a difference value obtained by subtracting the other from one. However, the color difference data is a color difference value (Cb value, Cr value) of the base image data and a color difference value (Cb value, Cr value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). Color difference ratio data representing a color ratio that is a ratio of Further, the color difference value and the color ratio may be values calculated using R value, G value, and B value instead of the color difference value. Note that the color gamut expansion processing can be said to be processing for reproducing colors that cannot be expressed by the base image data.

Note that the number of bits of the input image data (HDR image data) may be more or less than 32 bits.
The HDR image data may be still image data or moving image data. JPEG-HDR, which is a format in which HDR image data is expressed by base image data and difference data, can be applied to, for example, moving image data in MPEG format or MOTION-JPEG format. The format of moving image data in MPEG format, which is HDR image data in which image data of each frame is expressed by base image data and difference data, may be referred to as MPEG-HDR. The image processing apparatus 100 may perform the process of generating and outputting the data corresponding to the HDR image data only once, or the streaming output for generating and outputting the data corresponding to the HDR image data one after another. You may go.

  A generation process for generating base image data and difference data is performed by the dynamic range reduction unit 101 and the difference generation unit 102 based on input image data which is HDR image data.

  The dynamic range reduction unit 101 generates base image data by performing reduction processing for reducing the dynamic range on the input image data. For example, the tone value of the base image data is calculated from the tone value of the input image data using a predetermined tone map that represents the correspondence between the tone value before reduction and the tone value after reduction. It can be said that the tone map used in the reduction process is a tone map in which the gradation value before reduction is an input value and the gradation value after reduction is an output value. In this embodiment, base image data in which each of the Y value, the Cb value, and the Cr value is a gradation value equivalent to 8 bits is generated.

Note that one tone map may be prepared for the entire area of the image, a tone map may be prepared for each pixel position, or a tone map may be prepared for each area.
A plurality of tone maps may be prepared in advance, and the tone value of the base image data may be calculated using any one of the plurality of tone maps.
Note that the number of bits of the base image data may be more or less than 8 bits.

  The difference generation unit 102 generates difference data based on the base image data and input image data generated by the dynamic range reduction unit 101. In the present embodiment, data representing the luminance ratio, which is the ratio between the luminance value (Y value) of the base image data and the luminance value of the input image data, is generated as luminance difference data. Data representing a color difference value representing a difference between the color of the base image data and the color of the input image data for each pixel is generated as the color difference data. Specifically, data representing the Cb difference value and the Cr difference value for each pixel is generated as the color difference data. The Cb difference value is a value obtained by subtracting the other from one of the Cb value of the base image data and the Cb value of the input image data. The Cr difference value is a value between the Cr value of the base image data and the Cr value of the input image data. A value obtained by subtracting the other from one.

Note that one of the luminance difference data and the color difference data may not be generated. For example, when the luminance value of the base image data and the luminance value of the input image data are the same, the luminance difference data may not be generated. When the color of the base image data and the color of the input image data are the same, the color difference data may not be generated.
Note that an inverted tone map in which the input value and the output value of the tone map used in the reduction process are exchanged may be generated as the luminance difference data.

The base JPEG encoder 103 encodes the base image data generated by the dynamic range reduction unit 101 into JPEG image data. Note that the dynamic range reduction unit 101 may generate JPEG image data as the base image data.
The difference JPEG encoder 104 encodes the luminance difference data and the color difference data generated by the difference generation unit 102 into JPEG format data, respectively. Note that the difference generation unit 102 may generate JPEG format data as luminance difference data and color difference data.
The JPEG-HDR generation unit 105 combines the JPEG format base image data generated by the base JPEG encoder 103 and the JPEG format luminance difference data and color difference data generated by the difference JPEG encoder 104 into one data file. Store. Thereby, a JPEG-HDR file is generated.

  The base flag adding unit 106, the difference flag adding unit 107, the content identification flag adding unit 108, and the SDI encoders 109 and 110 perform addition processing. In the present embodiment, as addition processing, processing for adding correspondence information to all of base image data, luminance difference data, and color difference data is executed. The correspondence information is information representing the correspondence relationship between the base image data, the luminance difference data, and the color difference data.

In addition processing, it is only necessary to add correspondence information to at least one of base image data, luminance difference data, and color difference data. For example, correspondence information may be added only to the base image data. In that case, the luminance difference data and the color difference data (luminance difference data and color difference data to be used for obtaining accurate HDR image data) corresponding to the base image data are determined by referring to the base image data. can do.
If the luminance difference data is not generated, the adding process may add the correspondence information to at least one of the base image data and the color difference data. In this case, information indicating the correspondence between the base image data and the color difference data may be added as the correspondence information. When the color difference data is not generated, in the addition process, the correspondence information may be added to at least one of the base image data and the luminance difference data. In that case, information indicating the correspondence between the base image data and the luminance difference data may be added as the correspondence information.

The base flag adding unit 106 and the difference flag adding unit 107 add type information indicating the type of data to each data generated by the generation process.
Note that the process of adding type information may be omitted.
In the present embodiment, an example will be described in which type information is added to each of base image data, luminance difference data, and color difference data. However, if luminance difference data is not generated, type information is added to the base image data and color difference data. If color difference data is not generated, type information is added to the base image data and luminance difference data. Is added.

The base flag adding unit 106 adds a type flag (type information) indicating base image data to the base image data generated by the dynamic range reducing unit 101. Then, the base flag adding unit 106 outputs the first additional base image data, which is the base image data to which the type flag is added, to the content identification flag adding unit 108.
The difference flag addition unit 107 adds a type flag (type information) indicating luminance difference data to the luminance difference data generated by the difference generation unit 102, and adds a color flag to the color difference data generated by the difference generation unit 102. A type flag (type information) indicating difference data is added. Then, the difference flag adding unit 107 identifies the first additional luminance difference data that is the luminance difference data to which the type flag is added and the first additional color difference data that is the color difference data to which the type flag is added. The data is output to the flag adding unit 108.

Further, the base flag adding unit 106 and the difference flag adding unit 107 convert each data generated by the generation processing into data that can be processed by the display device 120. Specifically, the base flag adding unit 106 and the difference flag adding unit 107 are the data to which the type information is added at the timing when the data is input to the display device 120 at the processing timing at which the display device 120 processes the data. Is output.
The processing for controlling the transmission timing for transmitting data may be executed by any functional unit.

  The content identification flag adding unit 108 includes a content identification flag indicating that each of the first additional base image data, the first additional luminance difference data, and the first additional color difference data is the same content data. (Correspondence information) is added. In the present embodiment, the same content identification flag as the content identification flag added to the first additional base image data is added to the first additional luminance difference data and the first additional color difference data. Then, the content identification flag adding unit 108 outputs the second additional base image data, which is the base image data to which the content identification flag is added, to the SDI encoder 109. Further, the content identification flag adding unit 108 includes second additional luminance difference data that is luminance difference data to which a content identification flag is added, second additional color difference data that is color difference data to which a content identification flag is added, Is output to the SDI encoder 110.

  In the present embodiment, the content identification flag adding unit 108 adds the HDR identification flag to the first additional base image data when generating the second additional base image data from the first additional base image data. The HDR identification flag represents base image data corresponding to HDR image data. In the example of FIG. 1, when a JPEG-HDR file is generated by the JPEG-HDR generation unit 105, the JPEG-HDR generation unit 105 notifies the content identification flag addition unit 108 that a JPEG-HDR file has been generated. To do. Then, the content identification flag adding unit 108 executes processing for adding an HDR identification flag in response to the notification from the JPEG-HDR generation unit 105.

When the process for adding the type information is omitted, the correspondence information is added to the data generated by the generation process.
Different correspondence information may be added for each data. For example, correspondence information representing luminance difference data and color difference data corresponding to the base image data may be added to the base image data. Correspondence information representing base image data and color difference data corresponding to the luminance difference data may be added to the luminance difference data. Then, correspondence information representing base image data and luminance difference data corresponding to the color difference data may be added to the color difference data. If such correspondence information is used, even if the correspondence information is added to only one of the base image data, the luminance difference data, and the color difference data, the base image data, the luminance difference data, and The correspondence between the color difference data can be determined.

SDI format serial data (SDI data) is generated by the SDI encoders 109 and 110 as the data after the additional processing.
The SDI encoder 109 converts the second additional base image data that is parallel data into SDI base image data that is SDI data, and outputs the SDI base image data to the SDI output terminal 111.
The SDI encoder 110 converts the second additional luminance difference data that is parallel data into SDI luminance difference data that is SDI data, and converts the second additional color difference data that is parallel data into SDI color difference data that is SDI data. . And the SDI encoder 110
Outputs SDI brightness difference data and SDI color difference data to the SDI output terminal 112.
Note that the data after the addition processing is not limited to SDI data.

The SDI output terminals 111 and 112 individually output the data after the addition processing is performed to the outside of the image processing apparatus 100.
The SDI output terminal 111 outputs the SDI base image data to the display device 120 which is an external device using the SDI cable 140.
The SDI output terminal 112 outputs SDI luminance difference data and SDI color difference data to the display device 120 using the SDI cable 150.

As described above, in this embodiment, each data after the additional processing is performed is individually output to the external device using the SDI cable. Further, in the present embodiment, the base image data after the addition processing is performed on the external device using a cable different from the difference data (luminance difference data and color difference data) after the addition processing is performed. Is output.
In addition, each data after the addition process is performed may be output to an external device using the same cable. Also, the cable used to output each data after the additional processing is performed to the external device is not limited to the SDI cable. The data may be output to an external device by wireless communication.
In addition, when outputting each data after an additional process is performed to an external device using one cable, the HDR identification flag mentioned above can also be used as correspondence information. For example, if the base image data D1 to which the HDR identification flag is added and the difference data D2 are output in that order, the difference data D2 corresponds to the data corresponding to the base image data D1 depending on the presence or absence of the HDR identification flag. Can be determined.

<Additional processing>
The additional processing according to the present embodiment will be described in detail with reference to FIG.
FIG. 2 shows an example of the data structure (data map) of SDI data. FIG. 2 shows a Full-HD data map.

The data map 200 is a data map of SDI base image data for one line.
The data map 201 is a data map of SDI data including SDI luminance difference data and SDI color difference data for one line.
The data map 202 is a data map of SDI data for one frame.
The vertical axis of the data maps 200 and 201 indicates the number of bits, and the horizontal axis indicates the sample number. In this embodiment, data is transmitted in order from the data with the smallest sample number.
The vertical axis of the data map 202 indicates the line number, and the horizontal axis indicates the sample number. In the present embodiment, data is transmitted in order from the data with the smallest line number.

  In FIG. 2, an area indicated by a horizontal line is an ancillary portion capable of storing auxiliary data (horizontal auxiliary data). The area indicated by the diagonal lines is also an ancillary portion that can store auxiliary data (vertical auxiliary data). In the present embodiment, the type information and the correspondence information are stored in the ancillary part by the addition process.

EAV (End of Active Video) stores an identifier representing the end point of the horizontal / vertical effective image area in TRS (Timing Reference Signal) of SDI data. In this embodiment, the EVF stores a synchronization identification code consisting of 4 words [3FF] [000] [000] [XYZ] as an identifier. EAV, SAV, field, and horizontal / vertical can be identified from the contents of [XYZ].
LN (Line Number) stores data for monitoring the line number of SDI data.
CRCC (Cyclic Redundancy Check Code) stores data for valid image area, EAV, and data for checking LN transmission errors.
SAV (Start of Active Video) stores an identifier representing a horizontal / vertical effective image area start point in TRS of SDI data. In this embodiment, the SAV stores a synchronization identification code consisting of 4 words [3FF] [000] [000] [XYZ] as an identifier. EAV, SAV, field, and horizontal / vertical can be identified from the contents of [XYZ].

In the example of FIG. 2, the data map 200 has a data width of 20 bits. In the example of FIG. 2, the pixel values (Y value, Cb value, and Cr value) of the base image data are mapped in a format that conforms to SMPTE425M. Pixel values are mapped to the effective image area.
In the data map 200, “Y” stores a Y value, “Cb” stores a Cb value, and “Cr” stores a Cr value. In addition, an alpha value can be stored in “A”. The numerical values indicated next to “Y”, “Cb”, “Cr”, and “A” are sample numbers. Since no alpha value is used in the present embodiment, no data is stored in “A”.
Since the pixel values of the base image data are mapped in a format compliant with SMPTE425M, the base image data can be displayed even on a display device that does not support HDR image data.

In the example of FIG. 2, the data map 201 also has a 20-bit data width like the data map 200. In the example of FIG. 2, the value of luminance difference data (luminance ratio) and the value of color difference data (color difference value; combination of Cb difference value and Cr difference value) are mapped to the effective image area.
“YR” shown in the data map 201 stores a luminance ratio, “CbR” stores a Cb difference value, and “CrR” stores a Cr difference value. “N / A” is a non-use area. The numerical values described next to “YR”, “CbR”, and “CrR” are sample numbers.

In the data map 202, an area filled with black is a payload ID in which format information or the like is stored. In the present embodiment, in the additional processing, type information and correspondence information are stored in the ancillary part of the line next to the line where the payload ID exists. Specifically, type information and correspondence information are stored at a position immediately below the payload ID in the data map. In the example of FIG. 2, the payload ID exists at the position (sample number, line number) = (1928, 10), and the type information and the correspondence information are stored at the position (1928, 11).
The storage position of the type information and the correspondence information is not limited to the above position. For example, type information and correspondence information are stored in an ancillary part (area indicated by diagonal lines) capable of storing vertical auxiliary data, not an ancillary part (area indicated by horizontal lines) capable of storing horizontal auxiliary data. Also good.

In the present embodiment, in the addition process, display impossibility information indicating that display on the display device cannot be performed is added to the difference data generated in the generation process. For example, the display impossibility information is added by the difference flag adding unit 107 or the content identification flag adding unit 108.
In this embodiment, display disable information is added to the payload ID of SDI data including SDI luminance difference data and SDI color difference data. Specifically, a code (word) that cannot be decoded by the display device is stored as display-impossible information in a data identification word (DID) constituting the payload ID. Thereby, it is possible to prevent the SDI data including the SDI brightness difference data and the SDI color difference data from being displayed erroneously on the display device.

  The code that cannot be decoded by the display device is, for example, a code that is not defined in the standard. In the case of level A, 241h is stored in DID. In this case, for example, by rewriting 241h to 3FFh or 000h, the display device can determine that the format of the SDI data is an incompatible format. As a result, it is possible to suppress erroneous display of SDI data including SDI luminance difference data and SDI color difference data.

FIG. 2 shows an example in which SDI data including SDI brightness difference data and SDI color difference data is generated, but the SDI brightness difference data and SDI color difference data may be transmitted separately. Good. For example, the SDI luminance difference data may be transmitted using a cable different from the SDI color difference data. In that case, display disable information may be added individually to the SDI luminance difference data and the SDI color difference data.
It should be noted that the process of adding display impossible information may be omitted. Even if the process of adding the display impossibility information is omitted, if the type information is added, it is possible to prevent the luminance difference data and the color difference data from being erroneously displayed.
The display impossible information is not limited to a code that cannot be decoded by the display device. The non-displayable information may be any information as long as it is information indicating that display on the display device cannot be performed.

<Display device>
Each functional unit of the display device 120 will be described with reference to the flowchart of FIG. FIG. 3 is a flowchart illustrating an example of the operation flow (processing flow) of the display device 120. In the display device 120, the CPU 131 executes a program stored in the memory 132 (ROM). Thereby, each processing is executed in the processing flow shown in FIG.

First, at least one of input base image data that is base image data and input difference data that is difference data is acquired by the SDI input terminals 121 and 122 and the SDI decoders 123 and 124 (S301). As input difference data, for example, at least one of input luminance difference data that is luminance difference data and input color difference data that is color difference data is acquired.
In this embodiment, input base image data and input difference data are acquired separately.

  The SDI input terminal 121 acquires SDI base image data from the image processing apparatus 100 using the SDI cable 140. The SDI input terminal 121 outputs the acquired data to the SDI decoder 123.

  The SDI input terminal 122 acquires SDI brightness difference data and SDI color difference data from the image processing apparatus 100 using the SDI cable 150. Then, the SDI input terminal 122 outputs the acquired data to the SDI decoder 124.

  The SDI decoder 123 converts the SDI base image data that is serial data acquired at the SDI input terminal 121 into input base image data that is parallel data. Then, the SDI decoder 123 outputs the input base image data to the data format determination unit 125.

  The SDI decoder 124 converts the SDI luminance difference data and SDI color difference data, which are serial data acquired at the SDI input terminal 122, into input luminance difference data and input color difference data, which are parallel data. Then, the SDI decoder 123 outputs the input luminance difference data and the input color difference data to the data format determination unit 125.

  After the SDI data is input to at least one of the SDI input terminal 121 and the SDI input terminal 122 and the input SDI data is decoded into parallel data, the process proceeds to S302.

In the present embodiment, an example in which the display device 120 acquires data from the image processing device 100 will be described, but the external device of the display device 120 is not limited to the image processing device 100. For example, the display device 120 may acquire data from a storage device that stores base image data, luminance difference data, color difference data, and the like.
In this embodiment, an example in which data is acquired using an SDI cable will be described. However, a cable used for acquiring data is not limited to an SDI cable. Data may be acquired by wireless communication.
Each data may be acquired from an external device using the same cable. Further, the SDI brightness difference data and the SDI color difference data may be acquired by different input terminals.

In S <b> 302, the data format determination unit 125 determines the type of data input to the data format determination unit 125. In this embodiment, it is determined whether or not type information is added to the data. Then, the type of data to which the type information is added is determined based on the type information. Specifically, it is determined whether the data to which the type information is added is input base image data, input luminance difference data, or input color difference data. When a plurality of data is input to the data format determination unit 125, determination is performed for each data.
Note that data to which type information is not added may be excluded from processing targets as unknown data, or not. For example, data to which type information is not added may be determined as image data (input base image data).
In the present embodiment, it is assumed that conventional SDI format image data (image data for which there is no corresponding luminance difference data) is input base image data.

Next, the CPU 131 determines whether or not input base image data has been acquired based on the determination result of S302 (S303).
When the input base image data is acquired (S303: YES), the process proceeds to S304. When the input base image data is not acquired (S303: NO), the process proceeds to S311.

In S304, the CPU 131 determines whether both input luminance difference data and input color difference data have been acquired based on the determination result in S302.
When both the input luminance difference data and the input color difference data are acquired (S304: YES), the process proceeds to S305, and at least one of the input luminance difference data and the input color difference data is not acquired. (S303: NO), the process proceeds to S310.
As described above, only luminance difference data may be generated as data corresponding to the base image data, and color difference data may not be generated. In that case, there is no color difference data corresponding to the input base image data. Further, only the color difference data may be generated as the data corresponding to the base image data, and the luminance difference data may not be generated. In that case, there is no luminance difference data corresponding to the input base image data. Therefore, in S304, it may be determined whether or not at least one of the input luminance difference data and the input color difference data has been acquired. If at least one of the input luminance difference data and the input color difference data is acquired, the process proceeds to S305. If both the input luminance difference data and the input color difference data are not acquired, the process proceeds to S310. May be.

In S305, the content identification flag analysis unit 126 determines whether both the input luminance difference data and the input color difference data are data corresponding to the input base image data. In this embodiment, it is determined whether or not a content identification flag (corresponding information) is added to the input base image data, the input luminance difference data, and the input color difference data. Then, when correspondence information is added to the input base image data, the input luminance difference data, and the input color difference data, both the input luminance difference data and the input color difference data are input based on the correspondence information. It is determined whether the data corresponds to the base image data. Specifically, when the same content identification flag is added to the input base image data, the input luminance difference data, and the input color difference data, both the input luminance difference data and the input color difference data are It is determined that the data corresponds to the data. When a content identification flag different from the input base image data is added to the input luminance difference data, it is determined that the input luminance difference data is not data corresponding to the input base image data. If a content identification flag different from the input base image data is added to the input color difference data, it is determined that the input color difference data is not data corresponding to the input base image data.
If it is determined that both the input luminance difference data and the input color difference data are data corresponding to the input base image data (S305: YES), the process proceeds to S306. If it is determined that at least one of the input luminance difference data and the input color difference data is not data corresponding to the input base image data (S305: NO), the process proceeds to S310.

As described above, if a content identification flag is added to at least one of the input base image data, the input luminance difference data, and the input color difference data, the determination process in S305 can be performed.
If the content identification flag is not added to at least one of the input base image data, the input luminance difference data, and the input color difference data, the determination process of S305 cannot be executed. The process proceeds to S310. However, it is preferable that the process proceeds to S310 in order to prevent erroneous luminance range expansion processing and color gamut expansion processing from being performed on the input base image data. “Incorrect luminance gamut expansion processing” is luminance luminance expansion processing using luminance difference data that does not correspond to input base image data. “Incorrect color gamut expansion processing” corresponds to input base image data. This is a color gamut expansion process using no color difference data.
As described above, one of the luminance difference data and the color difference data corresponding to the input base image data may not exist. Therefore, when it is determined that at least one of the input luminance difference data and the input color difference data is data corresponding to the input base image data, the process may proceed to S306. Then, when it is determined that both the input luminance difference data and the input color difference data are not data corresponding to the input base image data, the process may be advanced to S310. If the input color difference data corresponding to the input base image data is not acquired even though the luminance difference data and the color difference data corresponding to the input base image data are generated, the process proceeds to S310 from the viewpoint of image quality. It is preferable that the process proceeds. Even if the luminance difference data and the color difference data corresponding to the input base image data are generated, but the input luminance difference data corresponding to the input base image data is not acquired, the process proceeds to S310 from the viewpoint of image quality. It is preferable that the process proceeds.

In step S306, the enlargement processing unit 127 performs range gamut expansion processing using the input luminance difference data and color gamut expansion processing using the input color difference data on the input base image data, thereby generating range expansion image data. . The range expansion image data is, for example, HDR image data in which each of the Y value, the Cb value, and the Cr value is a gradation value equivalent to 32 bits.
When the number of bits of the image data (processed image data) after the luminance gamut expansion processing and the color gamut expansion processing is larger than the number of bits of the gradation value that can be displayed on the display device 120, the display device The number of bits of the processed image data is reduced to the number of bits that can be displayed. Thereby, range-enlarged image data is generated. That is, the range-enhanced image data is generated by reducing the dynamic range of the processed image data to the dynamic range of the image data that can be displayed on the display device. The number of bits of the gradation value that can be displayed by the display device 120 is, for example, the number of bits of the gradation value that can be processed by the image processing unit 129.

Note that the number of bits of the range-enlarged image data may be a value determined in advance by the manufacturer, or may be a value that can be set or changed by the user. The number of bits of the range-enlarged image data depends on the usage environment of the display device, the type of base image data (illustration, photograph, video, still image, etc.), application (for ornamental use, diagnostic use, etc.) May be determined automatically.
Note that the number of bits of HDR image data may be more or less than 32 bits.
If the input luminance difference data corresponding to the input base image data does not exist or has not been acquired, the luminance range expansion process is not performed in S306. If the input color difference data corresponding to the input base image data does not exist or is not acquired, the color gamut expansion process is not performed in S306.

Next, the CPU 131 determines whether or not the time required for the processing for generating the range expansion image data (for example, the luminance region expansion processing and the color gamut expansion processing) is equal to or less than the threshold (S307).
When the time (processing time) required for the processing for generating the range enlarged image data is equal to or smaller than the threshold (S307: YES), the processing proceeds to S308, and when the processing time is longer than the threshold, the processing proceeds to S310. It is advanced.
The threshold value to be compared with the processing time may be a value determined in advance by the manufacturer, or may be a value that can be set or changed by the user. The threshold value may be automatically determined by the display device according to the use environment of the display device, the type of base image data, the application, and the like.

  In S308, the image processing unit 129 and the display unit 130 display an image based on the range expansion image data. In the present embodiment, the image processing unit 129 generates display image data by performing predetermined image processing on the range-enlarged image data. Then, the image processing unit 129 outputs the display image data to the display unit 130. Thereby, the display image data is displayed on the screen. Specifically, the display image data is displayed on the screen by controlling the transmittance of the liquid crystal panel based on the display image data. The predetermined image processing includes, for example, gamma conversion processing, color conversion processing different from color conversion processing using color difference data, and the like.

Note that range-enhanced image data may be used as display image data without performing predetermined image processing.
Note that not only the dynamic range and color gamut of the base image data, but also the light emission of the backlight may be controlled based on the difference data. Specifically, input difference data may be input to the BL control unit 128. Then, the BL control unit 128 may control the light emission of the backlight based on the input difference data. As a result, the brightness that could not be displayed simply by controlling the transmittance of the liquid crystal panel can be assigned to the light emission brightness of the backlight. Can be assigned. As a result, it is possible to obtain a display image (image displayed on the screen) having a wider dynamic range and color gamut. For example, when the number of bits of the gradation value that can be displayed on the display device is 10 bits, a display image having a wider dynamic range than the dynamic range corresponding to 10 bits can be obtained.

Next, the CPU 131 determines whether or not at least one of the input luminance difference data and the input color difference data is not acquired during the display of the image based on the range expansion image data (S309). In the present embodiment, the CPU 131 monitors the SDI input terminal 122 used for acquiring the input luminance difference data and the input color difference data, and the SDI cable 150 is connected to the display device based on the connection state of the SDI input terminal 122. It is determined whether or not it has been removed from 120. When the SDI cable 150 is connected to the display device 120, it is determined that both input luminance difference data and input color difference data have been acquired. When the SDI cable 150 is removed from the display device 120, it is determined that both the input luminance difference data and the input color difference data are no longer acquired.
If both the input luminance difference data and the input color difference data have been acquired (S309: NO), this flow ends. If at least one of the input luminance difference data and the input color difference data is not acquired (S309: YES), the range enlarged image data cannot be generated, and the process proceeds to S310.

  As described above, one of the luminance difference data and the color difference data corresponding to the input base image data may not exist. Therefore, in S309, it may be determined whether or not both input luminance difference data and input color difference data are acquired. Then, when at least one of the input luminance difference data and the input color difference data has been acquired, this flow is ended, and when both the input luminance difference data and the input color difference data are not acquired, the process proceeds to S310. May be. When the luminance difference data and the color difference data are generated as data corresponding to the input base image data, from the viewpoint of image quality, even when one of the input luminance difference data and the input color difference data is not acquired, S310 is performed. It is preferable that the process proceeds.

In S310, the image processing unit 129 and the display unit 130 display an image based on the input base image data. The image display method is the same as S308.
In the present embodiment, images based on the input base image data are displayed in the following cases 1 to 3. In cases 1 and 2, an image based on the input base image data is displayed without performing the process of generating the range enlarged image data. In case 3, processing for generating range-enlarged image data is performed, but an image based on the input base image data is displayed.

Case 1: When at least one of the input luminance difference data and the input color difference data is not acquired (S304: NO and S309: YES)
Case 2: When it is determined that at least one of the input luminance difference data and the input color difference data is not data corresponding to the input base image data (S305: NO)
Case 3: When the time required for the processing for generating the range-enlarged image data is longer than the threshold (S307: NO)

By performing the process of S310, an image can be displayed even when an image based on the range expansion image data cannot be displayed.

The display device performs the first notification for notifying the user when the processing for generating the range-enlarged image data is not performed and the image based on the input base image data is displayed (for example, cases 1 and 2). May have a part. In the first notification unit, for example, the user is notified that data necessary for the process of generating the range expansion image data has not been acquired. The first notification unit may notify that an image based on the input base image data is displayed, that processing for generating range-enlarged image data cannot be performed, and the like.
The display device includes a second notification unit that performs notification to the user when processing for generating range-enlarged image data is performed and an image based on the input base image data is displayed (for example, in case 3). You may have. In the second notification unit, for example, the user is notified that the process of generating the range enlarged image data is being executed. The second notification unit may notify that an image based on the input base image data is displayed, that the total processing load on the display device is high, and the like.
By the notification by the first notification unit or the second notification unit, the user can grasp the cause of the display of the base image data. The notification to the user can be performed using, for example, voice or light. Specifically, the user can be notified by displaying an image (message, icon, etc.) corresponding to the notification content on the screen. In addition, the user is notified by sounding according to the notification content or by turning on a light source provided outside the screen with a blinking pattern (including continuous lighting) or color according to the notification content. Can do.
The processing of the first notification unit and the second notification unit may be realized by the same functional unit, and the processing of the first notification unit is realized by a functional unit different from the functional unit that realizes the processing of the second notification unit. Also good. The processing of the first notification unit and the second notification unit is performed by the CPU 131, for example.

  In case 3, the display may be switched from the image based on the input base image data to the image based on the range expanded image data after the range expanded image data is generated. As a result, image display can be performed in a short time, and image display with a wide dynamic range can be performed after the range expansion image data is generated.

  In S311, the CPU 131 notifies the user that the format of the data acquired in S301 is an incompatible format. In other words, the user is notified that the data acquired in S301 is data that cannot be displayed (third notification).

As described above, according to the present embodiment, image data (for example, HDR image data) is divided into base image data and difference data and transmitted. Thereby, image data with a wide dynamic range such as HDR image data can be transmitted in a small transmission band.
Further, according to the present embodiment, the correspondence information is added to at least one of the base image data and the difference data. Accordingly, it is possible to suppress erroneous enlargement processing on the base image data. As a result, it is possible to accurately restore the HDR image data using the base image data and the difference data. Specifically, according to the display device according to the present embodiment, the HDR image data is restored only when the correspondence relationship of the data is determined from the correspondence information and the input difference data is data corresponding to the input base image data. Is done. Thereby, according to the display apparatus which concerns on this embodiment, HDR image data can be decompress | restored correctly.
Further, according to the present embodiment, the base image data and the difference data are transmitted separately. Accordingly, it is possible to transmit the HDR image data in a stream format so that an appropriate image display can be performed by either an HDR compatible device or an HDR non-compatible device. Specifically, an HDR compatible device (a display device that supports HDR image data) can restore accurate HDR image data from base image data and difference data, and display the restored HDR image data. Base image data can be displayed on a non-HDR compatible device (a display device that does not support HDR image data). Specifically, since the base image data is transmitted in the same format as the conventional SDI format, the base image data can be acquired and displayed even by a non-HDR compatible device.

  Note that at least one of the processes of S303, S304, S307, S309, S310, and S311 may be omitted. Even if these processes are omitted, the above-described effects can be obtained.

<Other examples>
The present invention can also be implemented by a system (or a device such as a CPU or MPU) of a system or apparatus that implements the functions of the above-described embodiments by reading and executing a program recorded in a storage device. The present invention can also be implemented by a method comprising steps executed by a computer of a system or apparatus that implements the functions of the above-described embodiments by reading and executing a program recorded in a storage device, for example. . For this purpose, the program is stored in the computer from, for example, various types of recording media that can serve as the storage device (ie, computer-readable recording media that holds data non-temporarily). Provided to. Therefore, the computer (including devices such as CPU and MPU), the method, the program (including program code and program product), and the computer-readable recording medium that holds the program non-temporarily are all present. It is included in the category of the invention.

DESCRIPTION OF SYMBOLS 100 Image processing apparatus 101 Dynamic range reduction part 102 Difference production | generation part 108 Content identification flag addition part 111,112 SDI output terminal 120 Display apparatus 121,122 SDI input terminal 126 Content identification flag analysis part 127 Enlargement process part 130 Display part

Claims (28)

Based on input image data, base image data having at least one of a dynamic range and a color gamut narrower than the input image data, and difference data used in an enlargement process for enlarging at least one of the dynamic range and the color gamut of the image data And generating means for performing a generating process for generating
An adding means for performing an adding process of adding correspondence information representing a correspondence relationship between the base image data and the difference data to at least one of the base image data and the difference data;
Output means for individually outputting the base image data and the difference data after the additional processing is performed;
I have a,
The additional processing is the differential data generated by the generation process, the image processing apparatus according to claim Rukoto not available information is added indicating that it can not be displayed on the display device.   The image processing apparatus according to claim 1, wherein in the addition process, correspondence information is added to both the base image data and the difference data.   The image processing apparatus according to claim 2, wherein in the addition processing, the same correspondence information as the correspondence information added to the base image data is added to the difference data. The difference data includes luminance difference data used in luminance gamut expansion processing for expanding the dynamic range of image data, and color difference data used in color gamut expansion processing for expanding the color gamut of image data,
In the addition processing, correspondence information representing a correspondence relationship between the base image data, the luminance difference data, and the color difference data in at least one of the base image data, the luminance difference data, and the color difference data. Is added,
The image processing according to any one of claims 1 to 3, wherein the output means outputs base image data, luminance difference data, and color difference data after the addition processing is performed. apparatus. The image processing apparatus according to claim 4, wherein in the addition process, correspondence information is added to all of the base image data, the luminance difference data, and the color difference data.   6. The image processing apparatus according to claim 5, wherein in the addition process, the same correspondence information as the correspondence information added to the base image data is added to the luminance difference data and the color difference data.   The output means outputs the base image data after the addition processing is performed to an external device using a cable different from the difference data after the addition processing is performed. The image processing apparatus according to any one of 1 to 6.   The said output means outputs the base image data and difference data after the said addition process to an external device using an SDI cable, The one of Claims 1-7 characterized by the above-mentioned. Image processing device. In the addition process, SDI data is generated as data after the addition process is performed,
The image processing apparatus according to claim 1, wherein the correspondence information is stored in an ancillary portion of SDI data.   The image according to any one of claims 1 to 9, wherein in the addition process, type information indicating a type of data is added to the base image data and the difference data generated in the generation process. Processing equipment. Before Symbol not available information, the image processing apparatus according to claim 9, characterized in that stored in the payload ID of the SDI data. The display failure information, the image processing apparatus according to any one of claims 1 to 11, wherein the display device is a code that can not be deciphered. Acquisition means for individually acquiring input base image data that is base image data, and input difference data that is difference data used in enlargement processing for enlarging at least one of the dynamic range and color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. Determining means for determining whether the data corresponds to the base image data;
The input difference data is acquired by the acquisition means, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed as the input A range expansion image data is generated by applying to the base image data, and if the input difference data is not acquired by the acquisition unit, a generation unit that does not generate the range expansion image data ;
When the range expansion image data is generated by the generation unit, an image based on the range expansion image data is displayed. When the range expansion image data is not generated by the generation unit, the input base image data is displayed. Display means for displaying an image based on;
Data required for the processing for generating the range expansion image data is acquired when the range expansion image data is not generated by the generation unit and an image based on the input base image data is displayed by the display unit. First notification means for notifying the user that it has not been performed;
A display device comprising: The input difference data includes input luminance difference data which is luminance difference data used in luminance range expansion processing for expanding the dynamic range of image data, and color used in color gamut expansion processing for expanding the color gamut of image data. Input color difference data that is difference data, and
The determination means includes correspondence information representing a correspondence relationship between base image data, luminance difference data, and color difference data in at least one of the input base image data, the input luminance difference data, and the input color difference data. Is added, based on the correspondence information, it is determined whether both the input luminance difference data and the input color difference data are data corresponding to the input base image data,
The generation means, when it is determined that both the input luminance difference data and the input color difference data are data corresponding to the input base image data, a luminance range expansion process using the input luminance difference data; The display device according to claim 13 , wherein the range expansion image data is generated by performing color gamut expansion processing using the input color difference data on the input base image data. When the input difference data is acquired by the acquisition means, and it is determined that the acquired input difference data is not data corresponding to the input base image data,
The generating means does not perform the process of generating the range enlarged image data,
The display device according to claim 13 or 14 , wherein the display unit displays an image based on the input base image data. When at least one of the input luminance difference data and the input color difference data is not acquired by the acquisition unit,
The generating means does not perform the process of generating the range enlarged image data,
The display device according to claim 14 , wherein the display unit displays an image based on the input base image data. When it is determined that at least one of the input luminance difference data and the input color difference data is not data corresponding to the input base image data,
The generating means does not perform the process of generating the range enlarged image data,
The display device according to claim 14 or 16 , wherein the display unit displays an image based on the input base image data. Acquisition means for individually acquiring input base image data that is base image data, and input difference data that is difference data used in enlargement processing for enlarging at least one of the dynamic range and color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. Determining means for determining whether the data corresponds to the base image data;
The input difference data is acquired by the acquisition means, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed as the input A range expansion image data is generated by applying to the base image data, and if the input difference data is not acquired by the acquisition unit, a generation unit that does not generate the range expansion image data;
When the range expansion image data is generated by the generation unit, an image based on the range expansion image data is displayed. When the range expansion image data is not generated by the generation unit, the input base image data is displayed. Display means for displaying an image based on;
The display means displays an image based on the input base image data when the generation means generates the range expansion image data and the time required to generate the range expansion image data is longer than a threshold value. Viewing device characterized by. When the range-enlarged image data is generated by the generation unit and the time required for the process of generating the range-enlarged image data is longer than a threshold, the display unit is configured to generate the range-enlarged image data after the range-enlarged image data is generated. The display device according to claim 18 , wherein the display is switched from an image based on the input base image data to an image based on the range enlarged image data. The range enlarged image data is generated by the generating unit, and when the image based on the input base image data is displayed by the display unit, the process of generating the range expanded image data is being executed. the display device according to claim 18 or 19, further comprising a second notification unit configured to notify the user. 3. The apparatus according to claim 1, further comprising third notification means for notifying a user that the data acquired by the acquisition means cannot be displayed when the input base image data is not acquired by the acquisition means. Item 21. The display device according to any one of Items 13 to 20 . The display device according to any one of claims 13 to 21 , wherein the acquisition unit acquires the input base image data from an external device using a cable different from the input difference data. The display device according to any one of claims 13 to 22 , wherein the acquisition unit acquires data from an external device using an SDI cable. Based on input image data, base image data having at least one of a dynamic range and a color gamut narrower than the input image data, and difference data used in an enlargement process for enlarging at least one of the dynamic range and the color gamut of the image data And a generation step for performing a generation process for generating
An addition step of performing an addition process of adding correspondence information representing a correspondence relationship between the base image data and the difference data to at least one of the base image data and the difference data;
An output step of individually outputting the base image data and the luminance difference data after the additional processing is performed;
I have a,
In the adding step, display disable information indicating that display on a display device cannot be performed is added to the difference data generated in the generation process . An acquisition step of acquiring input base image data that is base image data, and input difference data that is difference data used in an enlargement process for enlarging at least one of a dynamic range and a color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. A determination step of determining whether the data corresponds to the base image data;
In the determination step, the input difference data is acquired, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed. By applying to the base image data, to generate a range expansion image data, if the input difference data is not acquired in the acquisition step, a generation step that does not generate the range expansion image data ,
When the range expansion image data is generated in the generation step, an image based on the range expansion image data is displayed . When the range expansion image data is not generated in the generation step, the input base image data is displayed. A display step for displaying an image based on ;
When the range enlarged image data is not generated in the generation step and an image based on the input base image data is displayed in the display step, data necessary for the processing for generating the range enlarged image data is acquired. A first notification step of notifying the user that it has not been performed;
A display method characterized by comprising: An acquisition step of individually acquiring input base image data that is base image data, and input difference data that is difference data used in an enlargement process for enlarging at least one of the dynamic range and color gamut of the image data;
When correspondence information indicating the correspondence between the base image data and the difference data is added to at least one of the input base image data and the input difference data, the input difference data is input based on the correspondence information. A determination step of determining whether the data corresponds to the base image data;
The input difference data is acquired in the acquisition step, and when it is determined that the acquired input difference data is data corresponding to the input base image data, an enlargement process using the input difference data is performed as the input A range expansion image data is generated by applying to the base image data, and if the input difference data is not acquired in the acquisition step, a generation step of not generating the range expansion image data;
When the range expansion image data is generated in the generation step, an image based on the range expansion image data is displayed. When the range expansion image data is not generated in the generation step, the input base image data is displayed. A display step for displaying an image based on;
In the generation step, when the range enlarged image data is generated and the time required for the processing for generating the range enlarged image data is longer than a threshold, the display step displays an image based on the input base image data Do
A display method characterized by that. A program for causing a computer to execute each step of the image processing method according to claim 25 . A program causing a computer to execute each step of the display method according to claim 26 or 27 .

Images