JP2017208719A - Image processing apparatus - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of obtaining desired image data, as the image data corresponding to HDR image data after being subjected to image processing, by a simple configuration with high accuracy.SOLUTION: An image processing apparatus has acquisition means for acquiring first image data, and difference data between the first image data and second image data having a larger data size than that of the first image data, processing means performing data processing by using the first image data, correction means for correcting the difference data based on the data processing, and generation means for generating third image data from the first image data after the data processing is performed, and the difference data after being corrected.SELECTED DRAWING: Figure 2

Description

  The present invention relates to an image processing apparatus.

  In recent years, opportunities to handle image data having a very large number of bits have increased. Such image data is called “HDR (High Dynamic Range) image data” or the like, and can handle very high luminance. The following methods have been proposed as HDR image data recording / restoring methods. First, the HDR image data is separated and compressed into base image data having a data size smaller than that of the HDR image data and difference data relating to a difference between the HDR image data and the base image data. Then, difference data is associated with the base image data as extended information of the base image data, and a combination of the base image data and the difference data is recorded. Further, the HDR image data is restored by combining the difference data and the base image data.

  An example of HDR image data, base image data, and difference data will be described. For example, each gradation value of the HDR image data is a 32-bit value, and each gradation value of the base image data is an 8-bit value corresponding to the core part of the gradation value (32 bits) of the HDR image data. The difference data is data for restoring the remaining part of the gradation value of the HDR image data (part other than the core part; the ninth and subsequent bits). As the difference data, for example, luminance ratio data indicating the ratio between the luminance of the HDR image data and the luminance of the base image data for each pixel, and the color difference difference indicating the difference between the color difference of the HDR image data and the color difference of the base image data for each pixel Data, etc. are used.

  A data format using a combination of base image data that is JPEG image data and difference data is called “JPEG-HDR” or the like. The JPEG-HDR data can be used as conventional JPEG image data (JPEG image data in which each gradation value is an 8-bit value), or can be used as data for restoring the HDR image data. .

  Conventional techniques relating to a JPEG-HDR data display method are disclosed in, for example, Patent Documents 1 and 2. 8A shows an example of the method disclosed in Patent Document 1, and FIG. 8B shows an example of the method disclosed in Patent Document 2. 8A and 8B show an example in which a display unit corresponding to a 10-bit gradation value is used.

  As shown in FIGS. 8A and 8B, in the methods disclosed in Patent Documents 1 and 2, each gradation value is a 32-bit value from base image data, luminance ratio data, and color difference difference data. Some HDR image data is generated. As shown in FIG. 8A, in the method disclosed in Patent Document 1, HDR image data is subjected to image processing, and each gradation value is converted to a 10-bit value for the image data obtained by the image processing. Tone mapping to be converted is applied. Then, the image data obtained by tone mapping is input to the display unit and displayed. On the other hand, as shown in FIG. 8B, in the method disclosed in Patent Document 2, tone mapping is performed on HDR image data, and image processing is performed on image data obtained by tone mapping. Then, the image data obtained by the image processing is input to the display unit and displayed.

JP 2012-44639 A JP 2011-250260 A

  However, when the above-described conventional technology is used, the following problems occur. In the method disclosed in Patent Document 1 (FIG. 8A), image processing is performed on HDR image data. Therefore, by using the method disclosed in Patent Document 1, the number of gradation value bits handled in image processing increases, and the processing load of image processing, the circuit scale of image processing, and the like increase. In the method disclosed in Patent Document 2 (FIG. 8B), since image processing is performed on image data obtained by tone mapping, conversion characteristics of tone mapping (tone values before conversion and post-conversion gradation values) The result of the image processing changes depending on the change in the correspondence relationship with the gradation value. Therefore, when the method disclosed in Patent Document 2 is used, a desired result (desired image data) may not be obtained as a result of image processing. In the case of using the method disclosed in Patent Document 2, in order to obtain a desired result as the result of image processing, tone mapping must be performed while predicting the result of image processing. However, the process of predicting the result of image processing results in an increase in processing load, an increase in circuit scale, and the like.

  As a JPEG-HDR data display method, the method shown in FIG. 9 is also conceivable. In the method of FIG. 9, image processing is performed on base image data, and image data obtained by image processing and unprocessed difference data (unprocessed luminance ratio data and unprocessed color difference difference data) are used. HDR image data is generated. Then, tone mapping is performed on the HDR image data, and the image data obtained by the tone mapping is input to the display unit and displayed.

  However, the HDR image data obtained from the base image data after the image processing and the unprocessed difference data match the image data obtained by performing the image processing on the HDR image data which is the original image data. Not always. For example, a difference in luminance, a difference in color, or the like may occur between the two HDR image data. Therefore, in the method of FIG. 9, desired image data may not be obtained as HDR image data. The “HDR image data as original image data” is “HDR image data used to obtain base image data before image processing and unprocessed difference data”. “HDR image data as original image data” can also be said to be “HDR image data obtained from base image data before image processing and unprocessed difference data”.

  An object of the present invention is to provide a technique capable of obtaining desired image data with high accuracy and simple configuration as image data corresponding to HDR image data after image processing.

The first aspect of the present invention is:
Acquisition means for acquiring first image data and difference data relating to a difference between the first image data and second image data having a data size larger than the first image data;
Processing means for performing data processing using the first image data;
Correction means for correcting the difference data based on the data processing;
Generating means for generating third image data from the first image data after the data processing and the corrected difference data;
An image processing apparatus comprising:

The second aspect of the present invention is:
Obtaining difference data relating to a difference between the first image data and the first image data and second image data having a data size larger than the first image data;
Performing data processing using the first image data;
Correcting the difference data based on the data processing;
Generating third image data from the first image data after the data processing is performed and the corrected difference data;
An image processing method characterized by comprising:

  A third aspect of the present invention is a program for causing a computer to execute each step of the above-described image processing method.

  According to the present invention, desired image data can be obtained with high accuracy and simple configuration as image data corresponding to HDR image data after image processing.

1 is a block diagram illustrating a configuration example of a display device according to a first embodiment. 1 is a block diagram illustrating a configuration example of an image processing unit according to a first embodiment. 7 is a flowchart illustrating an example of a processing flow of the display device according to the first embodiment. FIG. 9 is a block diagram illustrating a configuration example of an image processing unit according to the second embodiment. 10 is a flowchart illustrating an example of a processing flow of the display device according to the second embodiment. FIG. 9 is a block diagram illustrating a configuration example of an image processing unit according to a third embodiment. 10 is a flowchart illustrating an example of a processing flow of the display device according to the third embodiment. The figure explaining an example of the subject solved by this invention The figure explaining an example of the subject solved by this invention

<Example 1>
Embodiment 1 of the present invention will be described below. In the following, an example in which the image processing apparatus according to the present embodiment is provided in a display device will be described. However, the image processing apparatus may be a separate device from the display device. For example, a personal computer may be used as the image processing apparatus.

  FIG. 1 is a block diagram illustrating a configuration example of the display device 1 according to the present embodiment. In addition, each function part of the display apparatus 1 may be implement | achieved by hardware, and may not be so. For example, the display device 1 may include a processor and a memory that stores a control program. Then, at least a part of the functional units (processing of the functional units) included in the display device 1 may be realized by the processor reading and executing the control program from the memory.

  The receiver 2 acquires (receives) base image data (first image data) and difference data relating to a difference between the base image data and the original image data (second image data). Specifically, the receiver 2 acquires data including base image data and difference data. Then, the receiver 2 outputs the acquired data to the decoding unit 3. The data acquired by the receiver 2 may be moving image data or still image data. Moreover, the transmission method of the data to the receiver 2 is not specifically limited. For example, data is transmitted to the receiver 2 in SDI format, DP format, file format, and the like.

  The original image data is image data having a data size larger than that of the base image data. For example, the original image data is image data obtained by enlarging at least one of the luminance range (dynamic range) and the color gamut of the base image data.

The base image data is, for example, image data obtained by down-sampling original image data (second image data). “Downsampling” can be said to be “gradation compression”. In that case, the number of bits of the base image data (first bit number) is smaller than the number of bits of the original image data (second bit number). That is, the number of bits of the original image data is larger than the number of bits of the base image data. For example, each gradation value of the base image data is an n-bit value corresponding to the core part of the gradation value (N bits) of the original image data. “N” is an integer of 2 or more, and “n” is an integer of 1 or more and less than N.

  Similar to the base image data, the data size of the difference data is smaller than the data size of the original image data. For example, the number of bits of difference data is smaller than the number of bits of original image data. Here, consider a case where each gradation value of the base image data is an n-bit value corresponding to the core part of the gradation value (N bits) of the original image data. In this case, data for restoring the remaining part of the gradation value of the original image data (part other than the core part; n + 1th and subsequent bits) is used as the difference data. The difference data includes, for example, luminance difference data, color difference data, or both.

  The luminance difference data is data indicating a difference between the luminance of the original image data (luminance gradation value; luminance value; Y value) and the luminance of the base image data. For example, the luminance difference data is luminance ratio data indicating a ratio (luminance ratio) between the luminance of the base image data and the luminance of the original image data for each pixel (or for each image region including a predetermined number of pixels). The luminance ratio is the ratio of the luminance of the HDR image data to the luminance of the base image data or the reciprocal thereof. The brightness difference data is a difference value (brightness difference value) obtained by subtracting the other from one of the brightness of the base image data and the brightness of the original image data for each pixel (or for each image area composed of a predetermined number of pixels). May be data indicating. The luminance difference data may be table data indicating a correspondence relationship between the luminance of the base image data and the luminance of the original image data.

  The color difference data is data indicating a difference between the color of the original image data and the color of the base image data. For example, the color difference data includes, for each pixel (or for each image area composed of a predetermined number of pixels), the color difference (color difference gradation value; color difference value; Cb value and Cr value) of the base image data and the color difference of the original image data. Is color difference difference data indicating a difference value (color difference difference value). The color difference difference value is a difference value obtained by subtracting the color difference of the original image data from the color difference of the base image data, or a difference value obtained by subtracting the color difference of the base image data from the color difference of the original image data. . The color difference data is color difference ratio data indicating a color difference ratio that is a ratio of the other of the color difference of the base image data and the color difference of the original image data for each pixel (or for each image area including a predetermined number of pixels). May be. The color difference data may be table data indicating a correspondence relationship between the color difference of the base image data and the color difference of the original image data. The color difference data may be data obtained using chromaticity coordinates or the like instead of the color difference.

  Image data having a very large number of bits is referred to as “HDR (High Dynamic Range) image data”. On the other hand, image data that does not have a large number of bits is called LDR (Low Dynamic Range) image data. When the original image data is HDR image data and the base image data is JPEG image data, the data format using a combination of the base image data and the difference data is “JPEG-HDR” or the like. be called.

In this embodiment, the original image data is HDR image data in which each pixel value is an RGB value and each gradation value (R value, G value, and B value) is a 32-bit value. The base image data is JPEG image data in which each pixel value is an RGB value and each gradation value (R value, G value, and B value) is an 8-bit value. Each gradation value of the base image data corresponds to the core part of the gradation value of the HDR image data. Therefore, in this embodiment, JPEG-HDR data is acquired by the receiver 2. In this embodiment, the difference data is data for restoring the remaining portion of the gradation value of the HDR image data (portion other than the core portion; the ninth and subsequent bits). The difference data includes luminance ratio data and color difference difference data. Each of the luminance ratio and the color difference difference value is an 8-bit value.

  The data format of the original image data, the data format of the base image data, and the data format of the difference data are not particularly limited. For example, each pixel value of the original image data may be a YCbCr value. The number of bits of each gradation value of the original image data may be more or less than 32 bits. The original image data may not be HDR image data. Each pixel value of the base image data may be a YCbCr value. The number of bits of each gradation value of the base image data may be more or less than 8 bits. The base image data may not be JPEG image data. The number of bits of each data value of the difference data may be more or less than 8 bits. The luminance difference data may not be luminance ratio data. The color difference data may not be the color difference difference data. The difference data may not include one of the luminance difference data and the color difference data.

  The decoding unit 3 decodes the data output from the receiver 2 to separate the data output from the receiver 2 into base image data, luminance ratio data, and color difference difference data. Then, the decoding unit 3 outputs base image data, luminance ratio data, and color difference difference data to the image processing unit 4.

  The image processing unit 4 performs data processing using the data (base image data, luminance ratio data, and color difference difference data) output from the decoding unit 3. In this embodiment, the image processing unit 4 includes a base image processing unit 4a, a luminance ratio processing unit 4b, and a color difference difference processing unit 4c. The base image processing unit 4a performs data processing (base image processing) using the base image data. The luminance ratio processing unit 4b performs data processing (luminance ratio processing) using the luminance ratio data. Specifically, the luminance ratio processing unit 4b corrects the luminance ratio data based on the base image processing. The color difference difference processing unit 4c performs data processing (color difference difference processing) using the color difference difference data. Specifically, the color difference difference processing unit 4c corrects the color difference difference data based on the base image processing. The base image processing unit 4 a outputs the base image data after the base image processing is performed to the synthesis unit 5. The luminance ratio processing unit 4 b outputs the luminance ratio data after the luminance ratio processing is performed to the synthesis unit 5. The color difference difference processing unit 4 c outputs the color difference difference data after the color difference difference processing is performed to the synthesis unit 5.

  The synthesizing unit 5 includes data (base image data, luminance ratio data, and color difference difference data) output from the image processing unit 4 (base image processing unit 4a, luminance ratio processing unit 4b, and color difference difference processing unit 4c). To get. Then, the synthesis unit 5 generates image data (third image data) from the acquired data, and outputs the generated image data to the tone mapping unit 6. Specifically, the combining unit 5 generates image data (combined image data) by combining the acquired data, and outputs the combined image data to the tone mapping unit 6. Note that the method of generating the composite image data is not particularly limited as long as the composite image data is generated using the data output from the image processing unit 4.

  In this embodiment, the composite image data has the same data format as the original image data. That is, the composite image data is HDR image data in which each pixel value is an RGB value and each gradation value (R value, G value, and B value) is a 32-bit value. The data format of the composite image data is not particularly limited. For example, each pixel value of the composite image data may be a YCbCr value. The number of bits of each gradation value of the composite image data may be more or less than 32 bits. The composite image data may not be HDR image data. The data format of the composite image data may be different from the data format of the original image data. The composite image data may be recorded in a storage unit (not shown).

  The tone mapping unit 6 converts the combined image data output from the combining unit 5 into display image data (fourth image data), and outputs the display image data to the display unit 7. The display image data has a data format corresponding to the display unit 7. The data size of the display image data is smaller than the data size of the composite image data. In the present embodiment, the display unit 7 corresponds to RGB values and 10-bit gradation values (R value, G value, and B value). That is, the display unit 7 can accurately display an RGB value composed of a 10-bit R value, a 10-bit G value, and a 10-bit B value, and can accurately display other pixel values. Can not. For this reason, in the present embodiment, the tone mapping unit 6 converts the display value to each of the display image data in which each pixel value is an RGB value and each gradation value (R value, G value, and B value) is a 10-bit value. , Convert the composite image data. Specifically, the tone mapping unit 6 generates display image data by applying tone mapping (gradation compression) for converting each gradation value from a 32-bit value to a 10-bit value to the composite image data.

  The tone mapping method is not particularly limited. For example, as tone mapping, log conversion using log characteristics as conversion characteristics (corresponding relationship between gradation values before conversion and gradation values after conversion) may be performed. The data format corresponding to the display unit 7 is not particularly limited. For example, the pixel value corresponding to the display unit 7 may be a YCbCr value. The number of bits of the gradation value corresponding to the display unit 7 may be more or less than 10 bits. The data format corresponding to the display unit 7 may include other information such as an image size.

  The display unit 7 displays an image on the screen based on the display image data output from the tone mapping unit 6.

  The CPU 9 controls processing of each functional unit described above. The memory 10 stores various data (software program, image data, control data, etc.) used in each functional unit (including the CPU 9) described above. For example, the CPU 9 controls the processing of each functional unit by reading and executing a software program from the memory 10. Each functional unit included in the display device 1 is connected to the bidirectional bus 8, and data transmission between the functional units is performed via the bidirectional bus 8.

  Although the above-described base image processing is not particularly limited, for example, the base image processing is at least a part of image processing (desired image processing) for changing the gradation value of image data. The desired image processing may be image processing that changes all the gradation values of the image data, or may be image processing that changes a part of the gradation values of the image data. The desired image processing may be image processing performed on the entire image region, or may be image processing performed only on a part of the image region. The receiver 2 may acquire single image data (fifth image data) without acquiring base image data and difference data. In that case, processing may be performed as follows.

  First, the tone mapping unit 6 performs tone mapping on the image data (acquired image data) acquired by the receiver 2 to generate tone mapping image data (sixth image data). The tone mapping image data has a data format corresponding to the display unit 7. The data size of the tone mapping image data is smaller than the data size of the acquired image data. Next, the image processing unit 4 generates display image data by performing the desired image processing on the tone mapping image data. And the process of the decoding part 3 and the process of the synthetic | combination part 5 are abbreviate | omitted, and the image based on display image data is displayed on the display part 7. FIG.

The processing load and power consumption of the display device 1 can be reduced by changing the processing to be executed, the order of the processing to be executed, and the like depending on the type of data acquired by the receiver 2. For example, in the above example, the processing of the decoding unit 3 and the processing of the combining unit 5 are omitted, so that the processing load and power consumption of the display device 1 can be reduced. Furthermore, in the above example, the order of desired image processing and tone mapping is switched. Thereby, the target image data that is a target of desired image processing can be changed from the acquired image data to tone mapping image data having a data size smaller than that of the acquired image data. That is, the data size of the target image data can be reduced. As a result, the processing load of desired image processing can be reduced, and the processing load and power consumption of the display device 1 can be further reduced.

  The data format of the acquired image data is not particularly limited. For example, each pixel value of the acquired image data is an RGB value, and each gradation value (R value, G value, and B value) of the acquired image data is a 12-bit value. The acquired image data may be HDR image data or may not be HDR image data.

  Note that the data size of the tone mapping image data may be larger than the data size of the acquired image data. Even in that case, if the processing of the decoding unit 3 and the processing of the combining unit 5 are omitted, the processing load and power consumption of the display device 1 can be reduced. In that case, the desired image processing may be performed prior to tone mapping. Thereby, acquired image data having a data size smaller than that of tone mapping image data can be used as target image data. That is, the data size of the target image data can be reduced. As a result, the processing load of desired image processing can be reduced, and the processing load and power consumption of the display device 1 can be further reduced.

  FIG. 2 is a block diagram illustrating a configuration example of the image processing unit 4. In FIG. 2, the same reference numerals as those in FIG. In the present embodiment, as the base image processing, image processing (desired image processing) for changing the gradation value of the image data is performed. Although the desired image processing is not particularly limited, in this embodiment, gain processing and offset processing are performed as desired image processing. The gain process is a process of multiplying the set gain value by the gradation value of the image data, and the offset value is a process of adding the set offset value to the gradation value of the image data. It can be said that the set gain value and offset value are “parameters used in the base image processing”. In this embodiment, the gain process is performed after the offset process, but the offset process may be performed after the gain process.

  The base image processing unit 4a includes a color space conversion unit 401, a coordinate acquisition unit 402, an offset setting unit 403, addition units 403a to 403c, a gain setting unit 404, multiplication units 404a to 404c, and a processing information generation unit 405. The base image data output from the decoding unit 3 is input to the color space conversion unit 401.

  The color space conversion unit 401 changes the pixel value of the base image data input to the color space conversion unit 401 from an RGB value (R value, G value, B value) = (R, G, B) to a YCbCr value (Y value, Cb value, Cr value) = (Y, Cb, Cr). The coordinate acquisition unit 402 acquires the coordinates of the pixel that is the target of the base image processing from the color space conversion unit 401, and outputs coordinate information indicating the acquired coordinates to the luminance ratio processing unit 4b and the color difference difference processing unit 4c.

The offset setting unit 403 sets offset values A, B, and C. The offset value A is an offset value added to the Y value, the offset value B is an offset value added to the Cb value, and the offset value C is an offset value added to the Cr value. For example, the offset setting unit 403 automatically sets the offset values A, B, and C according to the user operation, the usage status of the display device 1, and the like. The user operation is a user operation for specifying a parameter, a user operation for specifying an operation mode of the display device 1, and the usage status of the display device 1 is the ambient brightness of the display device 1, the temperature of the display device 1, etc. It is. The offset value A, offset value B, and offset value C may or may not be set individually. One offset value may be set as the offset value A, the offset value B, and the offset value C. Offset values A, B, and C may be set individually for each pixel (or for each image area composed of a predetermined number of pixels), and are common values between pixels (or for each image area composed of a predetermined number of pixels). May be set as offset values A, B, and C.

  The adding unit 403a adds the offset value A to the Y value (Y). The adding unit 403b adds the offset value B to the Cb value (Cb). Then, the addition unit 403c adds the offset value C to the Cr value (Cr).

  The gain setting unit 404 sets gain values P, Q, and R. The gain value P is a gain value to be multiplied by the Y value, the gain value Q is a gain value to be multiplied by the Cb value, and the gain value R is a gain value to be multiplied by the Cr value. For example, the gain setting unit 404 automatically sets the gain values P, Q, and R according to the user operation, the usage status of the display device 1, and the like. The gain value P, gain value Q, and gain value R may or may not be set individually. One gain value may be set as the gain value P, the gain value Q, and the gain value R. The gain values P, Q, and R may be set individually for each pixel (or for each image region composed of a predetermined number of pixels), and are common values between pixels (or for each image region composed of a predetermined number of pixels). May be set as gain values P, Q, and R.

  The multiplier 404a multiplies the Y value (Y + A) by the gain value P. The multiplier 404b multiplies the Cb value (Cb + A) by the gain value Q. Then, the multiplier 404c multiplies the Cr value (Cr + C) by the gain value R. Thereby, as a result of the base image processing, the Y value (Y ′ = (Y + A) × P), the Cb value (Cb ′ = (Cb + B) × Q), and the Cr value (Cr ′ = (Cr + C) × R) Is obtained. The Y value (Y ') is output to the luminance ratio processing unit 4b, and the Cb value (Cb') and the Cr value (Cr ') are output to the color difference difference processing unit 4c. YCbCr values (Y ′, Cb ′, Cr ′), which are pixel values of the base image data after the base image processing is performed, are output to the combining unit 5.

  The processing information generation unit 405 generates processing information and outputs the generated processing information to the luminance ratio processing unit 4b and the color difference difference processing unit 4c. The processing information is information related to base image processing. For example, the processing information indicates the type of base image processing, parameters used in the base image processing, and the like. In the present embodiment, processing information is generated based on the offset values A, B, and C set by the offset setting unit 403 and the gain values P, Q, and R set by the gain setting unit 404. In the present embodiment, the processing information indicates offset values A, B, and C and gain values P, Q, and R.

  The luminance ratio processing unit 4b includes a luminance ratio determination unit 406, an HDR-Y value determination unit 407, and a correction ratio determination unit 408. The luminance ratio data output from the decoding unit 3 is input to the luminance ratio determination unit 406.

  Based on the coordinate information output from the coordinate acquisition unit 402, the luminance ratio determination unit 406 calculates the luminance ratio r corresponding to the pixel that is the target of the base image processing from the luminance ratio data input to the luminance ratio determination unit 406. to decide. Then, the luminance ratio determining unit 406 outputs the luminance ratio r to the HDR-Y value determining unit 407.

  The HDR-Y value determination unit 407 determines a Y value (HDR_Y) obtained by performing desired image processing on the HDR image data that is the original image data from the luminance ratio r and the processing information. Then, the HDR-Y value determination unit 407 outputs the Y value (HDR_Y) to the correction ratio determination unit 408.

  The correction ratio determination unit 408 determines the correction ratio r ′ (the luminance ratio after correction) from the Y value (HDR_Y) and the Y value (Y ′) after the base image processing is performed. Then, the correction ratio determination unit 408 outputs the determined correction ratio r ′ to the synthesis unit 5.

  The color difference difference processing unit 4 c includes a color difference difference determination unit 409, an HDR-Cb / Cr value determination unit 410, and a correction difference determination unit 411. The color difference difference data output from the decoding unit 3 is input to the color difference difference determination unit 409.

  The color difference difference determination unit 409, based on the coordinate information output from the coordinate acquisition unit 402, from the color difference difference data input to the color difference difference determination unit 409, the color difference difference value db corresponding to the pixel that is the target of the base image processing. , Dr. The color difference difference db is a Cb value difference value, and the color difference difference dr is a Cr value difference value. Then, the color difference difference determination unit 409 outputs the color difference difference values db and dr to the HDR-Cb / Cr value determination unit 410.

  The HDR-Cb / Cr value determination unit 410 obtains the Cb value (HDR_Cb) and the Cr value (HDR_Cr) obtained by performing desired image processing on the HDR image data that is the original image data, and the color difference difference values db and dr. Judgment from processing information. Then, the HDR-Cb / Cr value determination unit 410 outputs the Cb value (HDR_Cb) and the Cr value (HDR_Cr) to the correction difference determination unit 411.

  The correction difference determination unit 411 uses the correction difference values db ′ and dr ′ as Cb values (HDR_Cb), Cr values (HDR_Cr), Cb values after the base image processing (Cb ′), and base image processing. Is determined from the Cr value (Cr ′) after the above. The correction difference value db 'is a Cb value difference value and a corrected color difference difference value. The correction difference value dr ′ is a difference value between Cr values and a color difference difference value after correction. Then, the correction difference determination unit 411 outputs the determined correction difference values db ′ and dr ′ to the synthesis unit 5.

  FIG. 3 is a flowchart illustrating an example of a processing flow of the display device 1. An example of the processing flow of the display device 1 will be described with reference to FIG. In the following description, the luminance ratio is the ratio of the luminance of the original image data to the luminance of the base image data. The color difference difference value is a difference value obtained by subtracting the color difference of the base image data from the color difference of the original image data.

  First, the receiver 2 acquires JPEG-HDR data and outputs the acquired data to the decoding unit 3 (S101). Next, the decoding unit 3 decodes the data acquired in S101, thereby separating the data acquired in S101 into base image data, luminance ratio data, and color difference difference data (S102). The decoding unit 3 outputs the acquired base image data to the base image processing unit 4a, outputs the acquired luminance ratio data to the luminance ratio processing unit 4b, and outputs the acquired color difference difference data to the color difference difference determination unit 409. Then, the color space conversion unit 401 changes the pixel value of the base image data acquired in S102 from the RGB value elementary values (R, G, B) that are RGB color space values to YCbCr color space values that are YCbCr color space values. Conversion into values (Y, Cb, Cr) (S103).

  Next, the coordinate acquisition unit 402 acquires the coordinates of the pixel that is the target of the base image processing (S105 and S106 described later) from the color space conversion unit 401, and outputs coordinate information indicating the acquired coordinates (S104). The coordinate information is output to the luminance ratio processing unit 4b and the color difference difference processing unit 4c. The timing of the process of S104 is not particularly limited. For example, the process of S104 may be performed after the process of S105, after the process of S106, or the like. The process of S104 may be performed in parallel with the processes of S103, S105, and S106.

  Then, the adding units 403a to 403c perform an offset process on the YCbCr values (Y, Cb, Cr) acquired in S103 (S105). Specifically, the adder 403a adds the offset value A to the Y value (Y), the adder 403b adds the offset value B to the Cb value (Cb), and the adder 403c offsets the Cr value (Cr). Add the value C. If the offset values A, B, and C are set before the process of S105, the timing at which the offset values A, B, and C are set is not particularly limited. In the present embodiment, offset values A, B, and C are set at any timing of the user.

  Next, the multipliers 404a to 404c perform gain processing on the YCbCr value (YCbCr value after offset processing (Y + A, Cb + B, Cr + C)) acquired in S105 (S106). Specifically, the multiplier 404a multiplies the Y value (Y + A) by the gain value P, the multiplier 404b multiplies the Cb value (Cb + A) by the gain value Q, and the multiplier 404c gains the Cr value (Cr + C) by the gain. Multiply the value R. If the gain values P, Q, and R are set before the processing of S106, the timing at which the gain values P, Q, and R are set is not particularly limited. In this embodiment, the gain values P, Q, and R are set at any timing of the user.

By the processes in S105 and S106, the values shown in Expression 1 are obtained as the Y value (Y ′), Cb value (Cb ′), and Cr value (Cr ′), which are the results of the base image processing. The Y value (Y ′) is output to the luminance ratio processing unit 4b, and the Cb value (Cb ′) and the Cr value (Cr ′) are output to the color difference difference processing unit 4c. YCbCr values (Y ′, Cb ′, Cr ′), which are pixel values of the base image data after the base image processing is performed, are output to the synthesis unit 5.

Y ′ = (Y + A) × P
Cb ′ = (Cb + B) × Q
Cr ′ = (Cr + C) × R
... (Formula 1)

  The processing information generation unit 405 generates processing information indicating the offset values A, B, and C used in S105 and the gain values P, Q, and R used in S106, and the generated processing information is subjected to luminance ratio processing. To the unit 4b and the color difference difference processing unit 4c. The output timing of the processing information is not particularly limited. For example, the process of outputting the offset values A, B, and C may be performed in parallel with the process of S105, and the process of outputting the gain values P, Q, and R may be performed in parallel with the process of S106. The process of outputting the offset values A, B, and C may be performed before the process of S105 or S106. The process of outputting the gain values P, Q, and R may be performed before the process of S105 or S106. The process of outputting the offset values A, B, and C may be performed after the process of S105 or S106. The process of outputting the gain values P, Q, and R may be performed after the process of S105 or S106.

  Following the process of S106, the process proceeds to S110. In the present embodiment, the processing of S107 to S109 is performed by the luminance ratio processing unit 4b and the color difference difference processing unit 4c after the processing of S104 and before the processing of S110. The process of S107-109 may be performed in parallel with the process of S105 and S106, and may not be so. The processing of S107 to 109 may be performed after the processing of S106. The processing of the luminance ratio processing unit 4b may or may not be performed in parallel with the processing of the color difference difference processing unit 4c.

When the luminance difference data acquired in S102 and the coordinate information output in S104 are input to the luminance ratio determining unit 406, the luminance ratio determining unit 406 performs the process of S107. In S107, the luminance ratio determining unit 406 determines the luminance ratio r corresponding to the pixel that is the target of the base image processing from the input luminance ratio data based on the input coordinate information. And
The luminance ratio determination unit 406 outputs the luminance ratio r to the HDR-Y value determination unit 407. Similarly, when the color difference difference data acquired in S102 and the coordinate information output in S104 are input to the color difference difference determination unit 409, the color difference difference determination unit 409 performs the process of S107. In S107, the color difference difference determination unit 409 determines the color difference difference values db and dr corresponding to the pixel that is the target of the base image processing from the input color difference difference data based on the input coordinate information. Then, the color difference difference determination unit 409 outputs the color difference difference values db and dr to the HDR-Cb / Cr value determination unit 410.

  When the luminance ratio r determined in S107 and the processing information output from the processing information generation unit 405 are input to the HDR-Y value determination unit 407, the HDR-Y value determination unit 407 performs the process of S108. In S108, the HDR-Y value determination unit 407 determines the Y value (HDR_Y) obtained by performing desired image processing on the HDR image data that is the original image data from the luminance ratio r and the processing information. Then, the HDR-Y value determination unit 407 outputs the Y value (HDR_Y) to the correction ratio determination unit 408. Similarly, when the color difference difference values db and dr determined in S107 and the processing information output from the processing information generation unit 405 are input to the HDR-Cb / Cr value determination unit 410, the HDR-Cb / Cr value is determined. The determination unit 410 performs the process of S108. In S108, the HDR-Cb / Cr value determination unit 410 obtains the Cb value (HDR_Cb) and the Cr value (HDR_Cr) obtained by performing desired image processing on the HDR image data which is the original image data, as a color difference difference value. Judgment is made from db, dr and processing information. Then, the HDR-Cb / Cr value determination unit 410 outputs the Cb value (HDR_Cb) and the Cr value (HDR_Cr) to the correction difference determination unit 411.

In this embodiment, a Y value (HDR_Y), a Cb value (HDR_Cb), and a Cr value (HDR_Cr) are calculated using the following Expression 2. Therefore, the YCbCr value (Y, Cb, Cr) of the base image data is included in the processing information, for example. The method of notifying YCbCr values (Y, Cb, Cr) to HDR-Y value determining unit 407 and HDR-Cb / Cr value determining unit 410 is not particularly limited. The YCbCr value (Y, Cb, Cr) may be included in the coordinate information. Then, the YCbCr value (Y, Cb, Cr) is notified from the luminance ratio determination unit 406 to the HDR-Y value determination unit 407, and the YCbCr value (Y, Y) is transmitted from the color difference difference determination unit 409 to the HDR-Cb / Cr value determination unit 410. Cb, Cr) may be notified.

HDR_Y = (Y × r + A) × P
HDR_Cb = (Cb + db + B) × Q
HDR_Cr = (Cr + dr + C) × R
... (Formula 2)

When the Y value (HDR_Y) determined in S108 and the Y value (Y ′) obtained in S106 are input to the correction ratio determination unit 408, the correction ratio determination unit 408 performs the process of S109. The Y value (Y ′) is the Y value after the base image processing is performed. In S109, the correction ratio determination unit 408 determines the correction ratio r ′ (the corrected luminance ratio) from the Y value (HDR_Y) and the Y value (Y ′). Then, the correction ratio determination unit 408 outputs the determined correction ratio r ′ to the synthesis unit 5. Similarly, the Cb value (HDR_Cb) and Cr value (HDR_Cr) determined in S108 and the Cb value (Cb ′) and Cr value (Cr ′) obtained in S106 are input to the correction difference determination unit 411. Then, the correction difference determination unit 411 performs the process of S109. The Cb value (Cb ′) is a Cb value after the base image processing is performed, and the Cr value (Cr ′) is a Cr value after the base image processing is performed. In S109, the correction difference determination unit 411 converts the correction difference values db ′ and dr ′ (corrected color difference difference values) into Cb values (HDR_Cb), Cr values (HDR_Cr), Cb values (Cb ′), and It is determined from the Cr value (Cr ′). Then, the correction difference determination unit 411 outputs the determined correction difference values db ′ and dr ′ to the synthesis unit 5. S109
Following this process, the process proceeds to S110.

The correspondence relationship between the Y value (HDR_Y), the Y value (Y ′), and the correction ratio r ′ is shown in Equation 3. In the present embodiment, the correction ratio r ′ is calculated using Expression 4 obtained by modifying Expression 3. Specifically, the correction ratio r ′ is calculated by dividing the Y value (HDR_Y) by the Y value (Y ′).

Y ′ × r ′ = HDR_Y = (Y × r + A) × P (Formula 3)

r ′ = HDR_Y / Y ′
= ((Y × r + A) × P) / ((Y + A) × P)
= (Y × r + A) / (Y + A)
... (Formula 4)

Correspondence relationship between Cb value (HDR_Cb), Cb value (Cb ′), and correction difference value db ′, and correspondence relationship between Cr value (HDR_Cr), Cr value (Cr ′), and correction difference value dr ′ It is shown in Formula 5. In the present embodiment, the correction difference values db ′ and dr ′ are calculated using Expression 6 obtained by modifying Expression 5. Specifically, the correction difference value db ′ is calculated by subtracting the Cb value (Cb ′) from the Cb value (HDR_Cb), and the correction difference by subtracting the Cr value (Cr ′) from the Cr value (HDR_Cr). The value dr ′ is calculated.

Cb ′ + db ′ = HDR_Cb = (Cb + db + B) × Q
Cr ′ + dr ′ = HDR_Cr = (Cr + dr + C) × R
... (Formula 5)

db ′ = HDR_Cb−Cb ′
= (Cb + db + B) * Q- (Cb + B) * Q
= Db × Q
dr '= HDR_Cr-Cr'
= (Cr + dr + B) × R− (Cr + C) × R
= Dr * R
... (Formula 6)

Following the processing of S106 and S109, the processing of S110 is performed. In S110, the synthesis unit 5 outputs data (base image data, luminance ratio data, and data) output from the image processing unit 4 (base image processing unit 4a, luminance ratio processing unit 4b, and color difference difference processing unit 4c). Color difference data). Then, the synthesizing unit 5 generates synthesized image data (third image data) from the acquired data. Specifically, the synthesizer 5 outputs the YCbCr value (Y ′, Cb ′, Cr ′) output in S106, the correction ratio r ′ output in S109, and the correction difference value db ′ output in S109. , Dr ′, the YCbCr value of the composite image data is calculated. Then, the synthesis unit 5 converts the calculated YCbCr value into RGB values (HDR_R, HDR_G, HDR_B) of the synthesized image data. In the present embodiment, the YCbCr value of the composite image data is calculated using the following Expression 7.

Y value of composite image data = Y ′ × r ′
Cb value of composite image data = Cb ′ + db ′
Cr value of composite image data = Cr ′ + dr ′
... (Formula 7)

  From Equations 3, 5, and 7, it can be seen that the composite image data substantially matches the image data obtained by performing desired image processing on the HDR image data that is the original image data (“substantially match” means “completely match” Match "). Specifically, it can be seen that the composite image data completely matches the image data obtained by performing desired image processing on the HDR image data that is the original image data. Note that the composite image data may not completely match the image data obtained by performing desired image processing on the original image data.

  Thereafter, the tone mapping unit 6 generates display image data from the composite image data, and the display unit 7 displays an image based on the display image data.

  As described above, according to the present embodiment, base image processing (data processing using base image data) is performed, and difference data is corrected based on the performed base image processing. Then, composite image data is generated from the base image data after the base image processing is performed and the corrected difference data. Thereby, desired image data can be obtained with high accuracy and simple configuration as image data corresponding to HDR image data after image processing. Specifically, the desired image data can be increased with a simple configuration of “data processing using base image data and difference data whose data size is smaller than that of HDR image data” and “composition of data after data processing”. It can be obtained with accuracy. According to the above specific example, by performing “8-bit data processing” and “combining data after data processing”, the desired image processing is performed on 32-bit HDR image data (original image data). Image data substantially the same as the obtained image data can be obtained. As a result, the processing load, circuit scale, etc. of desired image processing can be reduced, and the processing load, circuit scale, power consumption, manufacturing cost, etc. of the display device (image processing apparatus) can be reduced. Note that “substantially identical” includes “completely identical”.

  Although an example in which both the luminance difference data and the color difference data are corrected has been described, only one of the luminance difference data and the color difference data may be corrected. In this embodiment, the example in which the difference data is corrected using the base image processing parameter and the base image processing result has been described. However, one of the base image processing parameter and the base image processing result is described. The difference data may be corrected using only. From Equation 6 (“db ′ = db × Q” and “dr ′ = dr × R”), it can be seen that the color difference difference data can be corrected using only the gain values Q and R. The desired image processing may include only one of gain processing and offset processing. The parameter may include only one of the gain value and the offset value. The desired image processing may include other image processing.

<Example 2>
Embodiment 2 of the present invention will be described below. Also in the present embodiment, as base image processing, image processing (desired image processing) for changing the gradation value of image data is performed. However, in the present embodiment, as the base image processing (desired image processing), a feature amount of image data is acquired, and a process of changing the gradation value of the image data using a parameter corresponding to the acquired feature amount is performed. Is called. Specifically, as the base image processing, a feature acquisition process for acquiring the feature amount of the image data and a tone conversion process for converting the tone characteristic of the image data with the conversion characteristic according to the acquired feature amount are performed. . In the case where a gamma curve is used as the conversion characteristic, the gradation conversion processing can be said to be “dynamic gamma processing”. Hereinafter, points (configuration and processing) that are different from the first embodiment will be described in detail, and description of the same points as those of the first embodiment will be omitted.

FIG. 4 is a block diagram illustrating a configuration example of the image processing unit 4 according to the present embodiment. 4, the same reference numerals as those in the first embodiment are assigned to the same functional units as those in the first embodiment (FIGS. 1 and 2). The base image processing unit 4a according to the present embodiment includes a color space conversion unit 401, a coordinate acquisition unit 402, a processing information generation unit 405, a feature amount acquisition unit 601, a feature amount storage unit 602, and a gradation conversion unit 603. .

  The feature amount acquisition unit 601 acquires the feature amount of the base image data (feature acquisition process). In this embodiment, the base image data output from the color space conversion unit 401 (base image data after the processing of the color space conversion unit 401; base image data in which each pixel value is a YCbCr value) is a feature amount. Input to the acquisition unit 601. The feature amount acquisition unit 601 acquires the feature amount of the base image data output from the color space conversion unit 401, and records the acquired feature amount in the feature amount storage unit 602. Also, the feature amount acquisition unit 601 outputs the base image data output from the color space conversion unit 401 to the gradation conversion unit 603. The feature amount storage unit 602 stores the feature amount acquired by the feature amount acquisition unit 601. As the feature amount storage unit 602, for example, an SRAM can be used.

  As the feature amount, for example, representative values (maximum value, minimum value, average value, intermediate value, mode value, etc.) of a plurality of gradation values possessed by image data, a histogram of a plurality of gradation values possessed by image data, Etc. are acquired. Information indicating a predetermined image area (an image area corresponding to a predetermined object, an image area whose luminance is higher than a threshold, an image area whose luminance is lower than a threshold, an image area having a predetermined color) as a feature amount, gradation The number, the magnitude of the image movement, and the like may be acquired.

  Note that the type of feature amount, the method for acquiring the feature amount, and the like are not particularly limited. For example, the feature amount may be acquired from the base image data before the processing of the color space conversion unit 401 is performed. A feature amount may be acquired for each type of gradation value, or a common feature amount may be acquired among a plurality of types of gradation values. Specifically, the feature value of the Y value, the feature value of the Cb value, and the feature value of the Cr value may be acquired individually. Only one of the Y-value feature quantity, the Cb-value feature quantity, the Cr-value feature quantity, and the YCbCr-value feature quantity may be acquired. A feature amount corresponding to the entire image region may be acquired, or a feature amount corresponding to only a part of the image region may be acquired. When the base image data is moving image data, a feature amount may be acquired for each frame, or a common feature amount may be acquired from a plurality of frames. All the frames may be used as feature quantity acquisition targets, or only some of the frames (some scenes) may be used as feature quantity acquisition targets.

  The gradation conversion unit 603 reads the feature amount acquired by the feature amount acquisition unit 601 from the feature amount storage unit 602, and determines conversion characteristics according to the read feature amount. Then, according to the determined conversion characteristic, the gradation conversion unit 603 determines the gradation characteristic of the base image data (base image data after the processing of the color space conversion unit 401) output from the feature amount acquisition unit 601. Conversion (gradation conversion processing). Thereby, base image data after base image processing (feature acquisition processing and gradation conversion processing) is performed is obtained. Similarly to the first embodiment, YCbCr values (Y ′, Cb ′, Cr ′), which are pixel values of the base image data after the base image processing is performed, are output to the synthesis unit 5. The Y value (Y ′) is output to the luminance ratio processing unit 4b as a result of the base image processing, and the Cb value (Cb ′) and the Cr value (Cr ′) are output as the color difference difference processing unit as a result of the base image processing. To 4c.

The method for determining the conversion characteristics is not particularly limited. One gamma curve corresponding to the feature amount may be determined as the conversion characteristic or not. A plurality of gamma curves corresponding to the feature amount may be determined, and one conversion characteristic may be determined by combining the plurality of gamma curves. For example, a first threshold value and a second threshold value that is a value equal to or lower than the first threshold value are predetermined, a gamma curve corresponding to a range of gradation values equal to or higher than the first threshold value, and a second threshold value equal to or higher than the second threshold value. The gamma curve corresponding to the range of gradation values may be determined according to the feature amount. Then, one conversion characteristic may be determined by connecting the two gamma curves. Each of the first threshold value and the second threshold value may be a fixed value determined in advance by the manufacturer, or may be a value that can be changed by the user.

  In this embodiment, the processing information generation unit 405 generates information indicating the conversion characteristics used in the gradation conversion unit 603 as processing information, and the generated processing information is used as the luminance ratio processing unit 4b and the color difference difference processing unit 4c. Output to. For example, the processing information generation unit 405 acquires one or more gamma values indicating the conversion characteristics used in the gradation conversion unit 603 from the gradation conversion unit 603, and outputs the acquired gamma values as processing information.

  FIG. 5 is a flowchart illustrating an example of a processing flow of the display device 1 according to the present embodiment. An example of the processing flow of the display device 1 according to the present embodiment will be described with reference to FIG. In FIG. 5, the same processes as those in the first embodiment (FIG. 3) are denoted by the same reference numerals as those in the first embodiment.

  First, similarly to the first embodiment, the processes of S101 to S104 are performed. Next, the feature quantity acquisition unit 601 performs the process of S201 (feature acquisition process). In S201, the feature amount acquisition unit 601 acquires the feature amount from the base image data (base image data after each pixel value is converted into a YCbCr value) after the processing of S103 is performed, and the acquired feature amount Is recorded in the feature amount storage unit 602.

  Then, the gradation conversion unit 603 performs the process of S202 (gradation conversion process). In S202, the gradation conversion unit 603 reads the feature amount recorded in S201 from the feature amount storage unit 602, and determines conversion characteristics according to the read feature amount. Then, the tone conversion unit 603 converts the tone characteristics of the base image data after the processing of S103 is performed according to the determined conversion characteristics.

  Through the processes of S201 and S202, base image data after being subjected to base image processing (base image data after being subjected to gradation conversion processing) is obtained. As described above, the YCbCr values (Y ′, Cb ′, Cr ′) that are the pixel values of the base image data after the base image processing is performed are output to the synthesizing unit 5. The Y value (Y ′) is output to the luminance ratio processing unit 4b as a result of the base image processing, and the Cb value (Cb ′) and the Cr value (Cr ′) are output as the color difference difference processing unit as a result of the base image processing. To 4c.

  Following the process of S202, the process proceeds to S110. Also in the present embodiment, the processing of S107 to S109 is performed by the luminance ratio processing unit 4b and the color difference difference processing unit 4c after the processing of S104 and before the processing of S110. However, in S108, the Y value (HDR_Y), the Cb value (HDR_Cb), and the Cr value (HDR_Cr) are determined based on the conversion characteristics, not the offset value or the gain value. The conversion characteristic is determined from the processing information. Following the processing of S109, the processing proceeds to S110. Also in the method of the present embodiment, the difference data is corrected based on the processing information (conversion characteristics) and the like, so that the synthesized image data is substantially the same as the image data obtained by performing desired image processing on the original image data. Image data can be obtained.

As described above, according to the present embodiment, the base image processing is performed as in the first embodiment, and the difference data is corrected based on the performed base image processing. Then, composite image data is generated from the base image data after the base image processing is performed and the corrected difference data. Thereby, desired image data can be obtained with high accuracy and simple configuration as image data corresponding to HDR image data after image processing. According to the above specific example, by performing “8-bit data processing” and “combining data after data processing”, by performing desired image processing on 32-bit HDR image data (original image data). Image data substantially the same as the obtained image data can be obtained. Therefore, it is possible to reduce the processing load of desired image processing, the circuit scale, and the like. Further, according to the above specific example, since the feature amount is acquired from the 8-bit base image data, it is possible to obtain a feature amount whose data size is smaller than the feature amount acquired from the 32-bit HDR image data. Therefore, the storage capacity (circuit scale) of the storage unit that stores the feature amount can be reduced. As a result, the processing load, circuit scale, power consumption, manufacturing cost, etc. of the display device (image processing device) can be reduced.

  Note that the “process for changing the tone value of image data using a parameter corresponding to the feature amount” is not limited to the tone conversion process. For example, “a process for blurring an image using a parameter corresponding to a feature amount” may be performed as “a process for changing a gradation value of image data using a parameter corresponding to a feature amount”. The feature quantity itself may be used as a parameter corresponding to the feature quantity. “Process for changing the gradation value of image data using a parameter corresponding to a feature amount” may not be included in the base image processing. Only the feature acquisition process may be performed as the base image process. Then, in the correction of the difference data, the feature amount may be used as a result of the base image processing.

<Example 3>
Embodiment 3 of the present invention will be described below. The base image processing according to the present embodiment is a part of image processing (desired image processing) for changing the gradation value of image data. Specifically, the desired image processing according to the present embodiment is processing for detecting a predetermined image region and changing the gradation value of the image data based on the detection result of the predetermined image region. The base image process is a process for detecting a predetermined image area. More specifically, the desired image processing according to the present embodiment includes an edge detection process for detecting an edge and an edge enhancement process for enhancing the detected edge. The base image processing according to the present embodiment is edge detection processing. In the following, points (configuration and processing) that are different from the first embodiment will be described in detail, and description of the same points as those of the first and second embodiments will be omitted.

  FIG. 6 is a block diagram illustrating a configuration example of the image processing unit 4 according to the present embodiment. In FIG. 6, the same reference numerals as those in the first embodiment are assigned to the same functional units as those in the first embodiment (FIGS. 1 and 2). The base image processing unit 4a according to the present embodiment includes a color space conversion unit 401, a coordinate acquisition unit 402, a processing information generation unit 405, and an edge processing unit 800.

  The edge processing unit 800 performs an edge detection process for detecting an edge from the base image data. In this embodiment, the base image data output from the color space conversion unit 401 (base image data after the processing of the color space conversion unit 401; base image data in which each pixel value is a YCbCr value) is edge processed. Input to the unit 800. Then, the edge processing unit 800 performs edge detection processing using the base image data output from the color space conversion unit 401. Note that the edge detection process may be performed using base image data before the processing of the color space conversion unit 401, instead of the base image data after the processing of the color space conversion unit 401. .

  The edge processing unit 800 obtains base image data after the base image processing (edge detection processing) is performed. Similarly to the first embodiment, YCbCr values (Y ′, Cb ′, Cr ′), which are pixel values of the base image data after the base image processing is performed, are output to the synthesis unit 5. The Y value (Y ') is output to the luminance ratio processing unit 4b, and the Cb value (Cb') and the Cr value (Cr ') are output to the color difference difference processing unit 4c. In the base image processing (edge detection processing) of this embodiment, the gradation values (Y value, Cd value, and Cr value) of the base image data are not changed. Therefore, the Y value (Y ′) is equal to the Y value (Y) that is the output value of the color space conversion unit 401, and the Cb value (Cb ′) is the Cb value (Cb) that is the output value of the color space conversion unit 401. ) And the Cr value (Cr ′) is equal to the Cr value (Cr) that is the output value of the color space conversion unit 401.

The configuration of the edge processing unit 800 is not particularly limited, but in this embodiment, the edge processing unit 800 is used.
Has a block buffer 800a, a filter 800b, and a table 800c. The block buffer 800a is a buffer that temporarily stores image data. The edge processing unit 800 records image data (base image data corresponding to a part of the image area) used in the base image processing (edge detection processing) in the block buffer 800a (buffering processing). The filter 800b is a filter for edge detection processing. The edge processing unit 800 detects an edge from the image data by performing a filtering process using the filter 800b on the image data stored in the block buffer 800a. In the present embodiment, a process for changing the Y value of the edge is assumed as the edge enhancement process. Specifically, as the edge enhancement process, a process of multiplying the Y value of the edge by a gain value (≠ 1) is assumed. The table 800c is a table showing a correspondence relationship between the edge detection result and the edge enhancement gain value. The edge processing unit 800 acquires a gain value corresponding to the edge detection result (the obtained detection result) from the table 800c.

  The edge enhancement process is not limited to the process of multiplying the Y value of the edge by the gain value (≠ 1). For example, in the edge enhancement process, the Y value of the edge may be changed by another process such as a process of adding an offset value. In the edge enhancement process, other types of gradation values (Cb value, Cr value, etc.) at the edge may be changed. The gradation value of the image area different from the edge may be changed.

  As described above, in this embodiment, the edge emphasis process is a process that changes only the Y value and does not change the Cb value and the Cr value. Therefore, in this embodiment, the processing information generation unit 405 outputs information indicating the edge detection result and the gain value of the edge enhancement processing to the luminance ratio processing unit 4b as processing information. Then, the processing information generation unit 405 outputs information indicating that the color difference difference data is not changed to the color difference difference processing unit 4c as processing information. Specifically, processing information indicating the edge detection result and the gain value of the edge enhancement processing is output to the HDR-Y value determination unit 407 and the correction ratio determination unit 408. Then, processing information indicating that the color difference difference data is not changed is output to the HDR-Cb / Cr value determination unit 410 and the correction difference determination unit 411. As a result, the luminance ratio processing unit 4b corrects and outputs the luminance ratio data, and the color difference difference processing unit 4c outputs the color difference difference data without being changed. In this embodiment, since the base image processing is edge detection processing, the “edge detection result” is “base image processing result”.

  FIG. 7 is a flowchart illustrating an example of a processing flow of the display device 1 according to the present embodiment. An example of the processing flow of the display device 1 according to the present embodiment will be described with reference to FIG. In FIG. 7, the same processes as those in the first embodiment (FIG. 3) are denoted by the same reference numerals as those in the first embodiment.

  First, similarly to the first embodiment, the processes of S101 to S104 are performed. Next, the edge processing unit 800 performs the process of S301. In S301, the edge processing unit 800 performs edge detection processing using the base image data (base image data after each pixel value is converted into a YCbCr value) after the processing in S103 is performed.

  Through the processing in S301, base image data after the base image processing (base image data after the edge enhancement processing) is obtained. As described above, the YCbCr values (Y ′, Cb ′, Cr ′) that are the pixel values of the base image data after the base image processing is performed are output to the synthesizing unit 5. The Y value (Y ′) is output to the luminance ratio processing unit 4b as a result of the base image processing, and the Cb value (Cb ′) and the Cr value (Cr ′) are output as the color difference difference processing unit as a result of the base image processing. To 4c. In this embodiment, the YCbCr value (Y ′, Cb ′, Cr ′) is equal to the YCbCr value (Y, Cb, Cr) that is the pixel value of the base image data before the base image processing is performed.

  Following the process of S301, the process proceeds to S110. Also in the present embodiment, the processing of S107 to S109 is performed by the luminance ratio processing unit 4b and the color difference difference processing unit 4c after the processing of S104 and before the processing of S110. Following the processing of S109, the processing proceeds to S110.

  However, the luminance ratio processing unit 4b (HDR-Y value determination unit 407) is based on the edge detection result and the gain value of the edge emphasis process in S108, not the offset value, the gain value, the conversion characteristic, or the like. , Y value (HDR_Y) is determined. For example, the Y value (HDR_Y) is calculated by multiplying the Y value (Y × r) of the original image data by the gain value of the edge enhancement process. By using a value different from 1 as the gain value of the edge and using 1 as the gain value of the image area different from the edge, the Y value of the image area different from the edge is obtained from the Y value of the original image data. Only the Y value of the edge is changed from the Y value of the original image data without being changed. For an image area different from the edge, the process of multiplying the Y value of the original image data by the gain value may be omitted. The edge detection result and the edge enhancement gain value are determined from processing information indicating the edge detection result and the edge enhancement gain value.

  The color difference difference processing unit 4c omits the processing of S108 (determination of Cb value (HDR_Cb) and Cr value (HDR_Cr)), and performs the processing of S107 and S109. The HDR-Cb / Cr value determination unit 410 determines to omit the processing of S108 from the processing information indicating that the color difference difference data is not changed. Then, the HDR-Cb / Cr value determination unit 410 outputs the color difference difference values db and dr to the correction difference determination unit 411. When processing information indicating that the color difference difference data is not changed and the color difference difference values db and dr are input to the correction difference determination unit 411, the correction difference determination unit 411 determines the color difference difference values db and dr in S109. Processing to determine the correction difference values db ′ and dr ′ is performed.

  Also in the method of this embodiment, difference data (luminance ratio data) is corrected based on processing information (edge detection result and edge enhancement gain value) and the like. Therefore, it is possible to obtain substantially the same image data as the composite image data, which is obtained by performing desired image processing on the original image data.

  As described above, according to the present embodiment, in the configuration in which a part of desired image processing is performed as the base image processing, the difference data is corrected based on the performed base image processing as in the first embodiment. Is done. Then, composite image data is generated from the base image data after the base image processing is performed and the corrected difference data. Thereby, desired image data can be obtained with high accuracy and simple configuration as image data corresponding to HDR image data after image processing. According to the above specific example, by performing “8-bit data processing” and “combining data after data processing”, by performing desired image processing on 32-bit HDR image data (original image data). Image data substantially the same as the obtained image data can be obtained. Therefore, it is possible to reduce the processing load of desired image processing, the circuit scale, and the like. For example, the data size of a filter used for filtering processing, the storage capacity of a block buffer used for buffering processing, and the like can be reduced. As a result, the processing load, power consumption, manufacturing cost, etc. of the display device (image processing device) can be reduced.

The predetermined image area is not limited to the edge. For example, the predetermined image area may be an image area corresponding to a predetermined object such as a mountain, a person, a car, or a building. The base image processing may include processing for changing the gradation value based on the detection result of a predetermined image area. In this case, the “predetermined image region (edge) detection result” can also be said to be “a parameter used in the base image processing”. The “process for changing the gradation value based on the detection result of the predetermined image area” is not limited to the edge enhancement process. For example, the “process for changing the gradation value based on the detection result of the predetermined image area” may be the “process for blurring the predetermined image area”. The gain value of the edge enhancement process may not be included in the process information. When the edge gain value is a predetermined fixed value, the luminance ratio data can be corrected using only the edge detection result (the result of the base image processing).

  In addition, Examples 1-3 are an example to the last, and the structure obtained by changing suitably and changing the structure of Examples 1-3 within the range of the summary of this invention is also contained in this invention. Configurations obtained by appropriately combining the configurations of Examples 1 to 3 are also included in the present invention.

<Other examples>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

1: Display device 2: Receiver 3: Decoding unit 4: Image processing unit 4a: Base image processing unit 4b: Luminance ratio processing unit 4c: Color difference difference processing unit 5: Composition unit

Claims (15)

Acquisition means for acquiring first image data and difference data relating to a difference between the first image data and second image data having a data size larger than the first image data;
Processing means for performing data processing using the first image data;
Correction means for correcting the difference data based on the data processing;
Generating means for generating third image data from the first image data after the data processing and the corrected difference data;
An image processing apparatus comprising: The image processing apparatus according to claim 1, wherein the third image data is image data having the same data format as the second image data. The data processing is at least a part of image processing for changing a gradation value of image data,
The image processing apparatus according to claim 1, wherein the third image data substantially matches image data obtained by performing the image processing on the second image data. The number of bits of the first image data is a first number of bits,
The number of bits of the second image data and the number of bits of the third image data are a second number of bits larger than the first number of bits. An image processing apparatus according to 1. 5. The image processing apparatus according to claim 1, wherein the second image data is image data obtained by enlarging at least one of a luminance range and a color gamut of the first image data. The difference data includes luminance difference data relating to a difference between the luminance of the first image data and the luminance of the second image data, and a color relating to a difference between the color of the first image data and the color of the second image data. Including at least one of the differential data,
The image processing apparatus according to claim 1, wherein the correction unit corrects at least one of the luminance difference data and the color difference data. The said correction | amendment means correct | amends the said difference data using at least one of the parameter used by the said data processing, and the result of the said data processing, The any one of Claims 1-6 characterized by the above-mentioned. Image processing apparatus. The data processing includes at least one of a process of multiplying the set gain value by the gradation value of the image data and a process of adding the set offset value to the gradation value of the image data,
The image processing apparatus according to claim 1, wherein the correction unit corrects the difference data using at least one of the gain value and the offset value. The data processing includes processing for acquiring a feature amount of image data,
The image processing apparatus according to claim 1, wherein the correction unit corrects the difference data using at least the feature amount. The data processing includes a process of acquiring a feature amount of image data and changing a gradation value of the image data using a parameter corresponding to the acquired feature amount,
The image processing apparatus according to claim 1, wherein the correction unit corrects the difference data by using at least the parameter according to the feature amount. The data processing includes processing for detecting a predetermined image area,
The image processing apparatus according to claim 1, wherein the correction unit corrects the difference data by using at least a detection result of the predetermined image region. The image processing apparatus further comprises conversion means for converting the third image data into fourth image data having a data format corresponding to a display unit and having a data size smaller than that of the third image data. Item 12. The image processing device according to any one of Items 1 to 11. The data processing is at least a part of image processing for changing a gradation value of image data,
When the fifth image data is acquired without acquiring the first image data and the difference data by the acquisition unit,
The converting means converts the fifth image data into sixth image data having the data format corresponding to the display unit and having a data size smaller than the fifth image data,
The processing means performs the image processing on the sixth image data,
The image processing apparatus according to claim 12, wherein the correction unit and the generation unit omit processing. Obtaining difference data relating to a difference between the first image data and the first image data and second image data having a data size larger than the first image data;
Performing data processing using the first image data;
Correcting the difference data based on the data processing;
Generating third image data from the first image data after the data processing is performed and the corrected difference data;
An image processing method comprising:   The program for making a computer perform each step of the image processing method of Claim 14.

Images