JP2016046701A - Image processing apparatus, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique which allows a user to easily deciding whether a saturation region, existing in an image represented by the image data after image processing, is a saturation region generated by image processing or a saturation region that has been generated before image processing.SOLUTION: An image processing apparatus has image processing means for generating the image processing data by performing image processing of input image data, first detection means for detecting at least one of a black crushing area or a whiteout area, among the area of image represented by the input image data, as a saturation area, second detection means for detecting at least one of a black crushing area or a whiteout area, among the area of image represented by the image processing data, as a saturation area, and generation means for generating display image data representing the saturation region of an image based on the image processing data detected by only the second detection means, and the saturation region detected by both first and second detection means identifiably.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  In recent years, opportunities to handle image data (multi-bit image data) having a large number of bits of gradation values are increasing. For example, a digital camera that captures and records RAW image data having 12-bit or 16-bit gradation values has been proposed. Further, JPEG-HDR has been proposed as a format of image data having 32-bit gradation values (image data with a wide dynamic range; HDR image data) (Patent Document 1). JPEG-HDR includes HDR image data, JPEG image data having an 8-bit gradation value, and difference data used in enlargement processing for enlarging at least one of the dynamic range and color gamut of the JPEG image data. It is the expressed format.

However, there are still many opportunities to use devices that do not support multi-bit image data (image display devices, portable terminals, etc.). “A device that does not support multi-bit image data” can also be referred to as a “device that cannot process multi-bit image data”.
Therefore, in many cases, the captured multi-bit image data is used after being converted into image data (small bit image data) having a small number of bits of the gradation value. The multi-bit image data is converted into small-bit image data using, for example, correspondence data (function or table) indicating the correspondence between the gradation value before conversion and the gradation value after conversion. The corresponding data can also be said to be data that compresses a range of possible gradation values.
Then, when multi-bit image data is converted into low-bit image data, whiteout or blackout may occur in the image.
Note that image processing that causes whiteout or blackout is not limited to image processing that converts multi-bit image data into low-bit image data.

In recent years, digital cameras, JPEG viewers, and the like often use a function of highlighting an image area in which whiteout has occurred (whiteout area) and an image area in which blackout has occurred (blackout area).
Conventional techniques relating to such functions are disclosed in, for example, Patent Documents 2 and 3.
The imaging apparatus disclosed in Patent Literature 2 notifies the photographer of the detection results of the whiteout area and the blackout area. In addition, the photographing apparatus disclosed in Patent Document 2 displays adjustment information indicating adjustment necessary for reducing overexposure and blackout, operation information for prompting an adjustment operation for adjusting the exposure to an appropriate value, and the like.
The image display device disclosed in Patent Document 3 has a range of gradation values that can be taken by input image data when displaying input image data having a gradation value range wider than the range of gradation values that can be displayed. Each gradation value of the input image data is converted so as to be compressed into a displayable gradation value range. Then, the image display device disclosed in Patent Document 3 displays the converted image data. The image display device disclosed in Patent Document 3 displays a gradation value histogram of input image data, and displays a range of gradation values of the displayed image data on the histogram.

Here, the whiteout area and the blackout area are referred to as “saturated areas”.
As a saturated region existing in an image represented by image data after image processing (for example, small bit image data), a saturated region (first saturated region) generated by the image processing and before the image processing was performed And a saturation region (second saturation region). The first saturation region and the second saturation region are different from each other in the method of reducing whiteout or blackout. For example, whiteout or blackout in the first saturation region can be reduced by changing the image processing method. On the other hand, whiteout or blackout in the second saturation region cannot be reduced even if the image processing method is changed. In order to reduce overexposure and blackout in the second saturation region, it is necessary to perform re-photographing of image data before image processing.
Therefore, in order to efficiently reduce overexposure and blackout, the user can easily determine whether the saturation region present in the image represented by the image data after image processing is the first saturation region or the second saturation region. It is desirable to be able to grasp.

However, when the technique disclosed in Patent Document 2 is used, the first saturated region and the second saturated region are handled without being distinguished.
Therefore, even if the technique of Patent Document 2 is used, the user can determine whether the saturation region existing in the image represented by the image data after image processing is the first saturation region or the second saturation region. Can not. As a result, it is not possible to efficiently reduce whiteout or blackout.
In the technique disclosed in Patent Document 3, a histogram of the gradation values of the input image data is displayed. From the histogram, the user has a saturated region present in the image represented by the image data after image processing as the first. It cannot be determined whether the saturation region or the second saturation region.
Therefore, even if the technique of Patent Document 3 is used, the user can determine whether the saturated region existing in the image represented by the image data after image processing is the first saturated region or the second saturated region. Can not. As a result, it is not possible to efficiently reduce whiteout or blackout.

  In the technique disclosed in Patent Document 3, the range of gradation values of the displayed image data is further displayed. However, as the image processing, image processing (gamma conversion processing, clipping processing, etc.) in which the gradation value after the change changes nonlinearly with respect to the change in the gradation value before the conversion may be performed. For this reason, even if the range of the gradation value of the displayed image data is displayed, a histogram of the gradation value of the displayed image data may not be expressed. As a result, the user may not be able to accurately grasp information related to the gradation value of the displayed image data.

JP 2011-193511 A JP 2007-235421 A Japanese Patent Application Laid-Open No. 09-016795

  The present invention determines whether a saturated area existing in an image represented by image data after image processing is a saturated area generated by image processing or a saturated area generated before image processing is performed. It is an object of the present invention to provide a technique that can easily make a judgment.

The first aspect of the present invention is:
Image processing means for generating processed image data by performing image processing on the input image data;
First detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the input image data based on the input image data Means,
Second detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the processed image data based on the processed image data Means,
Based on the processed image data, the detection result of the first detection means, and the detection result of the second detection means, a saturated region detected only by the second detection means of an image based on the processed image data; Generating means for generating image data for display expressing the saturation region detected by both the first detection means and the second detection means in an identifiable manner;
An image processing apparatus comprising:

The second aspect of the present invention is:
First acquisition means for acquiring first image data;
Second acquisition means for acquiring difference data used in an enlargement process for enlarging at least one of the dynamic range and color gamut of the first image data;
Enlargement means for generating second image data by applying enlargement processing using the difference data to the first image data;
Based on the second image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the second image data. First detecting means for
Based on the first image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the first image data. A second detection means;
Based on the first image data, the detection result of the first detection means, and the detection result of the second detection means, the saturation detected only by the second detection means of the image based on the first image data Generating means for generating display image data expressing the area and the saturated area detected by both the first detecting means and the second detecting means in an identifiable manner;
An image processing apparatus comprising:

The third aspect of the present invention is:
An image processing step for generating processed image data by performing image processing on the input image data;
First detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the input image data based on the input image data Steps,
Second detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the processed image data based on the processed image data Steps,
Based on the processed image data, the detection result of the first detection step, and the detection result of the second detection step, a saturation region detected only in the second detection step of the image based on the processed image data; A generation step of generating display image data expressing the saturation region detected in both the first detection step and the second detection step in an identifiable manner;
An image processing method characterized by comprising:

The fourth aspect of the present invention is:
A first acquisition step of acquiring first image data;
A second acquisition step of acquiring difference data used in an enlargement process for enlarging at least one of the dynamic range and the color gamut of the first image data;
An enlargement step of generating second image data by applying an enlargement process using the difference data to the first image data;
Based on the second image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the second image data. First detecting step,
Based on the first image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the first image data. A second detection step;
Saturation detected only in the second detection step of the image based on the first image data based on the first image data, the detection result of the first detection means, and the detection result of the second detection means A generation step of generating display image data expressing the region and the saturated region detected in both the first detection step and the second detection step in an identifiable manner;
An image processing method characterized by comprising:

  According to a fifth aspect of the present invention, there is provided a program that causes a computer to execute each step of the above-described image processing method.

  According to the present invention, it is determined whether the saturated area existing in the image represented by the image data after the image processing is a saturated area generated by the image processing or a saturated area generated before the image processing is performed. The user can easily make a determination. As a result, whiteout and blackout can be efficiently reduced.

FIG. 1 is a block diagram illustrating an example of a functional configuration of an image display device according to a first embodiment. The figure which shows an example of the correspondence data which concern on Example 1. The figure which shows an example of the detection result of the 1st saturation area | region detection part which concerns on Example 1. FIG. The figure which shows an example of the detection result of the 2nd saturation area | region detection part which concerns on Example 1. FIG. The figure which shows an example of the process image and display image which concern on Example 1. FIG. 7 is a flowchart illustrating an example of the operation of the image display apparatus according to the first embodiment. FIG. 7 is a block diagram illustrating an example of a functional configuration of an image display apparatus according to a second embodiment. The figure which shows an example of the display image which concerns on Example 2. FIG. 10 is a flowchart illustrating an example of the operation of the image display apparatus according to the second embodiment. FIG. 9 is a block diagram illustrating an example of a functional configuration of an image display apparatus according to a third embodiment. FIG. 9 is a block diagram illustrating an example of a functional configuration of an image display apparatus according to a fourth embodiment.

<Example 1>
An image processing apparatus and an image processing method according to Embodiment 1 of the present invention will be described.
In this embodiment, an example in which the image processing apparatus is an image display apparatus will be described, but the image processing apparatus may be a separate apparatus from the image display apparatus.

FIG. 1 is a block diagram illustrating an example of a functional configuration of the image display apparatus 100 according to the present embodiment.
As shown in FIG. 1, the image display apparatus 100 includes an image input unit 101, a tone mapping unit 102, an image processing unit 103, a first saturated region detecting unit 104, a second saturated region detecting unit 105, a saturated region drawing unit 106, And an image display unit 107.
When the image processing apparatus is a separate apparatus from the image display apparatus, the image input unit 101, the tone mapping unit 102, the image processing unit 103, the first saturation region detection unit 104, and the second saturation region detection unit 105 are used. , And a saturated region rendering unit 106 is provided in the image processing apparatus. An image display unit 107 is provided in the image display device.

The image input unit 101 acquires image data (input image data) and outputs the input image data to the tone mapping unit 102 and the first saturation region detection unit 104.
In addition, the acquisition method of input image data is not specifically limited. For example, input image data may be acquired from the outside of the image processing apparatus, or input image data may be acquired from a storage unit (not shown) included in the image processing apparatus.
The input image data may be any image data. In this embodiment, an example will be described in which RAW image data having 16-bit gradation values (gradation values of 0 or more and 65535 or less) is acquired as input image data. As the gradation value, a color component value, a luminance value, or the like can be used. The color component value is, for example, an R value corresponding to red, a G value corresponding to green, a B value corresponding to blue, and the like. The pixel value of the image data may be a value obtained by combining a plurality of color component values, or may be a value obtained by combining a luminance value and a color difference value. The pixel value obtained by combining a plurality of color component values is, for example, an RGB value obtained by combining an R value, a G value, and a B value. The pixel value obtained by combining the luminance value and the color difference value is, for example, a YCbCr value obtained by combining the luminance value (Y value) and two color difference values (Cb value and Cr value).

The tone mapping unit 102 is an image processing unit that generates processed image data (first processed image data) by performing image processing on the input image data output from the image input unit 101. In the present embodiment, the tone mapping unit 102 uses the corresponding data (function or table) representing the correspondence between the gradation value of the input image data and the gradation value of the first processed image data to calculate the level of the input image data. Image processing for converting the tone value into the tone value of the first processed image data is executed. Such image processing is called “tone mapping processing”.
Here, the gradation values of the input image data may include values outside the range of gradation values that can be processed by the image processing unit 103 and the image display unit 107. By performing the tone mapping process, the range of gradation values that can be taken by the input image data can be matched (compressed) with the range of gradation values that can be processed by the image processing unit 103 and the image display unit 107.
The tone mapping unit 102 outputs the generated first processed image data to the image processing unit 103 and the second saturation region detection unit 105.

The first processed image data may be any image data. The range of gradation values that can be processed by the image processing unit 103 and the image display unit 107 is not particularly limited. In this embodiment, the gradation value that can be processed by the image processing unit 103 and the image display unit 107 is an 8-bit gradation value (a gradation value of 0 or more and 255 or less), and has an 8-bit gradation value. An example in which the first processed image data is generated will be described.
Note that the image processing executed by the tone mapping unit 102 is not limited to the above image processing. For example, the image processing executed by the tone mapping unit 102 may be multiplication processing for multiplying the pixel value of the input image data by a coefficient, addition processing for adding the correction value to the pixel value of the input image data, and the like.

2A to 2C are diagrams illustrating an example of correspondence data.
2 (a) to 2 (c) indicate input values (gradation values before conversion; gradation values that can be taken by input image data; input gradation values). The vertical axis in FIGS. 2A to 2C indicates an output value (a converted gradation value; a gradation value that can be taken by the first processed image data; a processed gradation value).
The tone mapping unit 102 can change the image processing method executed by the tone mapping unit 102. Specifically, the tone mapping unit 102 can change the correspondence relationship represented by the correspondence data to be used. The correspondence relationship represented by the correspondence data to be used is changed according to, for example, the user operation, the characteristics of the image (input image) represented by the input image data, the installation environment of the image display device 100, and the like. By changing the correspondence relationship represented by the correspondence data to be used, for example, the correspondence data in which the processing gradation value is assigned to some of the input gradation value ranges with priority over the other ranges is obtained. be able to.

  The method for changing the correspondence relationship represented by the correspondence data to be used is not particularly limited. For example, the correspondence relationship represented by the correspondence data may be changed by correcting the correspondence data. One of the plurality of corresponding data may be selected as the corresponding data to be used. In that case, the correspondence relationship represented by the correspondence data to be used can be changed by changing the correspondence data to be selected.

The correspondence data in FIG. 2A represents a correspondence relationship in which the processing gradation value increases linearly as the input gradation value increases.
In the corresponding data in FIG. 2B, the processing gradation value is assigned to the low gradation range of the input gradation value range in preference to the other ranges. The low gradation range is a range of input gradation values on the low gradation side. For example, the low gradation range is a range from the minimum input gradation value to the first gradation value. The first gradation value is an input gradation value that is larger than the minimum value of the input gradation values. By using the corresponding data in FIG. 2B, the gradation of the dark area (the area in which the gradation value of the input image data is the input gradation value in the low gradation range) becomes the gradation of the other area. First processed image data representing a higher image (processed image) can be obtained.
In the correspondence data in FIG. 2C, the processing gradation value is assigned to the high gradation range in the input gradation value range in preference to the other ranges. The high gradation range is a range of input gradation values on the high gradation side. For example, the high gradation range is a range from the maximum input gradation value to the second gradation value. The second gradation value is an input gradation value that is smaller than the maximum value of the input gradation values. By using the corresponding data in FIG. 2C, the gradation of the bright area (the area where the gradation value of the input image data is the input gradation value in the high gradation range) is the gradation of the other area. First processed image data representing a higher processed image can be obtained.
Note that the first gradation value and the second gradation value may be any value. At least one of the first gradation value and the second gradation value may be a value determined in advance by the manufacturer, or according to a user operation, a feature of the input image, an installation environment of the image display device 100, or the like. It may be determined.

Based on the input image data output from the image input unit 101, the first saturation region detection unit 104 detects at least one of a blackened region and a whiteout region as a saturation region from the region of the input image ( First detection process). The blackout area is an area where blackout occurs, and the whiteout area is an area where whiteout occurs. In the present embodiment, the first saturation region detection unit 104 detects each of the blackout region and the whiteout region as a saturation region.
Then, the first saturation region detection unit 104 outputs the detection result of the saturation region to the saturation region drawing unit 106. For example, the first saturation region detection unit 104 outputs region information representing the saturation region as a detection result of the saturation region. The area information is, for example, coordinate information that represents the coordinates of each pixel included in the saturated area.

Note that the first saturation region detection unit 104 may detect one of a blackened region and a whiteout region as a saturated region.
The first saturated region detection unit 104 outputs information including region information and type information indicating whether each saturated region is a blackout region or a whiteout region as a saturation region detection result. May be.

  In the present embodiment, the first saturation region detection unit 104 detects a region where the gradation value of the input image data is equal to or greater than the first threshold value as a whiteout region, and the gradation value of the input image data is equal to or less than the second threshold value. A certain area is detected as a blackout area. The second threshold is a value smaller than the first threshold. According to such a method, not only the pixel whose gradation value of the input image data matches the maximum value of the input gradation value, but also the gradation value of the input image data is a value in the vicinity of the maximum value of the input gradation value. This pixel can also be detected as a pixel in a whiteout region. In addition, not only pixels whose gradation value of the input image data matches the minimum value of the input gradation value, but also pixels whose gradation value of the input image data is a value near the minimum value of the input gradation value are black. It can be detected as a pixel in the collapsed area.

Note that the first threshold value and the second threshold value may be any value. At least one of the first threshold value and the second threshold value may be a value determined in advance by the manufacturer, or may be a user operation, a feature of the input image, a range of the input gradation value, a size of the input image, and the image display device 100. It may be determined according to the installation environment, etc. The first threshold value may be the maximum value of the input gradation value. The second threshold value may be a minimum value of the input gradation value.
Note that the saturation region detection method is not limited to the above method. For example, an area where the gradation value of the input image data is greater than or equal to the first threshold and the size is greater than or equal to the threshold may be detected as a whiteout area. Then, an area where the gradation value of the input image data is equal to or smaller than the second threshold and the size is equal to or larger than the threshold may be detected as a blackout area.
The size threshold may be any value. The size threshold value may be a value determined in advance by the manufacturer, or may depend on the user operation, the characteristics of the input image, the range of the input gradation value, the size of the input image, the installation environment of the image display device 100, and the like. May be determined.

FIG. 3 shows an example of the detection result of the first saturation region detection unit 104.
In FIG. 3, a region obtained by dividing a thick solid line by a thin solid line represents a pixel of the input image. The numerical value described in the pixel is a flag (first flag) indicating the detection result of the saturated region. A pixel in which the first flag “0” is described is a pixel that has not been detected as a pixel in the saturation region. A pixel in which the first flag “1” is described is a pixel detected as a pixel in a whiteout region. A pixel in which the first flag “2” is described is a pixel detected as a pixel in a blackout area.
For each pixel of the input image, the first saturation region detection unit 104 sets a first flag (0 to 2) corresponding to a comparison result between the pixel value of the pixel and a threshold value (first threshold value and second threshold value). . Then, the first saturation region detection unit 104 outputs the first flag of each pixel as a saturation region detection result.

Based on the first processed image data output from the tone mapping unit 102, the second saturated area detection unit 105 detects at least one of a blackout area and a whiteout area as a saturated area from the processed image area. (Second detection process). In the present embodiment, the second saturation area detection unit 105 detects each of the blackout area and the whiteout area as a saturation area.
Then, the second saturation region detection unit 105 outputs the detection result of the saturation region to the saturation region drawing unit 106. For example, the second saturation region detection unit 105 outputs region information representing the saturation region as a detection result of the saturation region. The area information is, for example, coordinate information that represents the coordinates of each pixel included in the saturated area.

Note that the second saturation region detection unit 105 may detect one of a blackened region and a whiteout region as a saturated region.
The second saturated region detection unit 104 outputs information including region information and type information indicating whether each saturated region is a blackout region or a whiteout region as a saturation region detection result. May be.

  In the present embodiment, the second saturation area detecting unit 105 detects an area where the gradation value of the first processed image data is equal to or greater than the third threshold as a whiteout area, and the gradation value of the first processed image data is the first value. An area having a threshold value of 4 or less is detected as a black area. The fourth threshold value is smaller than the third threshold value. According to such a method, not only the pixel whose gradation value of the first processed image data matches the maximum value of the processed gradation value, but also the gradation value of the first processed image data is the maximum value of the processed gradation value. A pixel having a value in the vicinity of can also be detected as a pixel in a whiteout region. Further, not only the pixel whose gradation value of the first processed image data matches the minimum value of the processed gradation value, but also the gradation value of the first processed image data is a value in the vicinity of the minimum value of the processed gradation value. Pixels can also be detected as pixels in the blackout area.

Since the range of the processing gradation value is different from the range of the input gradation value, a value different from the first threshold is used as the third threshold, and a value different from the second threshold is used as the fourth threshold.
Note that the third threshold value and the fourth threshold value may be any value. At least one of the third threshold value and the fourth threshold value may be a value determined in advance by the manufacturer, or may be a user operation, a characteristic of the processed image, a range of the processed gradation value, a size of the processed image, the image display device 100. It may be determined according to the installation environment, etc. The third threshold value may be the maximum processing gradation value. The fourth threshold value may be a minimum processing gradation value.
Note that the saturation region detection method is not limited to the above method. For example, an area in which the gradation value of the first processed image data is greater than or equal to the third threshold and the size is greater than or equal to the threshold may be detected as a whiteout area. And the area | region where the gradation value of 1st process image data is below a 4th threshold value, and a size is more than a threshold value may be detected as a black-out area | region.
The size threshold may be any value. The size threshold value may be a value determined in advance by the manufacturer, or may depend on the user operation, the characteristics of the processed image, the range of the processed gradation value, the size of the processed image, the installation environment of the image display device 100, and the like. May be determined.

FIG. 4 shows an example of the detection result of the second saturation region detection unit 105.
In FIG. 4, a region obtained by dividing a thick solid line by a thin solid line represents a pixel of the processed image. The numerical value described in the pixel is a flag (second flag) indicating the detection result of the saturated region. A pixel in which the second flag “0” is described is a pixel that has not been detected as a pixel in the saturation region. A pixel in which the second flag “1” is described is a pixel detected as a pixel in a whiteout region. A pixel in which the second flag “2” is described is a pixel detected as a pixel in a blackout area.
The second saturation region detection unit 105 sets, for each pixel of the processed image, a second flag (0 to 2) corresponding to a comparison result between the pixel value of the pixel and a threshold value (third threshold value and fourth threshold value). . Then, the second saturation region detection unit 105 outputs the second flag of each pixel as a detection result of the saturation region.

  Based on the first processed image data, the detection result of the first saturation region detection unit 104, and the detection result of the second saturation region detection unit 105 by the image processing unit 103 and the saturation region drawing unit 106, display image data ( (First display image data) is generated (first generation process). Specifically, image data expressing the first saturated region and the second saturated region of the image based on the first processed image data so as to be identifiable is generated as the display image data. The first saturation region is a saturation region detected only by the second saturation region detection unit 105, and the second saturation region is detected by both the first saturation region detection unit 104 and the second saturation region detection unit 105. It is a saturation region.

The image processing unit 103 generates second processed image data by performing predetermined image processing on the first processed image data output from the tone mapping unit 102. The predetermined image processing includes color conversion processing, luminance conversion processing, resolution conversion processing, frame rate conversion processing, and the like. The image processing unit 103 outputs the second processed image data to the saturated region drawing unit 106.
Note that the image display device 100 may not include the image processing unit 103.

  The saturated region drawing unit 106 performs processing on the second processed image data output from the image processing unit 103 based on the detection result of the first saturated region detection unit 104 and the detection result of the second saturation region detection unit 105. . Specifically, the second processed image data is processed so that the first saturated region and the second saturated region are expressed in different formats. Thereby, display image data in which the first saturation region and the second saturation region can be identified is generated. The saturated area drawing unit 106 outputs the display image data to the image display unit 107.

The first saturated region (saturated region detected only by the second saturated region detecting unit 105) is a saturated region generated by the image processing of the tone mapping unit 102.
The second saturation region (saturation region detected by both the first saturation region detection unit 104 and the second saturation region detection unit 105) is a saturation region that has occurred before the image processing of the tone mapping unit 102 is performed. is there.

FIG. 5A shows an example of an image (processed image) represented by the processed image data, and FIG. 5B shows an example of an image (display image) represented by the display image data.
A hatched region 601 (a hatched region) in FIG. 5B is a first saturation region. In the present embodiment, image processing for changing the color of the pixel in the first saturation region to yellow (first color) is performed on the second processed image data. Thereby, display image data that clearly represents the first saturation region is generated.
A filled area 602 (area filled in black) in FIG. 5B is a second saturated area. In the present embodiment, image processing for changing the color of the pixels in the second saturation region to white (second color) is performed on the second processed image data. Thereby, display image data that clearly represents the second saturation region is generated. Further, the first saturated region and the second saturated region are clearly distinguished from each other by making the color of the pixel in the first saturated region different from the color of the pixel in the second saturated region.
Whether the pixel of the second processed image data is a pixel in the first saturation region, a pixel in the second saturation region, or a pixel other than these is, for example, the first flag and the second flag of the pixel Can be determined by comparing.

Note that the first color and the second color may be any color as long as the first color is different from the second color.
Note that the processing applied to the second processed image data is not limited to the above processing.
For example, the second processed image data may be subjected to a process of changing the luminance instead of the pixel color. Specifically, the image processing for changing the luminance of the pixel in the first saturated region to the first value and changing the luminance of the pixel in the second saturated region to the second value (≠ first value) It may be applied to the two-process image data.
In the image represented by the second processed image data, the first frame line representing the outline of the first saturated region and the second frame line representing the contour of the second saturated region are drawn. Luminance and color may be changed. The first frame line is a frame line that is different from the second frame line in color, thickness, line type, and the like.
A graphic representing a graphic image in which a saturated region detected only by the second saturated region detector 105 and a saturated region detected by both the first saturated region detector 104 and the second saturated region detector 105 can be identified. Image data may be generated. Then, the graphic image data may be combined with the second processed image data so that the graphic image is superimposed on the image represented by the second processed image data (compositing process; superimposing process). Specifically, a superimposition process for superimposing the first graphic image on the first saturation region and superimposing the second graphic image on the second saturation region may be performed on the second processed image data. The first graphic image is, for example, an image that is different from the second graphic image in color, brightness, transparency, and display pattern. The above-described first frame line can be used as the first graphic image, and the above-described second frame line can be used as the second graphic image. Superimposition processing is performed on the second processed image data so that the first graphic image superimposed on the first saturation region and the second graphic image superimposed on the second saturation region are alternately displayed in a time division manner. May be. For example, a graphic image having the same shape and size as the saturated region is superimposed on the saturated region. If the display image data is generated so that the graphic image is blinked, the user can confirm the image represented by the second processed image data in the saturation region. Even when the display image data is generated so that the graphic image that passes through the background is superimposed, the user can confirm the image represented by the second processed image data in the saturation region.
Note that as the display image data (first display image data), image data that can further identify a saturated region that is a blackout region and a saturated region that is a whiteout region may be generated. Specifically, a first saturated area that is a blackout area, a first saturation area that is a whiteout area, a second saturation area that is a blackout area, and a second saturated area that is a whiteout area can be identified. First display image data may be generated.

The image display unit 107 displays an image based on the display image data output from the saturated region drawing unit 106 on the screen. As the image display unit 107, a liquid crystal display panel, an organic EL display panel, a plasma display panel, or the like can be used.
As described above, in the present embodiment, the saturated region drawing unit 106 generates display image data in which the first saturated region and the second saturated region are clearly distinguished and expressed. Therefore, in this embodiment, a display image (an image displayed on the screen) in which the first saturated region and the second saturated region are clearly distinguished and obtained is obtained. As a result, the user can easily determine whether the saturated region existing in the image represented by the second processed image data is the first saturated region or the second saturated region. The user can suitably adjust overexposure or blackout based on the determination result of whether the saturation region is the first saturation region or the second saturation region. As a result, whiteout and blackout can be efficiently reduced.

  If the saturated area that is a blackout area and the saturated area that is a whiteout area are expressed in different formats, what kind of area is the saturated area that exists in the image represented by the second processed image data. The user can easily and in detail determine whether there is any. Specifically, not only whether the saturated region is the first saturated region or the second saturated region, but also allows the user to easily determine whether the saturated region is a blacked-out region or a whiteout region. Can do. Then, the user can more suitably adjust the whiteout and the blackout based on the determination result of the saturated region. As a result, whiteout and blackout can be reduced more efficiently.

Next, the operation of the image display apparatus 100 will be described using the flowchart of FIG.
The flowchart in FIG. 6 is started, for example, in response to a user operation for displaying a saturated region.

  First, the tone mapping unit 102 determines whether or not the range of input gradation values includes gradation values that cannot be processed by the image processing unit 103 and the image display unit 107 (S701). When the range of the input tone values includes tone values that cannot be processed by the image processing unit 103 and the image display unit 107, tone mapping processing is performed, and the process proceeds to S702. If the input tone value range does not include tone values that cannot be processed by the image processing unit 103 and the image display unit 107, tone mapping processing is not performed, and the process proceeds to S705.

In S702, the first saturation region detection unit 104 detects a saturation region from the region of the input image.
Next, the second saturation region detection unit 105 detects a saturation region from the region of the processed image (S703).
Then, the saturated area drawing unit 106 generates display image data in which the first saturated area and the second saturated area are clearly distinguished based on the processing results of S702 and S703 (S704). Thereafter, the image display unit 107 displays a display image in which the first saturated region and the second saturated region are clearly distinguished and displayed on the screen.

When tone mapping processing is not performed, the same processed image data as input image data is output from the tone mapping unit 102.
In S705, the first saturated region detection unit 104 detects a saturation region from the input image region, and the second saturation region detection unit 105 detects the saturation region from the processed image region. At least one is done.
Then, the saturated region rendering unit 106 generates display image data in which the second saturated region is clearly expressed based on the processing result of S705 (S706). Thereafter, the image display unit 107 displays a display image in which the second saturation region is clearly expressed on the screen.

  As described above, according to the present embodiment, the first saturation region and the second saturation region are expressed in different formats. Thereby, the user can easily determine whether the saturation region existing in the image represented by the processed image data is the first saturation region or the second saturation region. Specifically, the user can easily (at a glance) discern whether the saturated region existing in the display image is the first saturated region or the second saturated region. As a result, it is possible to improve the convenience in reducing whiteout and blackout, and to efficiently reduce whiteout and blackout.

  Note that the image processing apparatus (image display apparatus) may further include a first generation unit. The first generation unit includes an image represented by the first display image data (first display image) and a histogram based on the input image data, the first processed image data, and the first display image data. The arranged second display image data is generated (second generation process). Specifically, the first generation unit is for second display representing an image in which a first display image, a histogram of gradation values of input image data, and a histogram of gradation values of processed image data are arranged. Generate image data. A display image in which the first display image and the histogram are arranged may be displayed on the screen. Accordingly, the user can easily grasp the gradation value histogram of the input image data and the gradation value histogram of the processed image data.

<Example 2>
An image processing apparatus and an image processing method according to Embodiment 2 of the present invention will be described.
FIG. 7 is a block diagram illustrating an example of a functional configuration of the image display apparatus 900 according to the present embodiment.
As illustrated in FIG. 7, the image display apparatus 900 further includes a saturation region analysis unit 901 and an additional information drawing unit 902 in addition to the functional units illustrated in FIG. 1.
In FIG. 7, the same functional parts as those in the first embodiment (FIG. 1) are denoted by the same reference numerals as those in the first embodiment, and the description thereof is omitted.

  The saturation region analysis unit 901 analyzes the detection result of the first saturation region detection unit 104 and the detection result of the second saturation region detection unit 105, thereby detecting a saturation region (first) detected only by the second saturation region detection unit 105. Information representing one saturation region) is acquired as an analysis result. Then, the saturation region analysis unit 901 outputs the analysis result to the additional information drawing unit 902.

  The additional information drawing unit 902 generates additional information based on the analysis result of the saturation region analysis unit 901. Then, the additional information drawing unit 902 is an image representing an image in which an image (first display image) represented by the first display image data generated by the saturated region drawing unit 106 and an image of the additional information are arranged. Generate data. The additional information drawing unit 902 outputs the generated image data to the image display unit 107 as display image data.

Specifically, the additional information drawing unit 902 acquires the input image data from the image input unit 101, acquires the analysis result of the saturation region from the saturation region analysis unit 901, and the first display image from the saturation region drawing unit 106. Get the data.
First, the additional information drawing unit 902 extracts image data in the first saturation region from the input image data based on the analysis result of the saturation region analysis unit 901 (extraction process).
Next, the additional information rendering unit 902 performs image processing different from the image processing executed by the tone mapping unit 102 on the extracted image data, thereby displaying a partially processed image representing an image in which blackout and whiteout are not generated. Data is generated (third generation process).
Then, the additional information drawing unit 902 is arranged with the first display image and the image (partial processing image) represented by the partial processing image data based on the first display image data and the partial processing image data. Image data representing an image is generated (fourth generation process).
Thereafter, the additional information drawing unit 902 outputs the image data generated by the third generation process as display image data.

In the second generation process, for example, the maximum value and the minimum value of the gradation values of the image data extracted by the extraction process are acquired. Corresponding data in which the processing gradation value linearly changes from the minimum value of the processing gradation value to the maximum value of the processing gradation value with respect to the change of the input gradation value from the acquired minimum value to the acquired maximum value. Is generated. Thereafter, the image data extracted by the extraction process is converted into the partially processed image data by image processing (tone mapping process) using the generated correspondence data.
Note that the extraction process, the second generation process, and the third generation process may be executed by different functional units. One of the extraction process, the second generation process, and the third generation process may be executed by a functional unit different from the functional units that execute the remaining two processes.

FIG. 8 shows an example of a display image according to the present embodiment.
In the example of FIG. 8, the first display image (the image of FIG. 5B) is arranged on the left side, and the partial processing image 801 is arranged on the right side.

Next, the operation of the image display apparatus 900 will be described using the flowchart of FIG.
The flowchart of FIG. 9 is started in response to a user operation for displaying additional information, for example.
Since the operation of the image display apparatus 900 until the generation of the first display image data is the same as that of the first embodiment, the description thereof is omitted.

First, the saturation region analysis unit 901 obtains information representing the first saturation region as an analysis result by analyzing the detection results shown in FIGS. 3 and 4 (S1001).
Next, the additional information drawing unit 902 extracts image data in the first saturation region from the input image data based on the analysis result of S1001, and generates partial processing image data based on the extracted image data. Corresponding data to be used is generated (S1002).
Then, the additional information drawing unit 902 performs image processing using the corresponding data generated in S1002 on the image data extracted in S1002, thereby generating partial processed image data (S1003).
Next, the additional information drawing unit 902 generates display image data representing an image in which the first display image and the partially processed image are arranged based on the first display image data and the partially processed image data. (S1004). Thereafter, the image display unit 107 displays a display image in which the first display image and the partial processing image are arranged on the screen.

As described above, according to the present embodiment, image data representing an image in which the first display image and the partially processed image are arranged is generated. Then, a display image in which the first display image and the partially processed image are arranged is displayed on the screen. The partially processed image corresponds to one of the image processing results of the tone mapping unit 102. Therefore, according to the present embodiment, the user can easily grasp one of the image processing results of the tone mapping unit 102. As a result, it is possible to improve convenience when the user determines whether to change the image processing method executed by the tone mapping unit 102.
Note that the additional information is not limited to the partially processed image data. For example, the additional information may be a message notified to the user.

Note that the image processing apparatus (image display apparatus) may further include a first notification unit. When the difference between the total area of the saturated regions detected by the first saturated region detector and the total area of the saturated regions detected by the second saturated region detector is greater than or equal to the fifth threshold value In addition, user notification is performed. The user notification notifies the user that the image processing method executed by the tone mapping unit may not be suitable. When there is a large difference between the total area of the saturated regions detected by the first saturated region detector and the total area of the saturated regions detected by the second saturated region detector, the image processing method of the tone mapping unit is preferable. It is likely not. By providing the first notification unit, when there is a high possibility that the image processing method of the tone mapping unit is not suitable, the user may be notified that the image processing method of the tone mapping unit may not be suitable. it can. As a result, the user can be prompted to change the image processing method of the tone mapping unit.
Note that the fifth threshold value may or may not be a value determined in advance by the manufacturer. The fifth threshold is a user operation, a feature of the input image, a feature of the processed image, a range of input gradation values, a range of processed gradation values, a size of the input image, a size of the processed image, an installation environment of the image display device 100, It may be determined according to the above.

Note that the image processing apparatus (image display apparatus) may further include a second notification unit. A 2nd notification part performs a user notification, when the distance between the position of a 1st saturation area | region and the center position of a process image is below a 6th threshold value. The user notification notifies the user that the image processing method executed by the tone mapping unit may not be suitable. When the position of the first saturation region is close to the center position of the processed image (display image; screen), there is a high possibility that the image processing method of the tone mapping unit is not suitable. By providing the second notification unit, when there is a high possibility that the image processing method of the tone mapping unit is not suitable, it is possible to notify the user that the image processing method of the tone mapping unit may not be suitable. it can. As a result, the user can be prompted to change the image processing method of the tone mapping unit.
Note that the sixth threshold value may or may not be a value predetermined by the manufacturer. The sixth threshold value is a user operation, an input image feature, a processed image feature, an input tone value range, a processed tone value range, an input image size, a processed image size, an installation environment of the image display device 100, It may be determined according to the above.

Note that the image processing apparatus (image display apparatus) may further include an object detection unit and a third notification unit. Based on one of the input image data and the processed image data, the object detection unit detects an area where a predetermined object is present as an object area from among the areas of the image represented by the image data (first area). 3 detection processing). The third notification unit performs user notification when the first saturation region is a region including the object region. The user notification notifies the user that the image processing method executed by the tone mapping unit may not be suitable. The “area including the object area” may be an area including a part of the object area or an area including the entire object area. The predetermined object is, for example, a human face. When the area of the object important for the user is the first saturation area, it is highly likely that the image processing method of the tone mapping unit is not suitable. By providing the object detection unit and the third notification unit, when there is a high possibility that the image processing method of the tone mapping unit is not suitable, it is possible to inform the user that the image processing method of the tone mapping unit may not be suitable. You can be notified. As a result, the user can be prompted to change the image processing method of the tone mapping unit.
The method for detecting the object area is not particularly limited. For example, the object region is detected by performing a pattern matching process using a detection pattern prepared in advance.

If there is a high possibility that the image processing method of the tone mapping unit is not suitable, the tone mapping unit automatically changes the image processing method of the tone mapping unit to automatically regenerate the processed image data. Also good.
For example, the tone mapping unit may change the image processing method executed by the tone mapping unit and generate the processed image data again when the difference in the total area is greater than or equal to the fifth threshold.
Further, the tone mapping unit changes the image processing method executed by the mapping unit when the distance between the position of the first saturation region and the center position of the processed image is equal to or smaller than the sixth threshold, and the processed image Data may be generated again.
The image processing apparatus may further include the object detection unit. Then, when the first saturated region is a region including the object region, the tone mapping unit may change the image processing method executed by the mapping unit and generate the processed image data again.
The method for changing the image processing method is not particularly limited. For example, the tone mapping unit changes the image processing method executed by the mapping unit based on the representative value of the gradation value of the input image data in the first saturation region, and generates the processed image data again. The representative value is an average value, a mode value, an intermediate value, a maximum value, a minimum value, etc. of the gradation values.

<Example 3>
An image processing apparatus and an image processing method according to Embodiment 3 of the present invention will be described.
In this embodiment, an example will be described in which first image data and difference data used in enlargement processing for enlarging at least one of the dynamic range and color gamut of the first image data are input to the image processing apparatus. The first image data and the difference data are, for example, JPEG-HDR data.
The first image data and the difference data may be input individually, or one image file including the first image data and the difference data may be input.

In this embodiment, the first image data is described as “base image data”.
The base image data is image data obtained by down-sampling (gradation compression) the second image data. The base image data is, for example, JPEG standard image data. The base image data is, for example, image data in which each of the Y value, Cb value, and Cr value is a gradation value equivalent to 8 bits. The second image data can be generated (restored) by performing an enlargement process using the difference data on the base image data. The pixel value of the base image data may be a YCbCr value or an RGB value.
In this embodiment, the second image data is described as “HDR image data”.
The HDR image data is, for example, image data in which each of the Y value, the Cb value, and the Cr value is a gradation value corresponding to 32 bits. Note that the pixel value of the HDR image data may be a YCbCr value or an RGB value.
Image data having a small number of gradation values can be referred to as “small bit image data”, and image data having a large number of gradation values can be referred to as “multi-bit image data”. Therefore, the base image data can be called “small bit image data”, and the HDR image data can also be called “multi-bit image data”.

In JPEG-HDR, the base image data is JPEG standard image data. Therefore, there are cases where JPEG-HDR data can be suitably processed even by an apparatus that does not support HDR image data. Specifically, the base image data can be suitably processed by an apparatus that supports conventional JPEG image data. “Apparatus that does not support image data” can also be referred to as “apparatus that cannot process image data suitably”.
The JPEG-HDR data includes base image data and difference data. Therefore, an apparatus that supports HDR image data can generate HDR image data from JPEG-HDR data (base image data and difference data), and can appropriately process the HDR image data.

The difference data includes, for example, luminance difference data, color difference data, or both.
The luminance difference data is data used in the luminance range expansion process for expanding the dynamic range of the image data, and is data representing the difference between the luminance values of the HDR image data and the base image data. For example, the luminance difference data is a ratio between the luminance value (gradation value) of the base image data and the luminance value (gradation value) of the HDR image data in pixel units (or in units of regions including a predetermined number of pixels). This is luminance ratio data representing the luminance ratio. That is, the luminance difference data is the ratio of the luminance value (gradation value) of the HDR image data to the luminance value (gradation value) of the base image data or the reciprocal thereof in pixel units (or an area unit composed of a predetermined number of pixels). Is luminance ratio data representing However, the luminance difference data is obtained by subtracting one from the luminance value (gradation value) of the base image data and the luminance value (gradation value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). Data representing a luminance difference value which is a subtracted difference value may be used. Further, the luminance difference data may be luminance conversion table data representing a correspondence relationship between the input luminance value and the output luminance value in the luminance range expansion process. Note that the luminance range expansion processing can also be said to be processing for reproducing luminance that cannot be expressed in the base image data.

  The color difference data is data used in the color gamut expansion process for expanding the color gamut of the image data, and is data representing the color difference between the HDR image data and the base image data. For example, the color difference data is a color difference value (Cb value, Cr value) of the base image data and a color difference value (Cb value, Cr value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). This is data representing a color difference value which is a difference value obtained by subtracting the other from one. However, the color difference data is a color difference value (Cb value, Cr value) of the base image data and a color difference value (Cb value, Cr value) of the HDR image data in units of pixels (or an area unit composed of a predetermined number of pixels). Color difference ratio data representing a color ratio that is a ratio of Further, the color difference value and the color ratio may be values calculated using R value, G value, and B value instead of the color difference value. Note that the color gamut expansion processing can be said to be processing for reproducing colors that cannot be expressed by the base image data.

FIG. 10 is a block diagram illustrating an example of a functional configuration of the image display apparatus 1100 according to the present embodiment.
The image display device 1100 includes a base image acquisition unit 1101, a difference data acquisition unit 1102, an HDR image generation unit 1103, an image processing unit 103, a first saturation region detection unit 104, a second saturation region detection unit 105, and a saturation region drawing unit 106. , And an image display unit 107.
In FIG. 10, the same reference numerals as those in the first embodiment are given to the same functional units as those in the first embodiment (FIG. 1).

The base image acquisition unit 1101 acquires base image data (first image data) (first acquisition process). Then, the base image acquisition unit 1101 outputs the acquired base image data to the image processing unit 103, the second saturation region detection unit 105, and the HDR image generation unit 1103.
The difference data acquisition unit 1102 acquires difference data (second acquisition process). Then, the difference data acquisition unit 1102 outputs the acquired difference data to the HDR image generation unit 1103.
In addition, the acquisition method of base image data and difference data is not specifically limited. For example, at least one of the base image data and the difference data may be acquired from the outside of the image processing apparatus, or may be acquired from a storage unit (not shown) included in the image processing apparatus.
The HDR image generation unit 1103 generates HDR image data by performing enlargement processing using the difference data output from the difference data acquisition unit 1102 on the base image data output from the base image acquisition unit 1101. Then, the HDR image generation unit 1103 outputs the generated HDR image data to the first saturation region detection unit 104.

Based on the HDR image data output from the HDR image generation unit 1103, the first saturation region detection unit 104 determines whether there is a whiteout region or a blackout region from the region of the image (HDR image) represented by the HDR image data. At least one is detected as a saturation region (first detection process). Then, the first saturation region detection unit 104 outputs the detection result of the saturation region to the saturation region drawing unit 106.
Based on the base image data output from the base image acquisition unit 1101, the second saturation area detection unit 105 detects a whiteout area and a blackout area from the area of the image (base image) represented by the base image data. At least one is detected as a saturation region (second detection process). Then, the second saturation region detection unit 105 outputs the detection result of the saturation region to the saturation region drawing unit 106.

Display image data is generated by the image processing unit 103 and the saturation region drawing unit 106 based on the base image data, the detection result of the first saturation region detection unit 104, and the detection result of the second saturation region detection unit 105. The Specifically, image data expressing the first saturated region and the second saturated region of the image based on the base image data so as to be distinguishable is generated as the display image data. The first saturation region is a saturation region detected only by the second saturation region detection unit 105, and the second saturation region is detected by both the first saturation region detection unit 104 and the second saturation region detection unit 105. It is a saturation region.
The base image data is, for example, image data generated by performing a reduction process for reducing at least one of the dynamic range and the color gamut on the HDR image data.
The first saturation region is, for example, a saturation region generated by converting HDR image data into base image data.
The second saturation region is, for example, a saturation region that has occurred before HDR image data is converted to base image data.

The image processing unit 103 generates processed image data by performing predetermined image processing on the base image data. Then, the image processing unit 103 outputs the processed image data to the saturated region drawing unit 106.
The saturated region drawing unit 106 performs processing on the processed image data output from the image processing unit 103 based on the detection result of the first saturated region detection unit 104 and the detection result of the second saturation region detection unit 105. Specifically, the processed image data is processed so that the first saturated region and the second saturated region are expressed in different formats. Thereby, display image data in which the first saturation region and the second saturation region can be identified is generated. The saturated area drawing unit 106 outputs the display image data to the image display unit 107.
The image display unit 107 displays an image based on the display image data output from the saturated region drawing unit 106 on the screen.

  As described above, according to the present embodiment, when the base image data and the difference data are acquired, the first saturated region and the second saturated region are expressed in different formats. Accordingly, the user can easily determine whether the saturation region existing in the image represented by the base image data is the first saturation region or the second saturation region. Specifically, the user can easily (at a glance) discern whether the saturated region existing in the display image is the first saturated region or the second saturated region. As a result, it is possible to improve the convenience in reducing whiteout and blackout, and to efficiently reduce whiteout and blackout.

<Example 4>
An image processing apparatus and an image processing method according to Embodiment 4 of the present invention will be described.
In the present embodiment, an example will be described in which a plurality of photographed image data generated by photographing the same photographing range a plurality of times under different photographing conditions is used as input image data. Shooting the same shooting range multiple times under different shooting conditions is called “bracket shooting” or the like. In this embodiment, the captured image data generated by bracket imaging is described as “bracket image data”. The shooting conditions include exposure, ISO sensitivity, and the like.

FIG. 11 is a block diagram illustrating an example of a functional configuration of the image display apparatus 1200 according to the present embodiment.
The image display device 1200 includes a bracket image storage unit 1201, a 1 image generation unit 1202, a bracket image acquisition unit 1203, an image processing unit 103, a first saturation region detection unit 1204, a second saturation region detection unit 105, and a saturation region drawing unit 106. , And an image display unit 107.
In FIG. 11, the same reference numerals as those in the first embodiment are assigned to the same functional units as those in the first embodiment (FIG. 1).

The bracket image storage unit 1201 acquires a plurality of bracket image data from an external device (digital camera, storage device, etc.), and stores the acquired plurality of bracket image data.
Note that the bracket image storage unit 1201 may be a separate device from the image display device 1200.

The one image generation unit 1202 executes image processing for generating one image data (processed image data; first processed image data) from the plurality of bracket image data stored in the bracket image storage unit 1201. Then, the one image generation unit 1202 outputs the generated first processed image data to the image processing unit 103 and the second saturation region detection unit 105.

  In this embodiment, the one image generation unit 1202 reads two or more bracket image data from the bracket image storage unit 1201. Then, the one image generation unit 1202 generates first processed image data by combining the two or more bracket image data according to a predetermined algorithm.

Specifically, one of the two or more bracket image data is selected as reference image data, and a saturated region of an image (reference image) represented by the reference image data is detected. The remaining bracket image data is used as reference image data.
Next, in the image (reference image) represented by the reference image data, it is determined whether or not the saturated region of the standard image is a saturated region. If there are a plurality of reference image data, the above-described determination is performed for each reference image data.
Then, when there is reference image data in which the saturation region of the standard image is not the saturation region, image data in the saturation region of the standard image is extracted from the reference image data. Thereafter, the reference image data in the saturation region of the reference image is replaced with the extracted image data. That is, the reference image in the saturation region (image that is not blacked out or whiteout) is trimmed, and the reference image in the saturation region (image that is blackout or whiteout) is replaced with the trimmed image.
Through the above processing, first processed image data is generated.

The one image generation unit 1202 can change the image processing method executed by the one image generation unit 1202. For example, the one-image generating unit 1202 can change the bracket image data used for generating the first processed image data, the number of bracket image data used for generating the first processed image data, and the like.
The method for generating the first processed image data is not particularly limited. For example, one of the plurality of bracket image data may be selected as the first processed image data. For each pixel, a representative value of the pixel values of the two or more bracket image data may be determined as the pixel value of the processed image data. The representative value is an average value, a mode value, an intermediate value, a maximum value, a minimum value, or the like.

  The bracket image acquisition unit 1203 reads all of the plurality of bracket image data from the bracket image storage unit 1201 and outputs the read plurality of bracket image data to the first saturation region detection unit 1204.

  For each bracket image data, the first saturation region detection unit 1204 includes a region where black crushing occurs, a region where whiteout occurs, among regions of an image (bracket image) represented by the bracket image data, At least one of them is detected. The first saturation region detection unit 1204 includes a region where blackout occurs in all of the plurality of bracket images represented by the plurality of bracket image data, a region where whiteout occurs in all of the plurality of bracket images, Is detected as a saturation region. Thereafter, the first saturation region detection unit 1204 outputs the detection result of the saturation region to the saturation region drawing unit 106.

  Based on the first processed image data output from the one image generating unit 1202, the second saturated area detecting unit 105 determines whether a whiteout area and a blackout area are included in the image area represented by the first processed image data. At least one is detected as a saturation region. Then, the second saturation region detection unit 105 outputs the detection result of the saturation region to the saturation region drawing unit 106.

Based on the first processed image data, the detection result of the first saturation region detection unit 104, and the detection result of the second saturation region detection by the image processing unit 103 and the saturation region drawing unit 106, display image data (first Display image data) is generated. Specifically, image data expressing the first saturated region and the second saturated region of the image based on the first processed image data so as to be identifiable is generated as the display image data. The first saturation region is a saturation region detected only by the second saturation region detection unit 105, and the second saturation region is detected by both the first saturation region detection unit 104 and the second saturation region detection unit 105. It is a saturation region.
The first saturation region is a saturation region generated by image processing that generates first processed image data from a plurality of bracket image data.
The second saturated region is a saturated region that has occurred before the image processing is performed, and is a saturated region that has occurred in all of the plurality of bracket image data.

The image processing unit 103 generates second processed image data by performing predetermined image processing on the first processed image data. Then, the image processing unit 103 outputs the second processed image data to the saturated region drawing unit 106.
The saturated region drawing unit 106 performs processing on the second processed image data output from the image processing unit 103 based on the detection result of the first saturated region detection unit 104 and the detection result of the second saturation region detection unit 105. . Specifically, the second processed image data is processed so that the first saturated region and the second saturated region are expressed in different formats. Thereby, display image data in which the first saturation region and the second saturation region can be identified is generated. The saturated area drawing unit 106 outputs the display image data to the image display unit 107.
The image display unit 107 displays an image based on the display image data output from the saturated region drawing unit 106 on the screen.

  As described above, according to the present embodiment, when a plurality of bracket image data is acquired, the first saturated region and the second saturated region are expressed in different formats. Accordingly, the user can easily determine whether the saturated region existing in the image represented by the first processed image data is the first saturated region or the second saturated region. Specifically, the user can easily (at a glance) discern whether the saturated region existing in the display image is the first saturated region or the second saturated region. As a result, it is possible to improve the convenience in reducing whiteout and blackout, and to efficiently reduce whiteout and blackout.

  As mentioned above, although preferable Example 1-4 of this invention was described, this invention is not limited to Examples 1-4, A various deformation | transformation and change are possible within the range of the summary. For example, the present invention may be modified or changed as appropriate according to the configuration of the apparatus to which the present invention is applied and various conditions. Moreover, it is also possible to combine suitably the structure of Examples 1-4.

<Other examples>
The present invention can also be implemented by a system (or a device such as a CPU or MPU) of a system or apparatus that implements the functions of the above-described embodiments by reading and executing a program recorded in a storage device. The present invention can also be implemented by a method comprising steps executed by a computer of a system or apparatus that implements the functions of the above-described embodiments by reading and executing a program recorded in a storage device, for example. . For this purpose, the program is stored in the computer from, for example, various types of recording media that can serve as the storage device (ie, computer-readable recording media that holds data non-temporarily). Provided to. Therefore, the computer (including devices such as CPU and MPU), the method, the program (including program code and program product), and the computer-readable recording medium that holds the program non-temporarily are all present. It is included in the category of the invention.

100, 900, 1100, 1200: Image display device 102: Tone mapping unit 103: Image processing unit 104, 1204: First saturation region detection unit 105: Second saturation region detection unit 106: Saturation region rendering unit 1101: Base image acquisition Unit 1102: difference data acquisition unit 1103: HDR image generation unit 1202: 1 image generation unit

Claims (18)

Image processing means for generating processed image data by performing image processing on the input image data;
First detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the input image data based on the input image data Means,
Second detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the processed image data based on the processed image data Means,
Based on the processed image data, the detection result of the first detection means, and the detection result of the second detection means, a saturated region detected only by the second detection means of an image based on the processed image data; First generation means for generating first display image data expressing the saturation region detected by both the first detection means and the second detection means in an identifiable manner;
An image processing apparatus comprising: The image processing uses the correspondence data representing the correspondence between the gradation value of the input image data and the gradation value of the processed image data, and converts the gradation value of the input image data to the gradation of the processed image data. The image processing apparatus according to claim 1, wherein the image processing is converted into a value. The input image data is a plurality of photographed image data generated by photographing the same photographing range multiple times under mutually different photographing conditions,
The image processing is image processing for generating one processed image data from the plurality of captured image data,
The first detection means has a blackout region in all of the plurality of images represented by the plurality of photographed image data and a whiteout in all of the plurality of images represented by the plurality of photographed image data. The image processing apparatus according to claim 1, wherein at least one of the region is detected as a saturated region. The first detecting means detects an area where the gradation value is equal to or greater than the first threshold as an area where whiteout occurs, and an area where the gradation value is equal to or less than the second threshold smaller than the first threshold is black. Detected as an area where crushing occurs,
The second detection means detects an area where the gradation value is equal to or greater than the third threshold as an area where whiteout occurs, and an area where the gradation value is equal to or less than the fourth threshold smaller than the third threshold is black. The image processing device according to claim 1, wherein the image processing device is detected as a region where crushing occurs. The first detection means detects each of a blackened area and a whiteout area as a saturated area,
The second detection means detects each of a blackened area and a whiteout area as a saturated area,
The first generation means may include, as the first display image data, image data that can further identify a saturated area that is a blackened area and a saturated area that is a whiteout area. The image processing apparatus according to claim 1, wherein the image processing apparatus generates the image processing apparatus. Based on the input image data, the processed image data, and the first display image data, an image represented by the first display image data, a gradation value histogram of the input image data, and the processed image The image according to any one of claims 1 to 5, further comprising second generation means for generating second display image data representing an image in which a histogram of gradation values of data is arranged. Processing equipment. Extraction means for extracting image data in the saturation region detected only by the second detection means from the input image data;
The image data extracted by the extracting means is subjected to image processing different from the image processing executed by the image processing means, thereby generating partially processed image data representing an image in which blackout and whiteout are not generated. 3 generating means;
Based on the first display image data and the partial processed image data, a first image representing an image in which an image represented by the first display image data and an image represented by the partial processed image data are arranged. Fourth generation means for generating 3 display image data;
The image processing apparatus according to claim 1, further comprising: When the difference between the total area of the saturated region detected by the second detecting unit and the total area of the saturated region detected by the first detecting unit is equal to or greater than a fifth threshold, the image processing unit executes The image processing apparatus according to claim 1, further comprising first notification means for notifying a user that the method of image processing performed may not be suitable. When the distance between the position of the saturated region detected only by the second detection means and the center position of the image represented by the processed image data is equal to or smaller than a sixth threshold value, the processing is executed by the image processing means The image processing apparatus according to claim 1, further comprising a second notification unit that notifies a user that the image processing method may not be suitable. Third detection means for detecting, as an object area, an area in which a predetermined object exists from an image area represented by the image data based on one of the input image data and the processed image data; ,
When the saturated area detected only by the second detection means is an area including the object area, the user is notified that the image processing method executed by the image processing means may not be suitable. A third notification means;
The image processing apparatus according to claim 1, further comprising: When the difference between the total area of the saturated region detected by the second detecting unit and the total area of the saturated region detected by the first detecting unit is equal to or greater than a fifth threshold, The image processing apparatus according to claim 1, wherein the image processing method executed by the image processing unit is changed to generate the processed image data again.   When the distance between the position of the saturated region detected only by the second detection unit and the center position of the image represented by the processed image data is equal to or smaller than a sixth threshold, the image processing unit The image processing apparatus according to claim 1, wherein the processed image data is generated again by changing a method of the image processing executed by a processing unit. Third detection means for detecting, as an object area, an area in which a predetermined object is present from an image area represented by the image data based on one of the input image data and the processed image data; Further comprising
The image processing means changes the image processing method executed by the image processing means when the saturated area detected only by the second detection means is an area including the object area, and the processed image The image processing apparatus according to claim 1, wherein data is generated again. The image processing means changes the image processing method executed by the image processing means based on a representative value of gradation values of the input image data in a saturation region detected only by the second detection means, The image processing apparatus according to claim 11, wherein the processed image data is generated again. First acquisition means for acquiring first image data;
Second acquisition means for acquiring difference data used in an enlargement process for enlarging at least one of the dynamic range and color gamut of the first image data;
Enlargement means for generating second image data by applying enlargement processing using the difference data to the first image data;
Based on the second image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the second image data. First detecting means for
Based on the first image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the first image data. A second detection means;
Based on the first image data, the detection result of the first detection means, and the detection result of the second detection means, the saturation detected only by the second detection means of the image based on the first image data Generating means for generating display image data expressing the area and the saturated area detected by both the first detecting means and the second detecting means in an identifiable manner;
An image processing apparatus comprising: An image processing step for generating processed image data by performing image processing on the input image data;
First detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the input image data based on the input image data Steps,
Second detection for detecting, as a saturated region, at least one of a blackened region and a whiteout region from an image region represented by the processed image data based on the processed image data Steps,
Based on the processed image data, the detection result of the first detection step, and the detection result of the second detection step, a saturation region detected only in the second detection step of the image based on the processed image data; A generation step of generating display image data expressing the saturation region detected in both the first detection step and the second detection step in an identifiable manner;
An image processing method comprising: A first acquisition step of acquiring first image data;
A second acquisition step of acquiring difference data used in an enlargement process for enlarging at least one of the dynamic range and the color gamut of the first image data;
An enlargement step of generating second image data by applying an enlargement process using the difference data to the first image data;
Based on the second image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the second image data. First detecting step,
Based on the first image data, at least one of a blackened area and a whiteout area is detected as a saturated area from the image area represented by the first image data. A second detection step;
Saturation detected only in the second detection step of the image based on the first image data based on the first image data, the detection result of the first detection means, and the detection result of the second detection means A generation step of generating display image data expressing the region and the saturated region detected in both the first detection step and the second detection step in an identifiable manner;
An image processing method comprising:   A program for causing a computer to execute each step of the image processing method according to claim 16 or 17.

Images