JP2016092576A - Image processing apparatus, image processing method and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a technique capable of acquiring image data having appropriate luminance.SOLUTION: An image processing apparatus includes: acquisition means for acquiring, from input image data, a luminance histogram which represents the total number of pixels, having a luminance value subordinate to a category as a frequency, on the basis of each category which is a luminance range including one or more luminance values; detection means for detecting, from the luminance histogram, a cluster which is a brightness range composed of one or more categories whose frequency is a first threshold or greater; determination means for determining whether or not the cluster detected by the detection means satisfies a predetermined condition; and correction means for correcting the input image data on the basis of the determination result of the determination means.SELECTED DRAWING: Figure 1

Description

  The present invention relates to an image processing apparatus, an image processing method, and a program.

  The luminance dynamic range that can be displayed by the conventional display device is not so wide and is narrower than the luminance dynamic range that can be recognized by humans. For this reason, the dynamic range (dynamic range of image data) defined by an image format generally used has not been wide.

  In recent years, display devices capable of displaying images having a wider dynamic range than before have been proposed. In addition, an image format in which a wider dynamic range than the conventional one has been proposed (Patent Document 1).

  Such a dynamic range wider than the conventional one is called “HDR (High Dynamic Range)”. An image having HDR is called “HDR image”, and image data representing the HDR image is called “HDR image data”. A display device capable of displaying an HDR image can simultaneously display both a dark image darker than the conventional image and a bright image brighter than the conventional image on the screen. Therefore, if a display device capable of displaying an HDR image is used, a display image (an image displayed on the screen) having high reality, gloss, brightness, power, and the like can be obtained. For example, as a display image of an object, a display image having a luminance close to the luminance perceived when a human views the object can be obtained. Similarly, HDR image data can express an image with high reality, glossiness, brightness, and the like.

  As described above, if a display device capable of displaying an HDR image is used, a display image with high reality, gloss, brightness, and the like can be obtained. However, when an HDR image is displayed, there is a possibility that a visual disturbance may occur such that the eyes become tired or feel unwell due to the effect of human visual adaptation. For example, a blinking scene in which the luminance of an image is repeatedly increased and decreased over time may cause the visual impairment.

  A conventional technique in view of such a problem is disclosed in Patent Document 2, for example. Patent Document 2 discloses a technique for detecting a blinking scene frame and narrowing the dynamic range of image data of the detected frame.

  If a display device capable of displaying an HDR image is used, a display image having “glare” that can be felt in daily life can be obtained. However, such “glare” may cause visual disturbance or discomfort. The “glare” that causes visual disturbance and discomfort is called “glare”. In the technique disclosed in Patent Document 2, the dynamic range of image data is narrowed only in the blinking scene. For this reason, even if the technique disclosed in Patent Document 2 is used, it is not possible to reduce visual disturbance and discomfort in scenes other than the blinking scene. For example, it is not possible to reduce “glare” in a scene where the sun hidden by a shield appears, a scene where light is emitted from a car headlight on a dark night road, and the like. It is unlikely that the user will focus on the sun or the headlight area, and the user usually focuses on the area of the object irradiated with the light emitted from the sun or the light. Therefore, it is unlikely that the user has a good impression that “realism, glossiness, shine, power, etc. are high” with respect to the “glare” in the sun and light areas. Is likely to be recognized as a disturbance by the user.

  Regardless of the scene, if the correction that narrows the dynamic range of the image data is always performed, the above-described visual disturbance, visual disturbance, and visual discomfort can be reduced. However, if correction for narrowing the dynamic range of image data is always performed, there is a possibility that a display image reflecting the intention of the image producer cannot be obtained. The performer's area, the glittering precious metal area, the glittering jewel area, etc. that are spotlighted on the stage are areas that the user is likely to pay attention to. In addition, it is unlikely that the user will recognize “glare” in the area that the user is paying attention as an obstruction, and the user is not able to recognize “realism, glossiness, brightness, power, etc.” It is likely to have a good impression of “high”. In addition, a region that is highly likely to be noticed by the user is highly likely to be a region where the image producer expects (intentions) to display an image with high reality, gloss, shine, power, and the like. However, if correction for narrowing the dynamic range of image data is always performed, a display image with low reality, glossiness, shine, force, and the like in a region that is noticed by the user can be obtained.

JP 2011-193511 A JP 2010-35148 A

  An object of this invention is to provide the technique which can obtain the image data of suitable brightness | luminance.

The first aspect of the present invention is:
Acquisition means for acquiring, from each input image data, a luminance histogram representing, as a frequency, the total number of pixels having luminance values belonging to the category for each category that is a luminance range including one or more luminance values;
Detecting means for detecting a cluster which is a luminance range composed of one or more categories whose frequency is equal to or greater than a first threshold from the luminance histogram;
Determining means for determining whether or not the cluster detected by the detecting means satisfies a predetermined condition;
Correction means for correcting the input image data based on the determination result of the determination means;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: consists of a number of categories equal to or less than the second threshold value Condition 2: an image processing device including a luminance value higher than the third threshold value .

The second aspect of the present invention is:
For each of a plurality of blocks constituting the image area represented by the input image data, the spatial frequency information related to the spatial frequency of the image in the block and the feature quantity related to the luminance of the image in the block are acquired from the input image data. Acquisition means;
For each of the plurality of blocks, determination means for determining whether the block satisfies a predetermined condition based on the spatial frequency information and the feature amount acquired for the block;
Correction means for correcting the input image data based on the determination result of the determination means;
Have
The determination means determines that the predetermined condition is satisfied for a block satisfying the first predetermined condition including the following condition 1 and condition 2: Condition 1: The spatial frequency of the image is a spatial frequency equal to or higher than the first threshold value. Condition 2 including a certain high-frequency component: An image processing apparatus characterized in that the luminance of an image is higher than a second threshold value.

The third aspect of the present invention is:
For each of a plurality of blocks constituting the region of the image represented by the input image data, an acquisition unit that acquires, from the input image data, a feature amount related to the luminance of the image in the block;
Detection means for detecting an image area of a predetermined object from an image area represented by the input image data;
A determination unit that determines, for each of the plurality of blocks, whether or not the block satisfies a predetermined condition based on the feature amount acquired for the block and the area detected by the detection unit; ,
Correction means for correcting the input image data based on the determination result of the determination means;
Have
The determination means determines that the predetermined condition is satisfied for a block satisfying a first predetermined condition including the following condition 1 and condition 2: Condition 1: includes an image area of the predetermined object Condition 2: Image Is an image processing device characterized in that the luminance of the image processing device is higher than the first threshold value.

The fourth aspect of the present invention is:
Acquisition means for acquiring, for each frame, a feature amount related to the brightness of the image represented by the input image data of the frame;
Detecting means for detecting a scene satisfying a predetermined condition as a blinking scene in which the luminance of the image is repeatedly increased and decreased based on the feature amount acquired by the acquiring means;
Correction means for correcting the input image data of the blinking scene detected by the detection means so that the representable range that is the luminance range from the lower limit value to the upper limit value of the luminance value gradually changes in time;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: The frequency at which the luminance of the image fluctuates within a fluctuation range of the first threshold or higher is the second threshold or higher. Condition 2: The maximum value of the luminance of the image is An image processing apparatus characterized by being higher than a third threshold value.

According to a fifth aspect of the present invention,
Obtaining a luminance histogram from the input image data for each category that is a luminance range including one or more luminance values, representing the total number of pixels having luminance values belonging to that category as a frequency;
Detecting from the luminance histogram a cluster that is a luminance range consisting of one or more categories whose frequency is equal to or greater than a first threshold;
A determination step of determining whether or not the cluster detected in the detection step satisfies a predetermined condition;
A correction step of correcting the input image data based on the determination result of the determination step;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: consists of a number of categories equal to or less than the second threshold value Condition 2: an image processing method including a luminance value higher than the third threshold value .

The sixth aspect of the present invention is:
For each of a plurality of blocks constituting the image area represented by the input image data, the spatial frequency information related to the spatial frequency of the image in the block and the feature quantity related to the luminance of the image in the block are acquired from the input image data. An acquisition step;
For each of the plurality of blocks, a determination step of determining whether or not the block satisfies a predetermined condition based on the spatial frequency information and the feature amount acquired for the block;
A correction step of correcting the input image data based on the determination result of the determination step;
Have
In the determination step, it is determined that the predetermined condition is satisfied for a block satisfying the first predetermined condition including the following conditions 1 and 2. Condition 1: The spatial frequency of the image is a spatial frequency equal to or higher than the first threshold. Condition 2 that includes a certain high-frequency component: An image processing method characterized in that the luminance of an image is higher than a second threshold value.

The seventh aspect of the present invention is
For each of a plurality of blocks constituting the region of the image represented by the input image data, an acquisition step of acquiring, from the input image data, a feature amount related to the luminance of the image in the block;
A detection step of detecting an image area of a predetermined object from an image area represented by the input image data;
For each of the plurality of blocks, a determination step for determining whether or not the block satisfies a predetermined condition based on the feature amount acquired for the block and the region detected in the detection step; ,
A correction step of correcting the input image data based on the determination result of the determination step;
Have
In the determination step, it is determined that the predetermined condition is satisfied for a block satisfying the first predetermined condition including the following condition 1 and condition 2: Condition 1: including an image area of the predetermined object Condition 2: Image The image processing method is characterized in that the luminance of the image is higher than the first threshold value.

The eighth aspect of the present invention is
For each frame, an acquisition step of acquiring a feature amount related to the brightness of the image represented by the input image data of the frame;
A detection step of detecting a scene satisfying a predetermined condition as a flickering scene in which an increase / decrease in the luminance of the image is repeated based on the feature amount acquired in the acquisition step;
A correction step of correcting the input image data of the blinking scene detected in the detection step so that an expressible range that is a luminance range from the lower limit value to the upper limit value of the luminance value gradually changes in time;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: The frequency at which the luminance of the image fluctuates within a fluctuation range of the first threshold or higher is the second threshold or higher. Condition 2: The maximum value of the luminance of the image is An image processing method characterized by being higher than a third threshold.

  A ninth aspect of the present invention is a program that causes a computer to execute each step of the above-described image processing method.

  According to the present invention, image data with suitable luminance can be obtained.

FIG. 6 is a diagram illustrating an example of images before and after correction according to the first embodiment. 1 is a block diagram illustrating an example of a functional configuration of an image processing apparatus according to a first embodiment. 7 is a flowchart illustrating an example of a processing flow of the image processing apparatus according to the first embodiment. FIG. 10 is a diagram illustrating a specific example of processing of the image processing apparatus according to the first embodiment. The figure which shows the example which reduces a luminance value using the gain value which concerns on Example 1. FIG. FIG. 10 is a diagram illustrating a specific example of processing of the image processing apparatus according to the first embodiment. 1 is a block diagram illustrating an example of a functional configuration of an image processing apparatus according to a first embodiment. FIG. 10 is a diagram illustrating an example of images before and after correction according to the second embodiment. FIG. 9 is a block diagram illustrating an example of a functional configuration of an image processing apparatus according to a second embodiment. FIG. 10 is a diagram illustrating a specific example of processing of the image processing apparatus according to the second embodiment. FIG. 10 is a diagram illustrating a specific example of processing of the image processing apparatus according to the second embodiment. The figure which shows an example of the image before and behind the correction | amendment which concerns on Example 3, 4. FIG. 9 is a block diagram illustrating an example of a functional configuration of an image processing apparatus according to a third embodiment. The figure which shows an example of the correspondence of the luminance value before and behind correction which concerns on Example 3, 4. FIG. 9 is a block diagram illustrating an example of a functional configuration of an image processing apparatus according to a fourth embodiment.

<Example 1>
Hereinafter, an image processing apparatus and an image processing method according to Embodiment 1 of the present invention will be described.
Image data representing an image having a wide dynamic range can be input to the image processing apparatus according to the present embodiment. In this embodiment, a wide dynamic range is described as “HDR (High Dynamic Range)”, an image having HDR is described as “HDR image”, and image data representing the HDR image is described as “HDR image data”. The HDR image is an image having a very bright region such as a region of an object illuminated by illumination light, a region of a light emitter (sun, illumination, etc.), and the like.

  In this embodiment, the brightness histogram of the HDR image data is analyzed. As a result, the high brightness region of the HDR image is a region where the user has a good impression that “reality, glossiness, shine, power, etc. are high” with respect to “glare”. It is determined whether the area is recognized as a disturbance by the user. Then, the brightness of the HDR image data is corrected to a suitable brightness based on the determination result of the high brightness area. A region where the user has a good impression that “the reality, glossiness, brightness, power, etc. are high” with respect to “glare” is a region where the user is likely to pay attention. Therefore, in this embodiment, an area where the user has a good impression that “realism, glossiness, shine, power, etc. are high” with respect to “glare” is described as “attention area”. In addition, a region where the user recognizes “glare” as an obstruction is a region where the user is not likely to pay attention. Therefore, in this embodiment, an area in which the user recognizes “glare” as an obstruction is described as a “non-attention area”.

FIG. 2 is a block diagram illustrating an example of a functional configuration of the image processing apparatus according to the present embodiment.
The image processing apparatus according to the present embodiment performs image processing on input image data (image data input to the image processing apparatus), and outputs the image data subjected to the image processing to the display unit 104.
The display unit 104 is a display unit (display device) that can display an HDR image. The display unit 104 displays an image based on the image data input to the display unit 104 on the screen.

Note that the display method and configuration of the display unit 104 are not particularly limited. For example, a display device that includes a light-emitting portion and a display panel that displays an image on a screen by modulating light emitted from the light-emitting portion can be used as the display portion 104. Specifically, a liquid crystal display device having a backlight portion and a liquid crystal panel can be used as the display portion 104. A MEMS shutter type display device using a MEMS (Micro Electro Mechanical System) shutter instead of the liquid crystal element can also be used as the display unit 104. As the light emitting unit of the display unit 104, a light emitting unit whose emission luminance can be locally changed, a light emitting unit whose emission luminance cannot be locally changed, or the like can be used. If a light emitting unit capable of locally changing the light emission luminance is used, the dynamic range of the display image (image displayed on the screen) can be expanded by controlling the light emission luminance of the light emitting unit. Specifically, by controlling the emission luminance in a region where a bright image is displayed to a high value or controlling the emission luminance in a region where a dark image is displayed to a low value, The range can be expanded. A self-luminous display device such as a plasma display device or an organic EL display device can also be used as the display unit 104.
2 shows an example in which the display unit 104 is a separate device from the image processing apparatus, the display unit 104 may be an apparatus integrated with the image processing apparatus.

  The feature amount acquisition unit 101 acquires a feature amount related to the luminance of the input image data from the input image data. In the present embodiment, the feature amount acquisition unit 101 acquires a luminance histogram P from input image data. Then, the feature amount acquisition unit 101 outputs the luminance histogram P to the high luminance region determination unit 102. The luminance histogram P represents the total number of pixels having luminance values belonging to the category for each category that is a luminance range including one or more luminance values as the frequency F1. When a category includes one luminance value, “category” can be read as “luminance value”.

  Based on the feature amount acquired by the feature amount acquisition unit 101, the high luminance area determination unit 102 determines what kind of scene image the image (input image) represented by the input image data is. Specifically, the high brightness area determination unit 102 determines whether or not a high brightness area that is a non-attention area exists in the input image based on the feature amount acquired by the feature amount acquisition unit 101. Then, the high brightness area determination unit 102 outputs the determination result to the image correction unit 103. Details of the processing of the high brightness area determination unit 102 will be described later.

  The image correction unit 103 corrects the input image data based on the determination result of the high brightness area determination unit 102 (image processing). In this embodiment, when it is determined that “a high luminance area that is a non-attention area exists in the input image”, the image correction unit 103 determines that the luminance value of the pixels that constitute the high luminance area that is a non-attention area is The input image data is corrected so as to be reduced.

  The high luminance region determination unit 102 includes a cluster detection unit 105 and a non-attention region detection unit 106.

  The cluster detection unit 105 detects, from the luminance histogram P acquired by the feature amount acquisition unit 101, a cluster that is a luminance range composed of one or more categories whose frequency F1 is greater than or equal to a threshold TH_F (first threshold). Then, the cluster detection unit 105 outputs the detection result to the non-attention area detection unit 106. For example, information indicating the minimum luminance value and the maximum luminance value of the detected cluster is output as the detection result.

The non-attention area detection unit 106 determines whether the cluster detected by the cluster detection unit 105 satisfies a predetermined condition based on the detection result of the cluster detection unit 105. Then, the non-attention area detection unit 106 outputs the determination result to the image correction unit 103. For example, information indicating the minimum luminance value and the maximum luminance value of a cluster that satisfies a predetermined condition is output as a determination result. The predetermined condition according to the present embodiment includes the following condition 1 and condition 2. The predetermined condition may further include conditions other than conditions 1 and 2.
Condition 1: Consists of a number of categories equal to or less than the threshold TH_C (second threshold) Condition 2: Includes a luminance value higher than the reference value W (third threshold) In this embodiment, the predetermined condition (conditions 1 and 2) is satisfied Of the luminance values belonging to the cluster, a pixel region having a luminance value higher than the reference value W is treated as a high luminance region that is a non-attention region. Therefore, the process for determining whether or not a cluster satisfies a predetermined condition can also be referred to as “a process for determining whether or not a high-luminance area that is a non-attention area exists in the input image”.
Regardless of whether or not the luminance value is higher than the reference value W, a region of a pixel having a luminance value belonging to a cluster that satisfies a predetermined condition may be treated as a high luminance region that is a non-attention region.

A specific example of processing of the cluster detection unit 105 will be described.
In this embodiment, the cluster detection unit 105 binarizes the frequency F1 equal to or greater than the threshold TH_F to 1, and the frequency F1 less than the threshold TH_F to 0. Then, the cluster detection unit 105 detects a luminance range including one or more categories having a binarized frequency F2 of 1 as a cluster. In this embodiment, when the distance D between the two categories is less than the threshold TH_D and the frequency F2 of the two categories is 1, the two categories are treated as belonging to the same cluster. The distance D corresponds to a difference value (absolute value) between the maximum luminance value of the low luminance side category and the minimum luminance value of the high luminance side category.

A specific example of the processing of the non-attention area detection unit 106 will be described.
In this embodiment, the non-attention area detection unit 106 determines whether or not the cluster detection unit 105 has detected a plurality of clusters. When the total number of clusters detected by the cluster detection unit 105 is one, the non-attention area detection unit 106 assigns importance (attention level) “high” to the cluster. When the total number of clusters detected by the cluster detection unit 105 is two or more, the non-attention area detection unit 106 determines, for each cluster, the total number of categories included in the cluster. Assign importance. Specifically, the non-attention area detection unit 106 assigns importance “high” to clusters including more categories than the threshold TH_C, and assigns importance “low” to other clusters. . A cluster to which importance “low” is assigned is a cluster that satisfies condition 1. Then, the non-attention area detection unit 106 determines whether or not the cluster assigned the importance “low” includes a luminance value higher than the reference value W. A cluster to which importance “low” is assigned and includes a luminance value higher than the reference value W is a cluster that satisfies a predetermined condition (conditions 1 and 2).

  An example of the processing flow of the image processing apparatus according to the present embodiment will be described with reference to FIG. FIG. 3 is a flowchart illustrating an example of a processing flow of the image processing apparatus according to the present embodiment.

First, the feature amount acquisition unit 101 acquires a luminance histogram P from input image data (S01).
Next, the cluster detection unit 105 generates a binarized histogram Q by binarizing the frequency F1 of the luminance histogram P acquired in S01 (S02). Specifically, the cluster detection unit 105 generates a binarized histogram Q by binarizing the frequency F1 greater than or equal to the threshold TH_F to 1 and the frequency F1 less than the threshold TH_F to 0.

  And the process of S03-S11 is performed about each of a some category. In the present embodiment, the cluster detection unit 105 selects each of the plurality of categories as the processing target of S03 to S11 in order from the low luminance side category. In the present embodiment, N categories (N is an integer of 2 or more) are set, and for each of the N categories, the number increases by 1 as the luminance corresponding to the category increases. A number from 1 to N is set. Then, the cluster detection unit 105 repeatedly performs a process of selecting the category of the number n as a processing target while increasing the variable n by 1 from 1 to N.

  In S03, the cluster detection unit 105 determines whether the frequency F2 of the category number n (the frequency of the binarized histogram Q generated in S02) is 0. In FIG. 3, the frequency F2 (n) is the frequency F2 of the category of number n. If the frequency F2 (n) is 0, the process proceeds to S04, and if the frequency F2 (n) is 1, the process proceeds to S07.

In S04, the cluster detection unit 105 determines whether both of the following conditions A and B are satisfied. In this embodiment, the initial value of the distance D is 0.
Condition A: Distance D is 0 Condition B: Number n is not 1 (initial value) When both Condition A and Condition B are satisfied, the process proceeds to S05, and Condition A and Condition B If at least one is not satisfied, the process proceeds to S06.
In S05, the cluster detection unit 105 sets the number n−1 as the number Ye of the end category of the cluster for which the start category was determined immediately before. The start end category is a category corresponding to the start end (end on the low brightness side) of the cluster, and the end category is a category corresponding to the end end (end on the high brightness side) of the cluster. Further, the cluster detection unit 105 adds 1 to the total number of clusters (total number of clusters) CN. The initial value of the cluster total number CN is 0. Although the details will be described later, the start category is determined by the processing of S08.
In S06, the cluster detection unit 105 adds 1 to the distance D. Thereafter, when the process of selecting each of the N categories as the process target is completed, the process proceeds to S12. If there is a category that is not selected as a processing target, the number n is incremented by 1 to update the processing target category, and the process returns to S03.

  If there are a plurality of clusters, a plurality of combinations of the numbers Ys and Ye are determined. For this reason, it is preferable to prepare a storage device capable of storing a plurality of numbers Ys and a plurality of numbers Ye and record the numbers Ys and Ye in the storage device. As the storage device, a semiconductor memory, a magnetic disk, an optical disk, or the like can be used.

In S07, the cluster detection unit 105 determines whether at least one of the following condition C and condition D is satisfied. In this embodiment, the initial value of the cluster flag Fs is 1.
Condition C: Distance D is greater than or equal to threshold TH_D Condition D: Cluster flag Fs is 1 When at least one of condition C and condition D is satisfied, the process proceeds to S08, where condition C and condition D If neither of these is satisfied, the process proceeds to S09.
In S08, the cluster detection unit 105 sets the number n as the number Ys of the start category. In addition, the cluster detection unit 105 sets 0 as the value of the cluster flag Fs. Thereafter, the process proceeds to S09.
In S09, the cluster detection unit 105 resets the distance D to 0. Thereafter, the process proceeds to S10.

In S10, the cluster detection unit 105 determines whether the number n is equal to the number N. If the number n is equal to the number N, the process proceeds to S11. When the number n is smaller than the number N, the category to be processed is updated by incrementing the number n by 1, and the process returns to S03.
In S11, the cluster detection unit 105 sets the number n (= N) as the number Ye of the end category of the cluster for which the start category was determined immediately before. In addition, the cluster detection unit 105 adds 1 to the total cluster number CN. Thereafter, the process proceeds to S12.

In S12, the non-attention area detection unit 106 determines whether or not a plurality of clusters are detected. Specifically, the non-attention area detection unit 106 determines whether or not the total cluster number CN is 2 or more. If a plurality of clusters are detected, that is, if the total cluster number CN is 2 or more, the process proceeds to S13. When a plurality of clusters are not detected, that is, when the total cluster number CN is less than 2, this flowchart is terminated. Specifically, when the cluster total number CN is less than 2, the processing by the image correcting unit 103 (processing for correcting input image data) is omitted.

  When a plurality of clusters are detected, the processes of S13 to S15 are performed for each of the plurality of clusters. In the present embodiment, the non-attention area detection unit 106 selects each of the plurality of clusters in order as processing targets in S13 to S15.

  In S <b> 13, the non-attention area detection unit 106 determines whether the total number of categories constituting the processing target cluster is equal to or less than a threshold value. If the total number of categories constituting the cluster to be processed is equal to or less than the threshold TH_C, the process proceeds to S14. When the total number of categories constituting the cluster to be processed is larger than the threshold value TH_C and the process of selecting each cluster as the process target is completed, this flowchart is ended. If the total number of categories constituting the processing target cluster is greater than the threshold TH_C and there is a cluster not selected as the processing target, the processing target cluster is updated, and the process returns to S13.

  In S14, the non-attention area detection unit 106 determines whether or not the processing target cluster includes a luminance value higher than the reference value W. For example, in S14, the termination category of the cluster to be processed is determined from the number Ye corresponding to the cluster to be processed. When the maximum luminance value of the terminal category is larger than the reference value W, it is determined that “a luminance value higher than the reference value W is included in the cluster to be processed”. When the maximum luminance value of the terminal category is equal to or less than the reference value W, it is determined that “the processing target cluster does not include a luminance value higher than the reference value W”. If the cluster to be processed includes a luminance value higher than the reference value W, the process proceeds to S15. If the cluster to be processed does not include a luminance value higher than the reference value W and the process of selecting each cluster as the process target is completed, this flowchart is ended. If the cluster to be processed does not contain a luminance value higher than the reference value W and there is a cluster not selected as the process target, the cluster to be processed is updated, and the process returns to S13. It is.

  In S15, the image correction unit 103 corrects the input image data so that the luminance value of the pixel having a luminance value higher than the reference value W among the luminance values belonging to the cluster to be processed is reduced. Specifically, the image correction unit 103 subtracts the offset value from the luminance value of a pixel having a luminance value higher than the reference value W among the luminance values belonging to the processing target cluster. Thereafter, when the process of selecting each cluster as a processing target is completed, this flowchart is ended. If there is a cluster not selected as a processing target, the processing target cluster is updated, and the process returns to S13. By performing image processing for subtracting the offset value from the luminance value, the area of the pixel having the luminance value higher than the reference value W among the luminance values belonging to the cluster to be processed is reduced without reducing the gradation. Can be reduced.

By executing the flowchart, when a plurality of clusters are detected, it is determined that “a plurality of object regions (object regions) corresponding to the plurality of clusters exist in the input image”. When a plurality of clusters are detected, it is determined for each of the plurality of clusters whether the cluster satisfies a predetermined condition. Then, a region of a pixel having a luminance value higher than the reference value W among luminance values belonging to a cluster satisfying the predetermined condition is treated as a high luminance region that is a non-attention region, and the luminance value of the pixels constituting the region is determined. Image processing for subtracting the offset value is performed. When the total number of detected clusters is one, even if the cluster satisfies a predetermined condition, it is unlikely that a pixel region having a luminance value belonging to the cluster is a non-attention region. Therefore, in such a case, the process of determining whether or not the cluster satisfies a predetermined condition (S13 and S14) and the process of correcting the input image data (S15) are omitted.

  A specific example of the processing of the image processing apparatus according to the present embodiment will be described with reference to FIGS.

  FIGS. 4A to 4C illustrate a case where image data having a high luminance area that is an attention area and not having a high luminance area that is a non-attention area is input to the image processing apparatus. An example is shown. Specifically, FIGS. 4A to 4C show examples when image data representing the image shown in FIG. 1A is input to the image processing apparatus. In the image shown in FIG. 1 (A), there is an area of performers who are spotlighted on the stage. The performer's area that is spotlighted on the stage is an area where the user is likely to pay attention, and the user is “high in reality, gloss, brightness, power, etc.” It is an area that is likely to have a good impression.

  FIGS. 4D to 4F show an example in which image data having a high luminance area that is a non-attention area is input to the image processing apparatus. Specifically, FIGS. 4D to 4F show examples when image data representing the image shown in FIG. 1B is input to the image processing apparatus. In the image shown in FIG. 1B, there is a solar region. The sun region is a region where the user has a low possibility of paying attention and is a region where the user is likely to recognize the “glare” as an obstruction.

  4A and 4D show the luminance histogram P acquired by the feature amount acquisition unit 101. FIG. 4B and 4E show the binarized histogram Q generated by the cluster detection unit 105. FIG. 4C and 4F show luminance histograms of image data output from the image correction unit 103. FIG.

  First, an example in which image data representing the image shown in FIG. 1A is input to the image processing apparatus will be described.

The cluster detection unit 105 generates a binarized histogram Q based on the determination result of whether or not the frequency F1 of the luminance histogram P is equal to or greater than the threshold value TH_F. As the threshold value TH_F, for example, a value that can exclude the influence of noise included in the input image data is used. 1 may be used as the threshold TH_F.
In the luminance histogram P of FIG. 4A, the frequency F1 of the categories from number Y11 to number Y12 is greater than or equal to the threshold value TH_F, and the frequency F1 of other categories is less than the threshold value TH_F. Therefore, a binarized histogram Q in which the frequency F2 of the categories from number Y11 to number Y12 is 1 and the frequency F2 of other categories is 0 is generated (FIG. 4B).

Further, the cluster detection unit 105 detects a luminance range including one or more categories having a frequency F2 of 1 as a cluster.
In the binarized histogram Q of FIG. 4B, as described above, the frequency F2 of the categories from number Y11 to number Y12 is 1, and the frequency F2 of the other categories is 0. Therefore, a cluster 11 composed of a plurality of categories from number Y11 to number Y12 is detected (FIG. 4B). In the example of FIG. 4B, the total number of categories constituting the cluster 11 is M11 (= Y12−Y11 + 1). The cluster 11 straddles the reference value W. Here, it is assumed that the total number M11 is larger than the threshold value TH_C.

As the reference value W, for example, the brightness that the user feels dazzling in a predetermined visual environment is used. Such brightness (reference value W) can be determined based on the results of experiments on a large number of subjects (users), for example. Specifically, for each of a plurality of subjects, an experiment is performed to determine the brightness that the subject feels dazzling, and representative values (average value, mode value, Intermediate value, maximum value, minimum value) can be determined as the reference value W.

  In FIG. 4B, only one cluster 11 is detected. Therefore, the non-attention area detection unit 106 determines that “the area of the pixel having the luminance value belonging to the cluster 11 is the attention area”. Further, the non-attention area detection unit 106 determines that “the area of the pixel having a luminance value higher than the reference value W among the luminance values belonging to the cluster 11 is a high luminance area”. Specifically, the non-attention area detection unit 106 determines that the user has “reality, glossiness, brightness” with respect to “glare” of pixels having luminance values from the reference value W to the maximum luminance value of the cluster 11. The image correction unit 103 outputs the input image data without performing the process of correcting the input image data, and as a result, the image correction unit 103 outputs the input image data. The luminance histogram of C) matches the histogram of FIG.

  Next, an example where image data representing the image shown in FIG. 1B is input to the image processing apparatus will be described.

  In the luminance histogram P of FIG. 4D, the category frequency F1 from number Y21 to number Y22 and the category frequency F1 from number Y23 to number Y24 are equal to or greater than the threshold value TH_F, and the frequency F1 of other categories is the threshold value. It is less than TH_F. Therefore, a binarized histogram Q is generated in which the frequency F2 of the category from number Y21 to number Y22 and the frequency F2 of the category from number Y23 to number Y24 are 1 and the frequency F2 of the other category is 0. (FIG. 4E).

  In the binarized histogram Q of FIG. 4E, as described above, the frequency F2 of the categories from the number Y21 to the number Y22 and the frequency F2 of the categories from the number Y23 to the number Y24 are 1, and the other categories The frequency F2 is zero. Here, it is assumed that the distance D between the category of number Y22 and the category of number 23 is equal to or greater than the threshold value TH_D. Therefore, two clusters are detected: a cluster 21 composed of a plurality of categories from number Y21 to number Y22 and a cluster 22 composed of a plurality of categories from number Y23 to number Y24 (FIG. 4 (E)). In the example of FIG. 4E, the total number of categories that constitute the cluster 21 is M21, and the total number of categories that constitute the cluster 22 is M22. Further, the cluster 21 does not include a luminance value higher than the reference value W, and the cluster 22 includes only a luminance value higher than the reference value W. Here, it is assumed that the total number M21 is larger than the threshold value TH_C and the total number M22 is smaller than the threshold value TH_C.

  In FIG. 4E, two clusters 21 and 22 are detected. Therefore, the non-attention area detection unit 106 determines whether or not a predetermined condition (conditions 1 and 2) is satisfied for each of the two clusters 21 and 22.

  As described above, the cluster 21 does not include a luminance value higher than the reference value W, and the total number M21 of categories constituting the cluster 21 is larger than the threshold value TH_C. That is, the cluster 21 does not satisfy the predetermined condition. Therefore, the non-attention area detection unit 106 determines that “the area of the pixel having the luminance value belonging to the cluster 21 is the attention area”.

As described above, the cluster 22 includes only luminance values higher than the reference value W, and the total number M22 of categories constituting the cluster 22 is smaller than the threshold value TH_C. That is, the cluster 22 satisfies a predetermined condition. Therefore, the non-attention area detection unit 106 determines that “the area of the pixel having the luminance value belonging to the cluster 22 is the non-attention area”. The non-attention area detection unit 106
It is determined that “a pixel region having a luminance value higher than the reference value W among luminance values belonging to the cluster 22 is a high luminance region”. Specifically, the non-attention area detection unit 106 determines that “the user recognizes“ glare ”of pixels belonging to the cluster 22 and having a luminance value higher than the reference value W as interference”.

  Then, the image correcting unit 103 performs image processing for subtracting the offset value from the luminance value of the pixel having a luminance value higher than the reference value W among the luminance values belonging to the cluster 22, and the image after the image processing Output data. With the above image processing, the luminance histogram changes from the luminance histogram of FIG. 4D to the luminance histogram of FIG. Further, the image changes from the image of FIG. 1B to the image of FIG. In FIG. 1C, the brightness of the solar region is reduced. That is, the “glare” of the solar region is reduced.

  When the distance D in FIG. 4E is less than the threshold TH_D, a group of a plurality of categories from the category with the number Y21 to the category with the number Y24 is detected as one cluster. When the distance D in FIG. 4E is equal to or greater than the threshold TH_D, the start category number Ys = Y21 is set in S08, and then the end category number Ye = Y22 is set in S04. Thereafter, the number Ys = Y23 is set in S08, and the number Ye = Y24 is set in S04. However, when the distance D in FIG. 4E is less than the threshold TH_D, after the number Ys = Y21 and the number Ye = Y22 are set, the number Ys = Y23 is not set, and the number Ye is set in S04. = Y24 is set. In that case, a group of a plurality of categories from the category of the number Y21 to the category of the number Y24 is treated as one cluster, and the process of adding 1 to the total number of clusters CN when setting the number Ye = Y24 is omitted. Is done.

  The effect of the present embodiment will be described.

  In this embodiment, a pixel area having a luminance value higher than the reference value W is treated as a high luminance area. Specifically, a pixel region having a luminance value higher than the reference value W among luminance values belonging to the cluster satisfying the condition 2 is treated as a high luminance region.

  When the luminance range of the object area is narrow, there is a high possibility that the object area does not include details (fine expression), and the possibility that the user pays attention to the object area in order to confirm the detail is low. In addition, it is unlikely that the user will pay attention to the area of the illuminant (sun, illumination, etc.), and the luminance range of the illuminant area is often narrow. Then, the “glare” in the non-attention area (the area where the user is less likely to pay attention) is more likely to be the “glare” that the user recognizes as interference.

  On the other hand, when the luminance range of the object area is wide, there is a high possibility that the object area includes details, and there is a high possibility that the user pays attention to the object area in order to confirm the details. In addition, there is a high possibility that the user pays attention to the area of the object illuminated by the illumination light, and the luminance range of the object area illuminated by the illumination light is often wide. The “glare” in the attention area (the area where the user is highly likely to pay attention) is the “glare” that gives the user a good impression that “reality, glossiness, brightness, power, etc. are high”. There is a high possibility.

Therefore, when the cluster satisfies the condition 1, since the luminance range of the cluster is narrow, there is a high possibility that the pixel area having the luminance value belonging to the cluster is a non-attention area (an area where the user is unlikely to focus). When the cluster does not satisfy the condition 1, since the luminance range of the cluster is wide, there is a high possibility that the pixel region having the luminance value belonging to the cluster is the attention region (the region where the user is highly likely to pay attention). The “glare” in the non-attention area is highly likely to be “glare” that the user recognizes as interference. In addition, “glare” in the attention area
Is likely to be “glare” with a good impression that the user has “high reality, gloss, shine, power, etc.”.

  According to the present embodiment, a pixel area having a luminance value higher than the reference value W among luminance values belonging to a cluster that satisfies a predetermined condition (conditions 1 and 2) is treated as a high luminance area that is a non-attention area. The brightness value of the pixels constituting the high brightness area which is the non-attention area is reduced. Thereby, it is possible to reduce the “glare” that the user recognizes as an obstruction. Further, since the other luminance values are not corrected, it is possible to obtain image data that provides “glare” with a good impression that the user has “high reality, gloss, shine, power, etc.”. Specifically, it is possible to obtain image data that realizes image display in which “glare” in the region of interest is maintained.

  In the present embodiment, the example in which the process of correcting the input image data is omitted when the total number of detected clusters is one has been described. However, the present invention is not limited to this. For example, regardless of the total number of detected clusters, when there are clusters that satisfy a predetermined condition, a process for correcting the input image data may be performed.

  Note that the various threshold values described in the present embodiment may be fixed values predetermined by a manufacturer or the like, or may be values preset by a user (changeable values). The various threshold values described in this embodiment may be values that are automatically determined and changed by an image processing apparatus or the like.

  Note that a luminance value of image data corresponding to white luminance in a predetermined viewing environment (illumination condition) may be used as the reference value W.

  Further, in the imaging device, when generating image data for the LDR display device, the imaging parameter value is set so that the sensor value of the imaging device when shooting white is a predetermined value according to the viewing environment. Often. The LDR display device is a display device that cannot display an HDR image. In other words, the LDR display device is a display device that can display only images with a narrow dynamic range. When the image data for the LDR display device is generated, the image pickup device often generates image data in which the luminance value corresponding to the white luminance matches the maximum luminance value that can be displayed by the LDR display device. . Such a luminance value (luminance value corresponding to white; maximum luminance value that can be displayed by the LDR display device) may be used as the reference value W. By using such a reference value W, it is possible to suitably control the “glare” caused by the brightness value higher than the maximum brightness value that can be displayed by the LDR display device.

  Further, the reference value W may be determined based on the visual environment of the image based on the image data output from the image processing apparatus. Specifically, the reference value W is based on the detection value of an external light sensor (sensor that detects external light of the display device) provided in the display device that displays an image based on the image data output from the image processing device. May be determined.

  The display device generally has a user image quality adjustment unit. The user image quality adjustment unit is a functional unit that performs image quality adjustment such as brightness adjustment and gamma adjustment using an adjustment value according to a user operation. The reference value W may be determined based on the adjustment value acquired from the user image quality adjustment unit. Thereby, even when the image quality of the image based on the image data output from the image processing apparatus is adjusted by the user image quality adjustment unit, the above-described effect of the present embodiment can be obtained.

The non-attention area detection unit 106 may assign importance “high” to a cluster including both a luminance value higher than the reference value W and a luminance value equal to or lower than the reference value W. The non-attention area detection unit 106 may assign importance “low” to a cluster that includes a luminance value higher than the reference value W and does not include a luminance value equal to or lower than the reference value W. That is, the predetermined condition is the following condition 3
May further be included.
Condition 3: Does not include luminance values below the reference value W

  When image processing for reducing only a luminance value higher than the reference value W is performed on an object region including a luminance value higher than the reference value W and a luminance value equal to or lower than the reference value W, the image in the object region Impressions can change significantly. By taking the above condition 3 into consideration, a cluster including a luminance value equal to or less than the reference value W does not satisfy the predetermined condition. Thereby, it can suppress that the said image processing is performed with respect to the object area | region containing the luminance value higher than the reference value W, and the luminance value below the reference value W. And it can suppress that the impression of the image in the object area | region containing the luminance value higher than the reference value W and the luminance value below the reference value W changes by the said image processing. In addition, for an object region that includes only a luminance value higher than the reference value W, the luminance value of all the pixels in the object region is uniformly reduced by the image processing. Will not change significantly.

  Note that the offset value used in the image correction unit 103 may be determined based on the distance D. For example, a value less than or equal to the difference value (absolute value) between the minimum luminance value of the processing target cluster and the maximum luminance value of the cluster existing on the lower luminance side than the processing target cluster is used as the offset value. Is preferred. There may be a plurality of clusters on the lower luminance side than the cluster to be processed. In that case, a value equal to or smaller than a difference value between the minimum luminance value of the cluster to be processed and the maximum luminance value of the cluster closest to the cluster to be processed that exists on the lower luminance side than the cluster to be processed, It is preferably used as an offset value. Thereby, it is possible to prevent the luminance value belonging to the cluster to be processed from being corrected to the luminance value belonging to another cluster, and it is possible to suppress image quality deterioration due to gradation collapse or the like.

  Note that the correction method of the input image data is not limited to the above method of subtracting the offset value from the luminance value. For example, the input image data may be corrected by multiplying the luminance value by a gain value smaller than 1. As described above, it is preferable to use a value equal to or smaller than the difference value as the offset value. However, when a value equal to or smaller than the difference value is used as the offset value, the luminance value may not be sufficiently reduced. If gradation compression is performed using the gain value, the number of gradations assigned to a predetermined target cluster is reduced, but the luminance value can be sufficiently reduced. FIG. 5 shows how the luminance value is reduced using the gain value.

  If the input image data is still image data (including image data of a still scene with no image movement (small)), the amount of decrease in the luminance value may be gradually decreased over time. Good. For example, the offset value may gradually approach 0 or the gain value may gradually approach 1 over time. When a user views a display image with high brightness, the user initially feels dazzled, but gradually becomes accustomed to glare and gradually loses glare due to adaptation of human vision. If the reduction amount is gradually reduced in consideration of visual adaptation, it is possible to prevent the user from recognizing “glare” as an obstacle, and finally to the original display luminance (the luminance value of the input image data). Display luminance) can be obtained.

Note that the size of the object area may be taken into account when determining whether or not the object area is a non-attention area. For example, when the frequency of the cluster to be processed (the sum of the frequencies F1 of the categories belonging to the cluster) is equal to or less than the threshold value TH_S, “a pixel area having a luminance value belonging to the cluster to be processed is a non-attention area”. It may be judged. Then, when the frequency of the cluster to be processed is equal to or greater than the threshold value TH_S, it may be determined that “a pixel region having a luminance value belonging to the cluster to be processed is a region of interest”. That is, the predetermined condition may further include the following condition 4.
Condition 4: The frequency is less than the threshold TH_S By considering the size of the object area, it can be determined with higher accuracy whether the object area is a non-attention area. Specifically, by considering the frequency of the cluster, it is possible to determine with higher accuracy whether or not the region of the pixel having the luminance value belonging to the cluster is a non-attention region.

  Specific examples in consideration of the size of the object area are shown in FIGS. 6 (A) to 6 (F). 6 (A), 6 (C), 6 (D), and 6 (F) are the same as FIGS. 4 (A), 4 (C), 4 (D), and 4 (F). 6B and 6E are histograms generated in consideration of the size of the object area. 6B and 6E can be generated from the luminance histogram P and the binarized histogram Q. The histograms in FIGS. 6B and 6E represent the frequency for each cluster. As the frequency of the cluster, the total sum of the frequencies F1 of the categories belonging to the cluster is used. In FIG. 6A, the total frequency of the categories from number Y11 to number Y12 is TF11. Therefore, in FIG. 6B, TF11 is set as the frequency of the cluster 11. In FIG. 6D, the total frequency of the categories from number Y21 to number Y22 is TF21, and the total frequency of the categories from number Y23 to number Y24 is TF22. Therefore, in FIG. 6E, TF21 is set as the frequency of the cluster 21, and TF22 is set as the frequency of the cluster 22.

  As shown in FIG. 7, the image processing apparatus may further include a space LPF unit 107. The spatial LPF unit 107 performs a smoothing process (two-dimensional spatial low-pass filter process) on the input image data. Thereby, image data representing an image obtained by spatially smoothing the input image is generated. Then, the luminance histogram may be acquired not from input image data but from image data generated by the spatial LPF unit 107. The smoothing process can be realized by using a general two-dimensional FIR filter or IIR filter, but the method of the smoothing process is not particularly limited.

  The user is more likely to feel “glare” as the area of the spatially continuous high luminance region is larger. For example, in an image of a starry sky, the area of each star region (star region) is small, so that the user does not feel “glare” with respect to the star region. If the brightness of the star region is reduced, the reality and brightness of the star region are impaired. Therefore, if smoothing processing is performed, the luminance of the star region can be brought close to the luminance of the background region (dark region of the night sky), or the luminance of the background region around the star region can be brought close to the luminance of the star region. As a result, it can be suppressed that the luminance range corresponding to the star region is detected as a cluster satisfying the predetermined condition, and the luminance of the star region can be suppressed from being reduced. As a result, it is possible to obtain image data representing a high-quality image with high reality and brightness of the star region.

  Note that the processing described in the present embodiment (the flowchart in FIG. 3) may or may not be performed in units of frames. For example, the processing described in the present embodiment may be performed for each of a plurality of blocks constituting the area of the input image. That is, the processing described in this embodiment may be performed for each block. One pixel region may be treated as a block, or a region composed of a plurality of pixels may be treated as a block. When a display device having a light emitting unit capable of individually controlling light emission luminance is used for each of a plurality of partial areas constituting a screen area, the image area to be displayed in the partial area is used as a block. Also good. If the process according to the present embodiment is performed in units of blocks, it is possible to suppress the correction of the luminance value in the area that does not require correction according to the present embodiment (the correction unnecessary area), and a higher quality image can be obtained. Representing image data can be obtained.

In the present embodiment, an example has been described in which the luminance of the high-luminance region that is the non-attention region is reduced and the luminance of the high-luminance region that is the attention region is not reduced (maintained).
There is also a user demand for reducing the brightness (glare) of a region of interest and improving the visibility of the region. For example, there may be a problem that the area of the face or character that is likely to be noticed by the user is too bright and the user cannot confirm the details of the area. There is also a user demand that the non-attention area may be dazzling.
Therefore, the luminance of the high-luminance area that is the attention area may be reduced without reducing the luminance of the high-luminance area that is the non-attention area. Such a configuration, for example, by correcting the input image data so that the luminance value of pixels having a luminance value higher than the reference value W among luminance values belonging to a cluster that does not satisfy the predetermined condition is reduced, Can be realized. Further, the user may be able to select which of the high luminance area that is the non-attention area and the high luminance area that is the attention area to reduce the luminance.

If the brightness of the high brightness area, which is the attention area, is maintained, image data representing an image having a high reality, glossiness, brightness, power, etc. of the area can be obtained. Further, if the brightness of the high brightness area, which is a non-attention area, is reduced, it is possible to reduce visual disturbance and discomfort due to the “glare” of the area.
On the other hand, if the brightness of the high brightness area, which is the attention area, is reduced, image data representing an image with improved visibility of the area can be obtained. Further, if the luminance of the high luminance region which is a non-attention region is maintained, image data representing an image with high reality, glossiness, shine, force, etc. of the region can be obtained.

<Example 2>
Hereinafter, an image processing apparatus and an image processing method according to Embodiment 2 of the present invention will be described.
As in the first embodiment, HDR image data can be input to the image processing apparatus according to the present embodiment. In addition, moving image data representing a moving image can be input to the image processing apparatus according to the present embodiment. That is, the image processing apparatus according to the present embodiment can input image data of a frame for each frame of the moving image.

As a scene of a moving image, there is a blinking scene in which increase / decrease in luminance of the image is repeated. The flickering scene is often included in various moving images such as a sales / motion promotion moving image and a theatrical moving image in order to enhance a sense of brightness, power, and the like.
However, a blinking scene with a sharp increase / decrease in brightness results in “glare” accompanied by visual disturbances such as eye fatigue and moodiness. In addition, the user does not seek a sense of brightness, force, or the like for such a blinking scene, and the blinking scene may be recognized as a visual disturbance by the user. For example, in a moving image of a news report, a user rarely asks for a sparkling feeling, force, etc. for a blinking scene in which a camera strobe emits light continuously, and the blinking scene is recognized as a visual disturbance by the user. There are many.

  In this embodiment, the temporal change in luminance of the input image is analyzed, and a blinking scene that is recognized as a visual disturbance by the user is detected. Based on the detection result of the blinking scene, the luminance of the input image data in the blinking scene is corrected to a suitable luminance.

FIG. 9 is a block diagram illustrating an example of a functional configuration of the image processing apparatus according to the present embodiment.
Similar to the first embodiment, the image processing apparatus according to the present embodiment performs image processing on input image data and outputs the image data subjected to the image processing to the display unit 104. The display unit 104 in FIG. 9 is the same as the display unit 104 in FIG.

The feature amount acquisition unit 201 acquires, for each frame, a feature amount related to the luminance of the image (input image) represented by the input image data of the frame. Then, the feature quantity acquisition unit 201 outputs the feature quantity acquired for each frame to the blinking scene detection unit 202 for each frame. In this embodiment, the average luminance value of the input image data is acquired as the feature amount.
Note that the feature amount is not limited to an average luminance value (an average value of luminance values). For example, the maximum value, the minimum value, the mode value, the intermediate value, and the like of the luminance value of the input image data may be acquired as the feature amount. The average value, maximum value, minimum value, mode value, intermediate value, and the like can be referred to as “representative values”. A luminance value histogram (luminance histogram) of the input image data may be acquired as the feature amount.

  Based on the feature amount acquired by the feature amount acquisition unit 201, the blinking scene detection unit 202 detects a scene that satisfies a predetermined condition as a blinking scene. In this embodiment, the blinking scene detection unit 202 determines whether the scenes of the plurality of frames are blinking scenes (scenes satisfying a predetermined condition) based on the plurality of feature amounts acquired for the plurality of frames. For each frame, the blinking scene detection unit 202 determines whether the frame is a blinking scene frame based on a plurality of feature amounts acquired for a determination target period using the frame as a period reference. The determination target period is at least one of one or more frames that are temporally consecutive before the period-based frame and one or more frames that are temporally consecutive after the period-based frame. It is a period including the time position. The determination target period may be a period including the time position of the period-based frame, or may be a period not including the time position of the period-based frame.

The predetermined condition according to the present embodiment includes the following condition 1 and condition 2. The predetermined condition may further include conditions other than conditions 1 and 2.
Condition 1: The frequency at which the luminance of the image (the average luminance value in this embodiment) fluctuates with a fluctuation width equal to or greater than the threshold TH_W (first threshold) is equal to or greater than the threshold TH_Fr (second threshold) Condition 2: The luminance of the image Maximum value is higher than threshold TH_Bm (third threshold)

  When a scene in which the luminance of the input image repeatedly fluctuates frequently with a large fluctuation range includes a high-luminance input image, there is a high possibility that the scene is recognized as a visual disturbance by the user. That is, there is a high possibility that a scene satisfying both condition 1 and condition 2 will be recognized as visual disturbance by the user. Therefore, the blinking scene detection unit 202 can detect a scene that is highly likely to be recognized as a visual disturbance by the user as a blinking scene.

  The image correction unit 203 corrects the input image data based on the detection result of the blinking scene detection unit 202. In this embodiment, the image correcting unit 203 corrects the input image data of the detected blinking scene so that the representable range gradually changes over time and the fluctuation range is limited to a value less than the threshold value TH_W. . The representable range is a luminance range from the lower limit value to the upper limit value of the luminance value. The fluctuation range can be reduced by correcting the luminance value of the input image data or narrowing the width of the representable range.

A specific example of the process of the blinking scene detection unit 202 will be described.
In this embodiment, the blinking scene detection unit 202 determines, for each frame, a difference value (absolute value) between an average luminance value acquired for the frame and an average luminance value acquired for an adjacent frame using the frame as an adjacent reference. Value) is calculated as the fluctuation range. In the present embodiment, a frame that is temporally adjacent to the adjacent reference frame is treated as an adjacent frame. Further, the blinking scene detection unit 202 treats the frame as a period reference frame for each frame. The blinking scene detection unit 202 stores at least the average luminance value and the difference value for the determination target period, and performs the following processing.
First, the blinking scene detection unit 202 counts the total number of difference values greater than or equal to the threshold value TH_W among the stored difference values as the frequency, and determines whether or not the counted total number TFr is equal to or greater than the threshold value TH_Fr. With this process, it is possible to determine whether or not the scene in the determination target period is a scene whose luminance increases or decreases at an interval of one frame. Further, the blinking scene detection unit 202 determines whether or not the stored maximum value Bm of the average luminance value is higher than the threshold value TH_Bm.
Then, when the total number TFr ≧ threshold value TH_Fr and maximum value Bm> threshold value TH_Bm, the blinking scene detection unit 202 determines that the scene in the determination target period is a blinking scene (a scene satisfying a predetermined condition), and It is determined that the frame is a blinking scene frame.

Note that the blinking scene detection method is not limited to the above method. For example, a frame that is temporally adjacent to an adjacent reference frame may be treated as an adjacent frame. Further, for each of a plurality of partial periods constituting a moving image period, it may be determined whether or not the partial period is a blinking scene.
Note that the blinking scene is not limited to a scene whose luminance increases or decreases at intervals of one frame. A scene in which the brightness of an image increases or decreases at a time interval such that the user feels blinking, and a scene that is highly likely to be recognized as a visual disturbance by the user may be detected as a blinking scene. A scene whose luminance increases or decreases at intervals of a plurality of frames may be detected as a blinking scene. For example, a scene in which a plurality of bright frames (bright image frames) and a plurality of dark frames (dark image frames) are alternately repeated may be detected as a blinking scene. Such a blinking scene can be detected by treating the difference value between the minimum brightness value and the maximum brightness value that are temporally adjacent to each other as the fluctuation range and performing the above processing. The minimum luminance value is the minimum value of luminance in the time variation of the luminance of the input image in the judgment target period or partial period, and the maximum luminance value is the luminance in the time change of the luminance of the input image in the judgment target period or partial period. Is the local maximum. Only the difference value when the time changes from the minimum brightness value to the maximum brightness value may be treated as the fluctuation range, or only the difference value when the time changes from the maximum brightness value to the minimum brightness value is treated as the fluctuation range. Alternatively, a difference value between them may be treated as a fluctuation range. A value representing the difference between the maximum luminance value and the two minimum luminance values before and after the maximum luminance value may be treated as a fluctuation range. For example, a difference value between the maximum luminance value and the average value of the two minimum luminance values may be treated as the fluctuation range.

  A specific example of the processing of the image correction unit 203 will be described with reference to FIGS. FIG. 10A shows an example of a change in luminance of the input image over time. In the example of FIG. 10A, the increase / decrease in the luminance of the input image over time is repeated. In the example of FIG. 10A, the maximum luminance value PH1 is high and the fluctuation range (the difference value between the maximum luminance value PH1 and the minimum luminance value PL) is large. For this reason, the scene showing the temporal change in FIG. 10A is likely to be recognized as a visual disturbance by the user or uncomfortable to the user. Here, it is assumed that the scene showing the temporal change in FIG. 10A is detected by the blinking scene detection unit 202 as a blinking scene (a scene satisfying a predetermined condition).

  Conventionally, as shown in FIG. 10B, the fluctuation range is reduced by reducing the maximum luminance value PH1 to the maximum luminance value PH2, thereby reducing visual disturbance and discomfort. However, if such a correction is made, the brightness of the scene will always be a dark value, and the brightness, power, etc. will be reduced.

Therefore, as shown in FIG. 10C, the image correction unit 203 detects the blinking scene detected so that the representable range gradually changes with time and the fluctuation range is limited to a value less than the threshold value TH_W. Correct the input image data. Thereby, it is possible not only to reduce the visual interference and discomfort, but also to suppress the reduction of the feeling of brightness, force, and the like.
For example, in the example of FIG. 10C, since the fluctuation ranges R1 to R4 are limited to values less than the threshold value TH_W, visual disturbance and discomfort are reduced. Further, since the maximum luminance value gradually increases from the luminance value PH2 to the luminance value PH1, the brightness, the force, and the like are gradually increased.
Note that it is not an essential process to limit the fluctuation range to a value less than the threshold value TH_W. “The visual disturbance and discomfort can be reduced, and the reduction in brightness, power, etc. can be suppressed by simply changing the expressible range gradually over time.” The above effects can also be obtained.

The effect of the present embodiment will be described with reference to FIGS. 8 (A) to 8 (C).
FIG. 8A shows an example of an input image. In the example of FIG. 8A, the increase / decrease in the luminance of the input image over time is repeated. Here, it is assumed that the scene including the frames 1 to 4 in FIG. 8A is detected by the blinking scene detection unit 202 as a blinking scene (a scene satisfying a predetermined condition).

  As described above, conventionally, the fluctuation range is reduced by reducing the maximum luminance value, and the visual disturbance and discomfort are reduced. For this reason, as shown in FIG. 8B, the blinking scene of FIG. 8A is always converted into a scene with low luminance.

  According to the present embodiment, as shown in FIG. 8C, the fluctuation range is reduced, so that visual disturbance and discomfort can be reduced. By gradually changing the representable range so that the luminance of the image is gradually increased, it is possible to gradually increase the feeling of brightness, force, and the like.

  Note that the various processes described in the present embodiment may or may not be performed in units of frames. For example, the processing described in the present embodiment may be performed for each of a plurality of blocks constituting the area of the input image. That is, the processing described in this embodiment may be performed for each block. One pixel region may be treated as a block, or a region composed of a plurality of pixels may be treated as a block. When a display device having a light emitting unit capable of individually controlling light emission luminance is used for each of a plurality of partial areas constituting a screen area, the image area to be displayed in the partial area is used as a block. Also good. FIG. 8C shows an example in which processing is performed in units of pixels, and FIG. 8D shows an example in which processing is performed in units of frames. 8 (C) and 8 (D), it can be seen that the same effects as those described above can be obtained even if processing is performed in units of frames. If processing is performed in units of pixels (block units), it is possible to suppress the correction of the luminance value in an area that does not require correction according to the present embodiment (an area that does not require correction), thereby representing a higher quality image. Image data can be obtained.

  It should be noted that the rate of change when the representable range is gradually changed is preferably determined based on the adaptation speed of human vision. In human vision, the recognizable dynamic range varies depending on the brightness of the visual field. When the blinking speed is faster than the visual adaptation speed, a human cannot accurately recognize the brightness of the image. Therefore, it is preferable to change the representable range so as to match the change in the recognizable dynamic range. Thereby, visual disturbance and discomfort can be reduced without causing the user to feel uncomfortable.

  It should be noted that the input image data of the blinking scene (scene satisfying a predetermined condition) is corrected so that the representable range periodically changes in a cycle longer than the cycle in which the increase / decrease in luminance of the image is repeated (flashing cycle). May be. FIG. 11A shows an example of temporal change in luminance of the input image. 11B and 11C show an example of a temporal change in luminance of an image in which the representable range is periodically changed in a cycle longer than the blinking cycle in FIG. FIG. 11B shows an example in which the fluctuation range is limited to a value less than the threshold TH_W, and FIG. 11C shows an example in which the fluctuation range is also changed in a cycle longer than the blinking cycle. By periodically changing the representable range with a period longer than the blinking period, the blinking frequency (flickering frequency) can be reduced in a pseudo manner. In the range of the flicker frequency where human vision can recognize flicker, the higher the flicker frequency, the greater the visual disturbance and discomfort. Therefore, by visually reducing the blinking frequency, it is possible to reduce visual interference and discomfort. In addition, since a higher brightness value than the conventional brightness value can be used as the maximum brightness value of the blinking scene, it is possible to obtain image data having a higher brightness, power, and the like of the blinking scene than before.

Note that the various threshold values described in the present embodiment may be fixed values predetermined by a manufacturer or the like, or may be values preset by a user (changeable values). The various threshold values described in this embodiment may be values that are automatically determined and changed by an image processing apparatus or the like.

<Example 3>
Hereinafter, an image processing apparatus and an image processing method according to Embodiment 3 of the present invention will be described.
As in the first embodiment, HDR image data can be input to the image processing apparatus according to the present embodiment.

  In this embodiment, the spatial frequency and luminance of the input image are analyzed. As a result, the input image area is an area where the user has a good impression that “realism, glossiness, shine, power, etc. are high” with respect to “glare”, or “glare” is determined by the user. It is determined whether the area is recognized as obstruction. Based on the determination result of the area, the luminance of the input image data is corrected to a suitable luminance.

FIG. 13 is a block diagram illustrating an example of a functional configuration of the image processing apparatus according to the present embodiment.
Similar to the first embodiment, the image processing apparatus according to the present embodiment performs image processing on input image data and outputs the image data subjected to the image processing to the display unit 104. The display unit 104 in FIG. 13 is the same as the display unit 104 in FIG.

  The block feature quantity acquisition unit 301 inputs, for each of a plurality of blocks constituting the area of the input image, spatial frequency information related to the spatial frequency of the input image in the block and a feature quantity related to the luminance of the input image in the block. Obtain from image data. Then, the block feature amount acquisition unit 301 outputs the spatial frequency information and the feature amount of each block to the high luminance region determination unit 302. In this embodiment, the maximum luminance value (maximum luminance value) of the input image data in the block is acquired as the feature amount of the block. Further, by performing general edge detection processing using image data in a block, edge image data is acquired as spatial frequency information of the block. The edge detection process is a process for detecting an edge existing in the image, and the edge image data is image data representing the detected edge.

The spatial frequency information of the block is not limited to edge image data. For example, a coefficient (DCT coefficient) may be acquired as the spatial frequency information of the block by performing discrete cosine transform (DCT) on the image data in the block.
Note that the feature amount of a block is not limited to the maximum luminance value of image data in the block. For example, the minimum value, mode value, intermediate value, average value, and the like of the luminance value of the input image data in the block may be acquired as the feature amount of the block. A histogram (luminance histogram) of luminance values of input image data in a block may be acquired as the feature amount of the block.

  The high luminance area determination unit 302 determines what kind of scene the input image is based on the spatial frequency information and the feature amount acquired by the block feature amount acquisition unit 301. Specifically, the high luminance region determination unit 302 determines whether or not a high luminance region that is a non-attention region exists in the input image based on the spatial frequency information and the feature amount acquired by the block feature amount acquisition unit 301. Judging. Then, the high brightness area determination unit 302 outputs the determination result to the image correction unit 303. Details of the processing of the high luminance area determination unit 302 will be described later.

  The image correction unit 303 corrects the input image data based on the determination result of the high brightness area determination unit 302. In this embodiment, when it is determined that “a high luminance area that is a non-attention area exists in the input image”, the image correction unit 303 determines that the luminance value of the pixels that constitute the high luminance area that is a non-attention area is The input image data is corrected so as to be reduced.

  The high luminance region determination unit 302 includes a spatial frequency analysis unit 305 and a non-attention region detection unit 306.

For each of the plurality of blocks, the spatial frequency analysis unit 305 determines whether the spatial frequency Sf of the input image in the block includes a high frequency component based on the spatial frequency information acquired for the block. The high frequency component is a spatial frequency equal to or higher than a threshold value TH_Sf (first threshold value). In the present embodiment, the spatial frequency analysis unit 305, when the spatial frequency of the input image in the block includes a high-frequency component in an amount equal to or greater than a threshold TH_Sa (third threshold), the spatial frequency analysis unit 305 Detect as. Specifically, when the total number of edge pixels (pixels constituting the edge) in the block is equal to or greater than a threshold value TH_Sa, the spatial frequency analysis unit 305 detects the block as a block whose spatial frequency includes a high frequency component. . Then, the spatial frequency analysis unit 305 outputs the processing result to the non-attention area detection unit 306. For example, information indicating a block whose spatial frequency includes a high frequency component is output as a processing result.
Note that a block whose spatial frequency includes a high frequency component may be detected regardless of whether the amount of the high frequency component is large.

  The non-attention area detection unit 306 determines, for each of the plurality of blocks, whether the block satisfies a predetermined condition. Then, the non-attention area detection unit 306 outputs the determination result to the image correction unit 303. For example, information representing a block that satisfies a predetermined condition is output as a determination result. In this embodiment, based on the processing result of the spatial frequency analysis unit 305 and the feature amount acquired by the block feature amount acquisition unit 301, it is determined whether or not the block satisfies a predetermined condition.

In this embodiment, the non-attention area detection unit 306 determines that a predetermined condition is satisfied for a block that satisfies the first predetermined condition including the following condition 1 and condition 2. The first predetermined condition may further include conditions other than the conditions 1 and 2.
Condition 1: The spatial frequency of the image includes a high frequency component Condition 2: The luminance of the image is higher than the threshold value TH_B (second threshold value) As described above, in this embodiment, the spatial frequency of the input image in the block is the threshold value. When a high frequency component of an amount equal to or higher than TH_Sa is included, the block is detected as a block whose spatial frequency includes a high frequency component. Therefore, in the present embodiment, Condition 1 is a condition that “the spatial frequency of the image includes a high-frequency component in an amount equal to or greater than the threshold value TH_Sa”.

Further, in this embodiment, the non-attention area detection unit 306 further determines that a predetermined condition is satisfied for a block that satisfies the second predetermined condition including the above condition 2 and the following condition 3. The second predetermined condition may further include conditions other than the conditions 2 and 3.
Condition 3: Adjacent to a block satisfying the first predetermined condition From the viewpoint of image quality, it is preferable that the predetermined condition is also determined for a block satisfying the second predetermined condition, but the second predetermined condition is satisfied. The block may not be determined to satisfy the predetermined condition.

In this embodiment, a pixel area having a luminance value higher than the reference value W (fourth threshold value) included in a block that satisfies a predetermined condition is treated as a high-luminance area that is an attention area. Then, an area of a pixel having a luminance value higher than the reference value W included in a block that does not satisfy the predetermined condition is treated as a high luminance area that is a non-attention area. For this reason, the process of determining whether or not a block satisfies a predetermined condition can also be referred to as “a process of determining whether or not a high-luminance area that is a non-target area exists in the input image”.
Regardless of whether or not the luminance value is higher than the reference value W, a block that satisfies the predetermined condition may be treated as a high luminance region that is a region of interest. Regardless of whether or not the luminance value is higher than the reference value W, a block that does not satisfy the predetermined condition may be treated as a high luminance region that is a non-attention region.

A specific example of the processing of the image correction unit 303 will be described with reference to FIG. FIG. 14 is a diagram illustrating an example of a correspondence relationship between a luminance value before correction (input luminance value) and a luminance value after correction (output luminance value).
The image correction unit 303 corrects the input image data so that the luminance value of the pixel having a luminance value higher than the reference value W included in the block that does not satisfy the predetermined condition is reduced. Specifically, the image correction unit 303 converts (nonlinear conversion) the luminance values of the pixels included in the block that does not satisfy the predetermined condition according to the correspondence relationship in FIG. According to the correspondence relationship in FIG. 14, the luminance value of a pixel having a luminance value equal to or lower than the reference value W is not converted, and the luminance value of a pixel having a luminance value higher than the reference value W is reduced.
The correspondence relationship between the luminance value before correction and the luminance value after correction is not limited to the correspondence relationship in FIG.

  According to the present embodiment, a pixel area having a luminance value higher than the reference value W included in a block that does not satisfy the predetermined condition is treated as a high luminance area that is a non-attention area, and a high area that is a non-attention area. The luminance value of the pixels constituting the luminance area is reduced. Thereby, it is possible to reduce the “glare” that the user recognizes as an obstruction. Further, since the other luminance values are not corrected, it is possible to obtain image data that provides “glare” with a good impression that the user has “high reality, gloss, shine, power, etc.”.

The effect of the present embodiment will be specifically described with reference to FIG.
In the image shown in FIG. 12 (A), there is an area of performers who are spotlighted on the stage. The spatial frequency of the performer's area includes high frequency components, and the brightness of the performer's area is high. Therefore, in the present embodiment, when the image data of FIG. 12A is input to the image processing apparatus, the brightness value of the performer's area is not reduced, and the reality, glossiness, brightness, force, Can be obtained.
The image shown in FIG. 12B includes a sun region and a person region. The spatial frequency of the solar region does not include high frequency components, and the spatial frequency of the human region includes high frequency components. In addition, the brightness of the sun region is high. Therefore, in the present embodiment, when the image data of FIG. 12B is input to the image processing apparatus, as shown in FIG. The luminance value is reduced. As a result, the “glare” of the sun region (“glare” recognized by the user as an obstruction) is reduced, and the “glare” of the person region (the region where the user is likely to focus) is maintained. Obtained image data can be obtained.

  Note that the various threshold values described in the present embodiment may be fixed values predetermined by a manufacturer or the like, or may be values preset by a user (changeable values). The various threshold values described in this embodiment may be values that are automatically determined and changed by an image processing apparatus or the like.

  Note that the luminance of the high-brightness area that is the attention area may be reduced without reducing the luminance of the high-luminance area that is the non-attention area. For example, the input image data may be corrected so that the luminance value of a pixel having a luminance value higher than the reference value W included in the block that satisfies the predetermined condition is reduced.

<Example 4>
Hereinafter, an image processing apparatus and an image processing method according to Embodiment 4 of the present invention will be described.
As in the first embodiment, HDR image data can be input to the image processing apparatus according to the present embodiment.

  In this embodiment, a specific object area (an area of an image of a predetermined object) is detected from the area of the input image. Then, based on the detection result, whether the area of the input image is an area where the user has a good impression that “realism, glossiness, brightness, power, etc. are high” with respect to “glare”. It is determined whether or not the user recognizes “glare” as an obstruction. Thereafter, based on the determination result, the luminance of the input image data is corrected to a suitable luminance.

FIG. 15 is a block diagram illustrating an example of a functional configuration of the image processing apparatus according to the present embodiment.
Similar to the first embodiment, the image processing apparatus according to the present embodiment performs image processing on input image data and outputs the image data subjected to the image processing to the display unit 104. The display unit 104 in FIG. 15 is the same as the display unit 104 in FIG.

The block feature amount acquisition unit 401 acquires, from the input image data, a feature amount relating to the luminance of the input image in each block for each of a plurality of blocks constituting the region of the input image. Then, the block feature amount acquisition unit 401 outputs the feature amount of each block to the high luminance region determination unit 402. In this embodiment, the maximum luminance value of the input image data in the block is acquired as the feature amount of the block.
Note that the feature amount of a block is not limited to the maximum luminance value of image data in the block. For example, the minimum value, mode value, intermediate value, average value, and the like of the luminance value of the input image data in the block may be acquired as the feature amount of the block. A histogram (luminance histogram) of luminance values of input image data in a block may be acquired as the feature amount of the block.

The high brightness area determination unit 402 determines what scene image the input image is based on the feature amount acquired by the block feature amount acquisition unit 401. Specifically, the high luminance area determination unit 402 determines whether or not a high luminance area that is a non-attention area exists in the input image based on the feature amount acquired by the block feature amount acquisition unit 401. Then, the high brightness area determination unit 402 outputs the determination result to the image correction unit 303. Details of the processing of the high luminance area determination unit 402 will be described later.
The image correction unit 303 in FIG. 15 is the same as the image correction unit 303 in FIG.

  The high brightness area determination unit 402 includes an object detection unit 405 and a non-attention area detection unit 406.

  The object detection unit 405 detects a specific object area from the area of the input image based on the input image data. In the present embodiment, a face image area (face area) is detected from the input image area. Then, the object detection unit 405 outputs the detection result to the non-attention area detection unit 406. For example, information representing a specific object area is output as a detection result.

  Note that the process of detecting the specific object area may be performed in units of images (frame units) or in units of blocks.

The specific object area detection method is not particularly limited. For example, the specific object region can be detected using pattern matching. Specifically, a skin color region is detected from the input image region, and the similarity between the input image in the detected skin color region and the face image (average face image) prepared in advance is determined by pattern matching. You may decide. If the determined similarity is equal to or greater than the threshold, it is determined that “the detected skin color region is a face region”, and if the determined similarity is less than the threshold, “the detected skin color region It may be determined that “is not a face area”.

  The predetermined object is not limited to the “face”. For example, a character image region (character region), a light emitter (sun, illumination, etc.) image region (light emitter region), and the like may be detected as the specific object region. The character area can be detected by using an OCR technique often used in a scanner, a copier, or the like. The character area can also be detected using pattern matching. The light emitter region can be detected using, for example, pattern matching. Specifically, an area having a brightness higher than the threshold value with respect to the average brightness of the image may be detected. Then, the similarity between the input image in the detected region and a previously prepared light emitter image (light emitter image) may be determined by pattern matching. Thereafter, when the determined similarity is equal to or greater than the threshold, it is determined that “the detected region is a light emitter region”, and when the determined similarity is less than the threshold, “the detected region is a light emitter. It may be determined that it is not an area. Usually, the shape of the sun seen from the earth does not change much. Therefore, if sun images (pattern images) at several times such as dawn, daytime, and dusk are prepared, the sun region can be detected by pattern matching. When detecting a plurality of specific object regions having different object types, for example, information indicating the detected specific object region and its type (object type) is output from the object detection unit 405.

  The non-attention area detection unit 406 determines, for each of the plurality of blocks, whether the block satisfies a predetermined condition. Then, the non-attention area detection unit 406 outputs the determination result to the image correction unit 303. For example, information representing a block that satisfies a predetermined condition is output as a determination result. In the present embodiment, it is determined whether or not the block satisfies a predetermined condition based on the detection result of the object detection unit 405 and the feature amount acquired by the block feature amount acquisition unit 401.

In the present embodiment, the non-attention area detection unit 406 determines that a predetermined condition is satisfied for a block that satisfies the first predetermined condition including the following conditions 1 and 2. The first predetermined condition may further include conditions other than the conditions 1 and 2.
Condition 1: includes a specific object area (face area) Condition 2: the luminance of the image is higher than the threshold value TH_B (first threshold value)

In the present embodiment, the non-attention area detection unit 406 further determines that a predetermined condition is satisfied for a block that satisfies the second predetermined condition including the above condition 2 and the following condition 3. The second predetermined condition may further include conditions other than the conditions 2 and 3.
Condition 3: Adjacent to a block satisfying the first predetermined condition From the viewpoint of image quality, it is preferable that the predetermined condition is also determined for a block satisfying the second predetermined condition, but the second predetermined condition is satisfied. The block may not be determined to satisfy the predetermined condition.

In the present embodiment, a face area that is highly likely to be noticed by the user is used as the specific object area that is the target of Condition 1. In this embodiment, a pixel area having a luminance value higher than the reference value W (second threshold value) included in the block that satisfies the predetermined condition is treated as a high luminance area that is an attention area. In addition, a region of a pixel having a luminance value higher than the reference value W included in a block that does not satisfy the predetermined condition is treated as a high luminance region that is a non-attention region. For this reason, the process of determining whether or not a block satisfies a predetermined condition can also be referred to as “a process of determining whether or not a high-luminance area that is a non-target area exists in the input image”.
Regardless of whether or not the luminance value is higher than the reference value W, a block that satisfies the predetermined condition may be treated as a high luminance region that is a region of interest. Regardless of whether or not the luminance value is higher than the reference value W, a block that does not satisfy the predetermined condition may be treated as a high luminance region that is a non-attention region.

  As described above, in this embodiment, an object area that is highly likely to be noticed by the user is used as the specific object area that is the target of the condition 1. However, as described above, a plurality of specific object regions having different object types may be detected. In such a case, for example, a block satisfying the first predetermined condition may be detected by performing the following processing.

First, the non-attention area detection unit 406 assigns importance (attention level) according to importance information prepared in advance to the detected specific object area. The importance level information represents the importance level for each object type. Since the user has a high possibility of paying attention to the face area, the character area, etc., the importance “high” is assigned, and the light emitting area, etc. is unlikely to be noticed by the user. , Importance “low” is assigned.
Next, the non-attention area detection unit 406 determines whether the luminance of the input image in the block assigned the importance “high” is higher than the threshold value TH_B.
Then, the non-attention area detection unit 406 detects a block to which importance “high” is assigned and the image brightness is higher than the threshold TH_B as a block satisfying the first predetermined condition.

  Thereafter, image processing (correction) is performed as in the third embodiment.

  According to the present embodiment, a pixel area having a luminance value higher than the reference value W included in a block that does not satisfy the predetermined condition is treated as a high luminance area that is a non-attention area, and a high area that is a non-attention area. The luminance value of the pixels constituting the luminance area is reduced. Thereby, it is possible to reduce the “glare” that the user recognizes as an obstruction. Further, since the other luminance values are not corrected, it is possible to obtain image data that provides “glare” with a good impression that the user has “high reality, gloss, shine, power, etc.”.

The effect of the present embodiment will be specifically described with reference to FIG.
In the image shown in FIG. 12 (A), there is an area of performers who are spotlighted on the stage. In the present embodiment, when the image data of FIG. 12A is input to the image processing apparatus, the entire performer region, the performer's face region, etc. are detected as specific object regions that are likely to be noticed by the user. Is done. Therefore, the brightness value of the performer's area is not reduced, and it is possible to obtain image data with high reality, gloss, shine, power, etc. of the performer's area.
The image shown in FIG. 12B includes a sun region and a person region. In this embodiment, when the image data of FIG. 12B is input to the image processing apparatus, the sun area is detected as a specific object area that is less likely to be noticed by the user, or the person area is the user. May be detected as a specific object region that is likely to be noticed. Therefore, as shown in FIG. 12C, the luminance value of the person region is not reduced, and the luminance value of the sun region is reduced. As a result, the “glare” of the sun region (“glare” recognized by the user as an obstruction) is reduced, and the “glare” of the person region (the region where the user is likely to focus) is maintained. Obtained image data can be obtained.

  Note that the various threshold values described in the present embodiment may be fixed values predetermined by a manufacturer or the like, or may be values preset by a user (changeable values). The various threshold values described in this embodiment may be values that are automatically determined and changed by an image processing apparatus or the like.

Note that the luminance of the high-brightness area that is the attention area may be reduced without reducing the luminance of the high-luminance area that is the non-attention area. For example, the input image data may be corrected so that the luminance value of a pixel having a luminance value higher than the reference value W included in the block that satisfies the predetermined condition is reduced.

  Note that, as the specific object area that is the target of the condition 1, an object area that is not likely to be noticed by the user may be used instead of an object area that is likely to be noticed by the user.

<Matters common to Examples 1 to 4>
In the first to fourth embodiments, the example in which the image processing apparatus outputs image data to the display device has been described. However, the present invention is not limited to this. The image processing apparatus may record the corrected image data in a storage device (magnetic disk, optical disk, semiconductor memory, etc.) instead of the display device. The storage device may be a device that can be attached to and detached from the image processing device, or a device that is not removable from the image processing device (a device built in the image processing device). The image processing apparatus may generate image data for editing instead of image data for display. If image data for editing is generated by the image processing device, the effort required by the image editor when creating high-quality image data in consideration of user safety is reduced, and the editing work performed by the image editor is efficient. Can be As the image processing apparatus, a dedicated apparatus may be prepared or not. The PC may function as an image processing apparatus by executing a software program (application program) for performing the processing described in the first to fourth embodiments on a general personal computer (PC).

  The effect when using the light emitting part capable of locally changing the light emission luminance as the light emitting part of the display device will be described. If such a light emitting unit is used, the dynamic range of the display image can be improved in units of partial areas. However, the dynamic range of the display image in the partial area does not exceed the contrast ratio of the display unit (display panel). This contrast ratio corresponds to a display luminance range that can be realized only by modulating the light emitted from the light emitting section. Therefore, when the light emission luminance of the partial area is controlled to a high value, a high value can be obtained as the display luminance of a bright image, but a high value can be obtained as the display luminance of a dark image. For this reason, a display image with degraded image quality may be obtained. Specifically, a display image in which black float, halo, or the like has occurred may be obtained. According to the first to fourth embodiments, the luminance value is set to a high value only for the high luminance region that is the attention region, or the luminance value is set to a high value only for the high luminance region that is the non-attention region. . Thereby, it is possible to suppress the emission luminance of the partial area from being unnecessarily increased, and to suppress the deterioration of the image quality. For example, if the configuration is such that the luminance value of the high luminance area that is the attention area is set to a high value and the luminance value of the high luminance area that is the non attention area is set to a high value, The light emission brightness is set to a low value. Thereby, it is possible to suppress the occurrence of black floating or halo in the high-luminance region which is a non-attention region or the periphery thereof.

<Other examples>
The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

101, 201: Feature amount acquisition unit 102, 302, 402: High luminance region determination unit 103, 203, 303: Image correction unit 105: Cluster detection unit 106, 306, 306: Non-attention region detection unit 202: Blink scene detection unit 301, 401: Block feature quantity acquisition unit 305: Spatial frequency analysis unit 405: Object detection unit

Claims (25)

Acquisition means for acquiring, from each input image data, a luminance histogram representing, as a frequency, the total number of pixels having luminance values belonging to the category for each category that is a luminance range including one or more luminance values;
Detecting means for detecting a cluster which is a luminance range composed of one or more categories whose frequency is equal to or greater than a first threshold from the luminance histogram;
Determining means for determining whether or not the cluster detected by the detecting means satisfies a predetermined condition;
Correction means for correcting the input image data based on the determination result of the determination means;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: consists of a number of categories equal to or less than the second threshold. Condition 2: an image processing apparatus including a luminance value higher than the third threshold. The image processing apparatus according to claim 1, wherein the predetermined condition further includes the following condition 3: Condition 3: a luminance value equal to or less than the third threshold value is not included. The image processing apparatus according to claim 1, wherein the correction unit omits the process of correcting the input image data when the total number of clusters detected by the detection unit is one. . Smoothing means for obtaining image data representing an image obtained by spatially smoothing an image represented by the input image data by performing a smoothing process on the input image data;
The image processing apparatus according to claim 1, wherein the acquisition unit acquires the luminance histogram from the image data generated by the smoothing unit.   The correction means corrects the input image data so that a luminance value of a pixel having a luminance value higher than the third threshold among luminance values belonging to a cluster satisfying the predetermined condition is reduced. The image processing apparatus according to any one of claims 1 to 4. The correction unit corrects the input image data so that a luminance value of a pixel having a luminance value higher than the third threshold among luminance values belonging to a cluster that does not satisfy the predetermined condition is reduced. The image processing apparatus according to any one of claims 1 to 4. For each of a plurality of blocks constituting the image area represented by the input image data, the spatial frequency information related to the spatial frequency of the image in the block and the feature quantity related to the luminance of the image in the block are acquired from the input image data. Acquisition means;
For each of the plurality of blocks, determination means for determining whether the block satisfies a predetermined condition based on the spatial frequency information and the feature amount acquired for the block;
Correction means for correcting the input image data based on the determination result of the determination means;
Have
The determination means determines that the predetermined condition is satisfied for a block satisfying the first predetermined condition including the following condition 1 and condition 2: Condition 1: The spatial frequency of the image is a spatial frequency equal to or higher than the first threshold value. A certain high-frequency component is included. Condition 2: The image processing apparatus is characterized in that the luminance of the image is higher than a second threshold value. The image processing apparatus according to claim 7, wherein the condition 1 is a condition that “the spatial frequency of the image includes a high-frequency component in an amount equal to or greater than a third threshold value”. The determination means further determines that the predetermined condition is satisfied for a block satisfying the second predetermined condition including the condition 2 and the following condition 3. Condition 3: Adjacent to the block satisfying the first predetermined condition The image processing apparatus according to claim 7, wherein the image processing apparatus is an image processing apparatus. The correction means corrects the input image data so that a luminance value of a pixel having a luminance value higher than a fourth threshold included in the block satisfying the predetermined condition is reduced. The image processing apparatus according to any one of 7 to 9. The correction means corrects the input image data so that a luminance value of a pixel having a luminance value higher than a fourth threshold included in a block that does not satisfy the predetermined condition is reduced. Item 10. The image processing apparatus according to any one of Items 7 to 9. For each of a plurality of blocks constituting the region of the image represented by the input image data, an acquisition unit that acquires, from the input image data, a feature amount related to the luminance of the image in the block;
Detection means for detecting an image area of a predetermined object from an image area represented by the input image data;
A determination unit that determines, for each of the plurality of blocks, whether or not the block satisfies a predetermined condition based on the feature amount acquired for the block and the area detected by the detection unit; ,
Correction means for correcting the input image data based on the determination result of the determination means;
Have
The determination means determines that the predetermined condition is satisfied for a block satisfying a first predetermined condition including the following condition 1 and condition 2: Condition 1: includes an image area of the predetermined object Condition 2: Image The image processing apparatus is characterized in that the luminance of is higher than the first threshold. The determination means further determines that the predetermined condition is satisfied for a block satisfying the second predetermined condition including the condition 2 and the following condition 3. Condition 3: Adjacent to the block satisfying the first predetermined condition The image processing apparatus according to claim 12, wherein: The image processing apparatus according to claim 12, wherein the predetermined object includes a light emitter. The image processing apparatus according to claim 12, wherein the predetermined object includes at least one of a face and a character. The said correction | amendment means correct | amends the said input image data so that the luminance value of the pixel which is contained in the block which satisfy | fills the said predetermined condition and which has a luminance value higher than a 2nd threshold value is reduced. The image processing device according to any one of 12 to 15. The correction means corrects the input image data so that a luminance value of a pixel having a luminance value higher than a second threshold included in a block that does not satisfy the predetermined condition is reduced. Item 16. The image processing device according to any one of Items 12 to 15. Acquisition means for acquiring, for each frame, a feature amount related to the brightness of the image represented by the input image data of the frame;
Detecting means for detecting a scene satisfying a predetermined condition as a blinking scene in which the luminance of the image is repeatedly increased and decreased based on the feature amount acquired by the acquiring means;
Correction means for correcting the input image data of the blinking scene detected by the detection means so that the representable range that is the luminance range from the lower limit value to the upper limit value of the luminance value gradually changes in time;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: The frequency at which the luminance of the image fluctuates within a fluctuation range of the first threshold or higher is the second threshold or higher. Condition 2: The maximum value of the luminance of the image is An image processing apparatus characterized by being higher than a third threshold. The image processing apparatus according to claim 18, wherein the correction unit corrects the input image data of the blinking scene so that the fluctuation range is limited to a value less than the first threshold value. The correction means corrects the input image data of the blinking scene so that the representable range periodically changes in a cycle longer than a cycle in which the increase / decrease in luminance of the image is repeated. The image processing apparatus according to claim 18 or 19. Obtaining a luminance histogram from the input image data for each category that is a luminance range including one or more luminance values, representing the total number of pixels having luminance values belonging to that category as a frequency;
Detecting from the luminance histogram a cluster that is a luminance range consisting of one or more categories whose frequency is equal to or greater than a first threshold;
A determination step of determining whether or not the cluster detected in the detection step satisfies a predetermined condition;
A correction step of correcting the input image data based on the determination result of the determination step;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: consists of a number of categories equal to or less than the second threshold. Condition 2: an image processing method including a luminance value higher than the third threshold. For each of a plurality of blocks constituting the image area represented by the input image data, the spatial frequency information related to the spatial frequency of the image in the block and the feature quantity related to the luminance of the image in the block are acquired from the input image data. An acquisition step;
For each of the plurality of blocks, a determination step of determining whether or not the block satisfies a predetermined condition based on the spatial frequency information and the feature amount acquired for the block;
A correction step of correcting the input image data based on the determination result of the determination step;
Have
In the determination step, it is determined that the predetermined condition is satisfied for a block satisfying the first predetermined condition including the following conditions 1 and 2. Condition 1: The spatial frequency of the image is a spatial frequency equal to or higher than the first threshold. Condition 2: including a certain high-frequency component: An image processing method, wherein the brightness of the image is higher than a second threshold value. For each of a plurality of blocks constituting the region of the image represented by the input image data, an acquisition step of acquiring, from the input image data, a feature amount related to the luminance of the image in the block;
A detection step of detecting an image area of a predetermined object from an image area represented by the input image data;
For each of the plurality of blocks, a determination step for determining whether or not the block satisfies a predetermined condition based on the feature amount acquired for the block and the region detected in the detection step; ,
A correction step of correcting the input image data based on the determination result of the determination step;
Have
In the determination step, it is determined that the predetermined condition is satisfied for a block satisfying the first predetermined condition including the following condition 1 and condition 2: Condition 1: including an image area of the predetermined object Condition 2: Image The image processing method is characterized in that the luminance of the image is higher than the first threshold value. For each frame, an acquisition step of acquiring a feature amount related to the brightness of the image represented by the input image data of the frame;
A detection step of detecting a scene satisfying a predetermined condition as a flickering scene in which an increase / decrease in the luminance of the image is repeated based on the feature amount acquired in the acquisition step;
A correction step of correcting the input image data of the blinking scene detected in the detection step so that an expressible range that is a luminance range from the lower limit value to the upper limit value of the luminance value gradually changes in time;
Have
The predetermined condition includes the following condition 1 and condition 2: Condition 1: The frequency at which the luminance of the image fluctuates within a fluctuation range of the first threshold or higher is the second threshold or higher. Condition 2: The maximum value of the luminance of the image is An image processing method characterized by being higher than a third threshold.   A program that causes a computer to execute each step of the image processing method according to any one of claims 21 to 24.

Images