JP5381565B2 - Image processing apparatus, image processing program, and image processing method - Google Patents

Description

  The present invention relates to an image processing apparatus, an image processing program, and an image processing method for performing image processing for backlight correction on an input digital image signal in accordance with the content of the image signal.

  With the recent spread of digital cameras, there is an increasing demand for photographing with digital cameras and printing of photographed images. However, digital cameras can be handled easily even by users who do not have enough knowledge about the cameras, and because they take pictures under various environments, they are not necessarily suitable for images with strong contrast such as backlight. Not only images taken under conditions. Even if the image was taken under unfavorable conditions, if the content is important to the photographer, it cannot be discarded and can be stored or printed in a suitable state after appropriate image correction. desired.

  In the case of an image having a person as a subject, a conventional image processing apparatus determines backlight from the brightness of an important subject area and other areas in the image, such as a person's face area, and the face area is predetermined. The image was processed so that the brightness was. In addition, in an image in which the region of interest is not clear or an image in which it is difficult to automatically detect an important region, whether or not the image is backlit is determined from the luminance histogram, and the degree of backlight is determined. Based on the image processing.

  In a conventional image processing apparatus, in order to perform backlight determination, in a luminance histogram divided into 16 classes, a low luminance region and a high luminance region are set, and backlight is determined from the distance between peaks in each luminance region. (For example, refer to Patent Document 1).

  Further, only an image that is known in advance as backlight is targeted, the degree of bipolarization is calculated from the luminance histogram, the degree of backlight is obtained, and image quality correction according to the degree is performed (see, for example, Patent Document 2). .)

JP 2008-118608 A JP 2002-092607 A

  In the conventional image processing apparatus, it is difficult to accurately calculate the characteristics of the luminance histogram of the backlight image when performing the backlight determination of the image from the luminance histogram. As a result, there is a problem that an image with a sense of incongruity is generated.

  In the conventional image processing apparatus, the backlight is determined based on the distance between the peaks of each of the low-luminance area and the high-luminance area. Therefore, the number of peaks is low in both the low-luminance area and the high-luminance area. If there are a plurality of peaks, it is not known which distance between peaks should be used, and there is a problem that a peak to be used for backlight determination cannot be detected correctly.

  In addition, when image quality correction is performed according to the degree of backlight obtained from the degree of polarization of the luminance histogram, since the amount of change in the luminance histogram is thresholded when calculating the degree of polarization, the luminance histogram There is a problem in that an image in which the image changes smoothly differs from an image in which the image changes sharply, which causes a false determination. This is because the amount of change in the luminance histogram varies depending on the size of the subject and does not depend on whether it is backlit. Since all of these judge the mountain part locally, there was a problem of making an erroneous judgment when viewed globally.

  The present invention has been made to solve the above-described problems, and provides an image processing apparatus capable of extracting a general rough feature of the shape of a histogram regardless of the position and change amount of the peak of the histogram. It is.

  In the image processing apparatus of the present invention, the histogram calculation means for creating a histogram of the luminance signal from the input image signal, and detecting a continuous class range from the histogram as a mountain portion, Histogram feature extraction means for calculating the interval between the peaks and the number of peaks as features of the histogram, backlight determination means for determining whether or not the image signal is backlight from the features of the histogram, And an image correction unit that corrects the image signal based on a result of the backlight determination.

  According to the present invention, it is possible to extract a general schematic feature of the shape of the histogram from the input image signal regardless of the position of the histogram peak and the amount of change.

It is a block diagram which shows Embodiment 1 of this invention. It is explanatory drawing for demonstrating operation | movement of the peak detection means in Embodiment 1 of this invention. It is explanatory drawing for demonstrating operation | movement of the peak detection means in Embodiment 1 of this invention. It is a figure which shows an example of the characteristic which the characteristic calculation means in Embodiment 1 of this invention extracts. It is explanatory drawing for demonstrating the characteristic of the histogram of a backlight image in Embodiment 1 of this invention. It is a flowchart which shows operation | movement of the backlight determination means in Embodiment 1 of this invention. It is a figure which shows the gamma curve which the image correction means in Embodiment 1 of this invention uses for image correction. It is a block diagram which shows Embodiment 2 of this invention. It is a flowchart which shows operation | movement of the backlight determination means in Embodiment 2 of this invention.

Embodiment 1 FIG.
FIG. 1 shows a block diagram of an image processing apparatus according to Embodiment 1 for carrying out the present invention. In the figure, an image processing apparatus includes a histogram calculation unit 1 that calculates a histogram of an input image signal, a histogram feature extraction unit 2 that extracts features relating to the shape of the histogram from the histogram calculated by the histogram calculation unit 1, and a histogram feature. Based on the characteristics of the histogram output from the extraction unit 2, the backlight determination unit 3 determines whether or not the input image is backlight, and the image correction unit 4 of the image signal input based on the determination result of the backlight determination unit 3 Prepare.

  Further, the histogram feature extraction means 2 detects a continuous class range having a frequency greater than the surroundings as a peak from the histogram, and calculates the feature of the shape of the histogram. Specifically, the peak detection means 21 detects the peak value of the distribution of the histogram as a peak from the histogram output from the histogram calculation means 1, and the peak frequency around the peak detected by the peak detection means 21 from the histogram. A peak detection means 22 for detecting a class (horizontal axis) range having a frequency equal to or higher than a predetermined ratio as a peak, and a feature for calculating a feature relating to the shape of the histogram based on the histogram, peak information, and peak information. Calculation means 23 is provided.

  Next, the operation will be described. First, the histogram calculation means 1 creates a histogram of the input image signal. The input image signal is preferably a signal representing brightness, such as a luminance signal. Hereinafter, in the present embodiment, the description will be made assuming that the histogram calculation unit 1 calculates the histogram of the luminance signal, but it may be a color difference signal or an RGB signal.

  Next, in order to eliminate frequency variation due to image size, normalization of the histogram is performed, and the normalized luminance histogram is sent to the histogram feature extraction means 2.

The histogram feature extraction unit 2 calculates features representing the shape of the histogram, such as a distribution bias and a less frequent class, with respect to the histogram output from the histogram calculation unit 1. The histogram feature extraction unit 2 includes a peak detection unit 21, a peak detection unit 22, and a feature calculation unit 23. In the following, the operation of each means will be described with reference to FIGS. 2, 3, and 4, taking the case where the luminance histogram of the input image has a shape as shown by the solid line in FIG. The solid line in FIG. 2 is a histogram created with a class width of 1 for a luminance level represented by 256 gradations (0 to 255). Therefore, the luminance level is a class value, and the cumulative number of pixels having each luminance level is the frequency of the histogram. Gradation, the number of steps representing the luminance. When the luminance signal is expressed by 256 gradations, the luminance level takes a value of 0 to 255. In the histogram, the luminance level of 256 gradations is divided by the number of classes, and the number of pixels having the luminance level corresponding to each class becomes the frequency of the histogram. When the number of classes is 8, the lowest class is the luminance level 0-31, and the width of the first class includes 32 gradations of the luminance level.

  FIG. 2 is an explanatory diagram showing an operation in which the peak detecting means 21 detects the maximum value of the histogram distribution as a peak. In the figure, the horizontal axis represents the class value, the vertical axis represents the frequency, and the solid line represents the luminance histogram 100. First, the peak detecting means 21 calculates the slope of the luminance histogram curve. The slope of the luminance histogram curve can be calculated from the difference between adjacent classes. The peak detection means 21 detects a change point at which the slope of the curve of the luminance histogram changes from positive to negative as a peak. For example, in the example of FIG. 2, points 101, 102, 103, 104, and 105 on the curve of the histogram 100 in FIG. 2 are detected as peaks.

  Next, if the frequency is low, such as 102, and the size of the entire histogram distribution is negligible, the peak is excluded from the peaks. Whether or not to exclude is determined by whether or not the frequency at the peak is smaller than a predetermined value. By ignoring the small distribution, the entire feature can be extracted without being affected by the small distribution.

  As described above, the peak detection unit 21 outputs the class and frequency of peaks (101, 103, 104, 105) having a frequency greater than a predetermined value to the peak detection unit 22. Hereinafter, the peak class is simply referred to as the peak position, and the frequency at the peak position is referred to as the peak frequency.

  FIG. 3 is a diagram for explaining the operation of the peak detection means 22. First, the peak detection unit 22 detects a high-frequency class as one peak candidate around the peak position detected by the peak detection unit 21 from the luminance histogram 100 output from the histogram calculation unit 1. To do. Here, the periphery of the peak position is a range of classes that satisfy a predetermined condition continuously on both the higher and lower class values with the peak position as the center. At this time, a class having a frequency equal to or higher than a predetermined ratio with respect to the frequency of the peak position is set as a high frequency class, and a candidate for one mountain portion is set. Assuming that the range around the peak position is up to the adjacent peak position and the predetermined ratio is 50%, the candidate for the peak around the peak 101 is the range 106 in FIG. Similarly, the peak candidate around the peak 103 is the range 107, the peak candidate around the peak 104 is the range 108, and the peak candidate around the peak 105 is the range 109. This means that a class region having a frequency equal to or higher than a predetermined ratio with respect to the maximum value is detected as a peak around the class having the maximum value.

  The periphery of the peak position is not limited to the peak of interest and the adjacent peak position, but may be a continuous class having a frequency of a predetermined ratio or more with respect to the peak frequency. Moreover, it is good also as a range to the nearest peak. Alternatively, a new minimum value of the luminance histogram may be obtained, and the range from the peak position to the minimum value closest to each of the high luminance side and the low luminance side may be used. Depending on the definition of the periphery of the peak position, the range of peak candidates obtained is somewhat different, but there is no problem because the purpose is to determine the peak range as a characteristic of the distribution of the luminance histogram.

Here, the use of a predetermined ratio value with respect to the peak frequency has the following advantages. If a fixed threshold is used when detecting a peak, it may or may not be detected depending on the height of the peak. In addition, when calculating the sharpness (kurtosis) of the distribution, which is a characteristic of the shape of the peak, there is a problem that the kurtosis of the high peak is small and the kurtosis of the low peak is large and cannot be calculated correctly. there were. However, if a predetermined ratio with respect to the peak frequency is used, it is possible to accurately grasp the distribution having the peak as a vertex. For example, if each class value of the pixels included in the peak is x _i , the average luminance of the peak is m, the standard deviation is S, and the cumulative frequency is n, the kurtosis α can be obtained by the following equation.

  Next, the peak detection means 22 identifies the peak from the peak candidates. Among the candidates for the mountain part, if the mountain candidate classes overlap or are in contact with each other as in the range 107 to 109, they are regarded as one mountain part. By using a single peak, it is possible to obtain a global distribution even with a distribution having a large number of histograms and having a plurality of peaks. The peak detection means 22 calculates the maximum class and the minimum class of the range obtained in the above manner, with the range 110 in FIG. 3 as the width of the first peak and the range 111 as the width of the second peak. It outputs to the means 23.

FIG. 4 is an explanatory diagram illustrating an operation in which the feature calculating unit 23 calculates the features of the histogram. The following are calculated as the characteristics of the histogram.
・ Peak frequency of mountain part ・ Width and range of mountain part (maximum class and minimum class)
・ Mount kurtosis ・ Number of ridges ・ Distance between ridges ・ Rough (rough) distribution

  The peak frequency of the peak is the maximum peak frequency among peaks existing in the peak. In the case of FIG. 4, the width of the first peak is the range 110, and the peak frequency of the first peak is the frequency of the peak 101. Further, the width of the second peak is the range 111, and the peak frequency of the second peak is the frequency of the peak 105. Further, the kurtosis of the peak part may be simply (peak frequency of peak part) / (peak part width). The number of peaks in the luminance histogram 100 is 2, and the distance between peaks is the length indicated by the class value difference 112 in FIG. In this method, the calculation process can be reduced as compared with the above equation 1.

  In addition, in order to capture a rough (rough) distribution, the cumulative frequency when the class is divided into several parts such as low luminance, intermediate luminance, and high luminance is calculated. This division method may be simply equal division, or may be divided according to the sensitivity of the eyes. Here, eye sensitivity has a relationship with luminance, and can be an index of image quality setting. For example, the low luminance can be limited to a region where it is easy to feel a difference in black color, or a luminance range where the color is easily felt can be set as an intermediate luminance. In FIG. 4, luminance values 0 to 64 are the low luminance region 113, and luminance values 96 to 196 are the intermediate luminance region 115. At this time, the low luminance frequency is the cumulative frequency of the region indicated by the range 114 in FIG. 4, and the intermediate luminance frequency is the cumulative frequency of the region indicated by the range 116. Here, the cumulative frequency is the sum of the frequencies of classes within a certain range.

  As described above, the characteristics of the histogram extracted through the peak detection unit 21, the peak detection unit 22, and the feature calculation unit 23 are output from the histogram feature extraction unit 2 to the backlight determination unit 3.

  The backlight determination unit 3 determines whether the input image is backlight from the features output from the histogram feature extraction unit 2. Below, the characteristic of the histogram of a backlight image determined by the backlight determination means 3 is demonstrated.

A backlight image is an image obtained by photographing a subject with a back of a lighted portion or a light source itself. Therefore, the backlight image has a high-brightness image region and a low-brightness image region. In addition, since there is a bias in the way the light strikes, there are few image areas with intermediate brightness. However, the degree of backlight depends on the amount of light in the portion that is exposed to light, and varies depending on the shooting situation. Further, the ratio of the high luminance and low luminance regions depends on the size of the subject. Backlit images are classified into the following three types according to the balance of light intensity and subject size.
Type 1: The amount of light in the shadowed portion is insufficient, and there is a blackened area in the image. In other areas, the gradation difference and the brightness are sufficient.
Type 2: There is an area that is white in the image because the amount of light in the portion that is exposed to light is too strong.
In other areas, the gradation difference and the brightness are sufficient.
Type 3: There is a small difference in the amount of light between the lighted part and the shadowed part, and there is an area where the gradation difference is insufficient but the gradation is not collapsed for each of the low luminance and high luminance. Exists.

  FIG. 5 shows a histogram of each type of backlight image. In the figure, the vertical axis represents frequency, the horizontal axis represents luminance, and the solid line represents a luminance histogram. The luminance histogram of type 1 has the shape shown in FIG. 5A, and is a low-luminance and high-kurtosis (mountain) range 201 that is a blackened portion, and other kurtosis is not high (mountain). Part) range 202. The luminance histogram of type 2 has the shape shown in FIG. 5B, and is a high-brightness and high-kurtosis (mountain) range 204, which is a white portion, and other non-kurtosis (mountains). Part) range 203. The type 3 luminance histogram has the shape shown in FIG. 5C, and has a low luminance side distribution (peak range) 205 and a high luminance side distribution (peak range) 206.

  Next, the determination method of the backlight determination unit 3 will be described with reference to the flowchart shown in FIG. First, in step 601, it is determined whether or not the kurtosis of the mountain including the lowest class is high. At this time, a predetermined threshold is used to determine whether the kurtosis is high. For example, when a luminance histogram is created with 256 classes as shown in FIG. 2, the threshold is set to 20/16. The threshold needs to be set according to the class width of the luminance histogram. If the kurtosis of the peak including the lowest class is high, it is determined that there is a blackened portion in the input image, the backlight determination result is backlight, and the process proceeds to step 605. When the kurtosis of the mountain part including the lowest class is low, the process proceeds to the next determination (step 602).

  Next, step 602 determines whether the kurtosis of the mountain part including the highest class is high. Similar to the determination of the presence or absence of black crushing, the determination is performed by setting a predetermined threshold value. At this time, the predetermined threshold value is set to 20/16 when a 256-class luminance histogram is created, as in the criterion for determining the kurtosis of the peak including the lowest luminance. If the kurtosis of the peak including the highest class is high, it is determined that there is a whiteout portion in the input image, the backlight determination result is backlight, and the process proceeds to step 605. When the kurtosis of the peak including the maximum luminance is low, the process proceeds to determination based on the number of peaks (step 603).

  In addition, said threshold value is a value at the time of performing kurtosis by the said simple formula. When it is determined by the above equation 1, it is generally assumed that the kurtosis α is 3 or more when the kurtosis is high.

  Next, step 603 determines whether or not the number of peaks is two. If the number of peaks is two, the process moves to step 604. Step 604 determines whether or not the distributions of the two peaks are separated, that is, separated into low luminance and high luminance. Here, the distance between the peaks is used, and if the distance between the peaks is equal to or greater than a predetermined value, it is assumed that the two peaks are separated, the backlight determination result is backlight, and the process proceeds to step 605. The distance between the peaks may be determined to be far away, for example, when the brightness is represented by 256 gradations and the brightness difference is 96 or more. On the other hand, if the distance between the peaks is less than the predetermined value, the two peaks are distributed at an easy-to-see intermediate luminance, the backlight determination result is forward light, and the process proceeds to step 606. Step 605 outputs information indicating the backlight to the image correction means 4 when it is determined that the backlight is back. Step 606 outputs information indicating the follow light to the image correction means 4. In addition, since it is a process at the time of determining that it is a follow light, you may perform the process which outputs without performing a special process.

  On the other hand, if the number of peaks is not two in step 603, the peak is one to three or more, and the histogram is distributed over all luminance levels. . Therefore, the back light determination result is assumed to be forward light, and the forward light processing in step 606 is performed.

  FIG. 6 is an example of a determination method, and any method may be used as long as it is a determination method based on the characteristics (such as types 1 to 3) of the backlight image described above.

  The image correction unit 4 corrects the input image based on the determination result of the backlight determination unit 3 and outputs a corrected image. The correction process is changed depending on whether the determination result of the backlight determination unit 3 is backlight or forward light, and both images are corrected to an easy-to-view image. An example of the correction process is shown below.

  When the backlight determination result is backlight, correction is performed to make it easier to see the region that is difficult to see due to backlight. For example, as shown in FIG. 7, when a gamma curve 301 is applied so as to widen the gradation of the low luminance area and narrow the gradation of the high luminance area, it becomes easy to see the blackened portion. If the determination result is forward light, the input image is output as it is. By doing so, it becomes possible to automatically select a backlight image from the input image and perform correction processing suitable for the backlight image.

  In addition, human eyes look at different ranges between dark places and bright places. There is a method in which the gamma curve is changed for each local region and applied to the backlight correction processing applying this principle (see Japanese Patent No. 4011073). When this method is used for correcting the backlight image in the image correction means 4, it is possible to correct the backlight photograph to an image that is close to the impression at the time of photographing and does not have a white area or a black area.

  In the above description, the image correction unit 4 is included in the image processing apparatus. However, the image correction unit 4 is not an essential component, and may be configured as an image processing apparatus that outputs the determination result of the backlight determination unit 3. In this case, a free image correction process can be performed outside based on the output from the backlight determination means 3.

  In the above description, the histogram calculation unit 1 calculates the histogram of the luminance signal, but is not limited to the luminance signal, and any image signal may be used. However, since the characteristics of the shape of the histogram of the backlight image differ depending on the image signal, the backlight determination criterion of the backlight determination unit 3 must be changed according to the characteristics. For example, when calculating a histogram of color difference signals, the backlight image has an extremely high frequency of intermediate values compared to the follow-up image, and therefore, it may be determined that there is a peak in the intermediate value. The reason why the frequency of the intermediate value is high is that the backlight image is often an achromatic image having no color. Further, when calculating a histogram of RGB signals, it is possible to determine backlight by handling the G signal in the same manner as the luminance signal.

  Note that the histogram calculation means 1 may calculate a histogram of a plurality of image signals without being limited to one type of image signal. In this case, the backlight determination criterion of the backlight determination means 3 is a combination of the criteria based on the shape feature of the histogram of one signal. By doing so, it is possible to make the backlight determination more precise.

  In the present invention, the backlight shape is determined from the brightness balance of the entire image using the shape of the histogram. However, if the important subject and its area in the image are known, the important subject area is given priority. The conventional backlight determination method may be used in combination. At this time, before inputting an image signal to the histogram calculation means 1, a means for confirming whether or not important subject area information has been input is added. For example, when an important subject is a human face, a conventional backlight determination is performed if a face area is detected, and if a face area is not detected, an image signal is input to the histogram calculation means 1 and the shape of the histogram is calculated. The backlight is determined.

  The image processing apparatus has been described above. However, an image processing program that causes a computer to function as each unit may be stored in a storage device, and this program may be read to cause the computer to execute each unit.

  As described above, according to the first embodiment, it is possible to automatically perform backlight determination on an image for which it is unknown whether it is backlight or not, and it is possible to perform correction suitable for the image according to the backlight determination result.

  In addition, according to the image processing apparatus of the present embodiment, since the characteristics of the shape of the histogram of the input image signal are grasped by the number of peaks and the distance between the peaks, the approximate characteristics of the histogram There is an effect that can be extracted.

  Further, according to the image processing apparatus of the present embodiment, the characteristics of the shape of the histogram of the input image signal depend on the number of peaks and the distance between peaks, regardless of the position and change amount of the peak of the histogram. Since it has been grasped, there is an effect that a general feature of the histogram can be extracted, which is close to evaluation of an image such as backlight performed by a person.

  Further, according to the image processing apparatus of the present embodiment, when calculating a histogram and detecting a portion having a frequency greater than a predetermined value as a peak as a feature of the histogram, the maximum value of the histogram is used. In addition, the position of the peak portion can be detected, and the backlight determination of the input image can be performed with high accuracy. That is, by making a class region having a frequency equal to or greater than a predetermined ratio with respect to the maximum value as a peak, it is possible to capture the characteristics of the histogram distribution more accurately than when detecting a peak with a constant value. This is because the class range having a frequency of a predetermined ratio of the maximum value is set as the range of the peak portion, and the threshold value for determining the peak portion changes according to the image feature (according to the maximum value). It is possible to appropriately determine an image having (histogram shape).

  In addition, according to the image processing apparatus of the present embodiment, since a class region around a class having a maximum value frequency and having a frequency equal to or greater than a predetermined ratio with respect to the maximum value is set as a candidate for a mountain part, The candidate of the peak of the class having the frequency of the maximum value to be determined can be determined as one peak, and there is an effect that the global feature of the shape of the histogram can be extracted.

  Further, according to the image processing apparatus of the present embodiment, the characteristics of the shape of the histogram of the input image signal depend on the number of peaks and the distance between peaks, regardless of the position and change amount of the peak of the histogram. Since it has been grasped, it is possible to extract a general schematic feature of the shape of the histogram, and to perform backlight correction equivalent to backlight correction performed by a person.

  Further, according to the image processing apparatus of the present embodiment, the characteristics of the histogram of the signal representing the color are used for the backlight determination, so that the characteristics of the histogram can be captured more accurately.

Embodiment 2. FIG.
In the first embodiment, the method for determining whether the input image is backlight or forward light has been described. In the second embodiment, a method will be described in which the input image is classified more finely according to the characteristics of the histogram and correction suitable for each characteristic is performed. Here, in the case of classification into five, the operation will be described below.

  FIG. 8 is a block diagram showing an image processing apparatus according to Embodiment 2 of the present invention. In the figure, the same parts as those of the first embodiment shown in FIG. 8 differs from FIG. 1 in the backlight determining unit 3a and the image correcting unit 4a. Hereinafter, operations of the backlight determination unit 3a and the image correction unit 4a will be described.

  The backlight determination unit 3a classifies the input image from the features output from the histogram feature extraction unit 2 into five types that can be classified by the features of the histogram. In these five types, the same image processing unit 4a that performs the same processing is the same type. A determination method of the backlight determination unit 3 will be described with reference to a flowchart shown in FIG.

  First, step 901 determines whether or not the kurtosis of the mountain including the lowest class is higher than a predetermined threshold. If the peak kurtosis is higher than the threshold, the input image is classified as type 1 (proceeds to step 908). Type 1 has a blackened portion in the input image, and is one type of the backlight image type described above. When the kurtosis of the peak is equal to or less than the threshold value, the process proceeds to the next determination (go to step 902).

  Next, step 902 determines whether or not the kurtosis of the peak including the highest class is higher than a predetermined threshold value. If the peak kurtosis is higher than a predetermined threshold, it is determined in step 903 whether the cumulative frequency of intermediate luminance is low. It is also assumed that a threshold value is set and the cumulative frequency of intermediate luminance is low when the threshold value is below. For example, when the class width is 1 as in the luminance histogram of FIG. 2, the threshold for determining whether or not the intermediate luminance is small may be an accumulation frequency of 35%. Step 903 classifies the input image as type 2 when the frequency of intermediate luminance is low (proceeds to step 909). If the frequency of intermediate luminance is high, the input image is classified as type 5 (proceeding to step 912).

  Both type 2 and type 5 have whiteout portions in the input image, and both are types of the above-described backlight image type, but type 5 has a large distribution of intermediate gradations and does not require backlight correction. Since there are cases, it can be regarded as a type different from type 2.

  Based on the determination method of the first embodiment, classification experiments were performed for cases where both type 2 and type 5 were considered as backlight, and when type 2 was backlight and type 5 was forward light. In the experiment, the result of automatic classification using the above-described determination method using a total of 300 images of 150 images of each of the backlight image and the backlight image was compared with the result of determining whether the person is subjectively backlighted or not. I went. As a result of calculating the ratio of the automatic classification result and the manual determination result as the correct answer rate, the determination method in which type 2 and type 5 are backlit is a correct answer rate of 85%, type 2 is backlit, and type 5 is frontlit. In the determination method, the correct answer rate was 92%. Therefore, an image with overexposure can be determined with higher accuracy when the condition for backlight determination is that the distribution of the intermediate gradation is small in addition to the kurtosis of the peak including the lowest luminance.

  Here, in order to use a rough distribution such as a distribution of intermediate gradations in the determination method, the histogram feature extraction unit 2 calculates the ratio of high luminance, intermediate luminance, and low luminance pixels from the histogram as the rough distribution. Note that the class values (or luminances) of the high luminance region, intermediate luminance region, and low luminance region are set in advance. In this luminance area, the class area or luminance area set in advance can be determined according to the human sense if the luminance area that is easy to perceive is widened in terms of the sensitivity characteristics of human eyes. Specifically, in accordance with the sensitivity characteristics of the human eye, the intermediate luminance region, the low luminance region, and the high luminance region are preferably arranged in order of increasing luminance region range. In addition, by setting the difference between the ranges of each luminance region to about 1/8 to 1/16 of the total luminance or 1/8 or less, it is possible to obtain a good result that matches human sense experimentally. Yes.

  Unlike the features that represent relative relationships such as the number of peaks and the spacing between peaks, this approximate distribution is a feature related to absolute values called class values, so it is possible to grasp luminance levels with a large distribution and luminance levels with a small distribution. It becomes possible to deal with more diverse classifications. In addition, only the relative relationship such as the number of peaks and the interval between peaks is equal to or greater than a predetermined value, but if the peak having the highest class is at intermediate luminance, it is misjudged as backlight. there is a possibility. If the ratio of the pixels in the intermediate luminance area also satisfies the condition that is less than or equal to the predetermined value, the absolute position (class) of the peak can be taken into account by adding the condition that it is determined to be backlit, and the above erroneous determination Can be prevented.

  Further, the predetermined value (threshold value) for determining backlighting according to the ratio of the pixels in the intermediate luminance area is the ratio of the luminance gradation (or class) of the intermediate luminance area to all the luminance gradations (or classes) ( Hereinafter, it can be considered as a reference. For example, as a predetermined value for determining backlighting, a luminance ratio, a luminance ratio × 0.8, and a luminance ratio × 0.5 can be set. In general, good results are obtained when the luminance ratio is equal to or lower than the luminance ratio.

  On the other hand, when the kurtosis of the peak including the highest class is equal to or less than a predetermined value, the number of peaks is classified. First, step 904 determines whether or not the number of peaks is two. If the number of peaks is two, it is determined in step 905 whether the distribution of the two peaks is separated. The distance between the peaks is used, and when the distance between peaks is equal to or greater than a predetermined value, it is assumed that the two peaks are separated from each other, and the input image is classified into type 3 (proceeds to step 910). Type 3 is a type of the backlight image shown above. Conversely, in step 905, if the distance between the peaks is less than the predetermined value, the input image is classified as type 5 (proceeds to step 912). At this time, the input image is an image which does not need to be corrected because pixels of intermediate luminance occupy most of the input image. If the number of peaks is not two in step 904, the process proceeds to the next determination (step 906).

  Next, step 906 determines whether or not the number of peaks is one. If the number of peaks is one, it is checked in step 907 whether the peaks are located at low luminance. When the frequency of low luminance is high, the input image is classified into type 4 (proceeding to step 911). Type 4 is not a backlight image having distributions in both low luminance and high luminance, but most of them are distributed in low luminance, so that the entire image becomes dark and some correction is required. If the frequency of low luminance is low, the input image is classified as type 5 (proceeds to step 912). At this time, since the input image is an image that is uniformly distributed over the previous luminance level, it is an image that does not need to be corrected.

  Finally, if the number of peaks is not one, the input image is classified as type 5 (proceeding to step 912). At this time, the histogram of the input image has three or more peaks, and therefore the histogram is distributed over all luminance levels. Therefore, it can be said to be a follow light image and does not need to be corrected.

  In this way, the backlight determination unit 3a classifies the input image into five based on the features output from the histogram feature extraction unit 2.

  In the above description, the backlight determination unit 3a classifies the input image into five types. However, the number is not limited as long as the number can be classified using the features output from the histogram feature extraction unit 2. There is no.

  In FIG. 6 of the first embodiment, when it is determined Yes in step 602, the same determination as in step 903 of FIG. 9 may be made. In this case, in step 602, if the ratio of pixels having a high kurtosis with high luminance and the intermediate luminance range is equal to or less than a predetermined value, it is determined that the backlight is in the range of step 605. Do processing. By doing so, even if the peak including the highest class has a high kurtosis, it is possible to prevent erroneous determination of backlighting when the peak of the peak is in the intermediate luminance range.

  In addition, it is not necessary to decide on any type, and by calculating the contribution rate to each type and setting the intensity (parameters, etc.) of image processing applied to each type according to the contribution rate, more detailed image processing Is possible. An example of a method for calculating the contribution rate is shown below.

  Type 1 is an image in which the kurtosis of the peak including the lowest class is greater than or equal to a threshold value. For example, when the threshold is 20/16, the contribution rate is 0% when the kurtosis of the peak is 0/16, and the contribution rate is 100% when the kurtosis is 20/16 or more. In the meantime, the contribution rate changes linearly. That is, when the kurtosis is 8/16, the contribution rate is 40%. Similarly, the contribution ratio to type 2, type 3, and type 4 is calculated. In the case of Type 2, when the cumulative frequency of intermediate luminance is 30% or more and the kurtosis of the peak including the highest class is 0/16, the contribution rate is 0%, the cumulative frequency of intermediate luminance is 0%, and When the kurtosis of the mountain including the highest class is 20/16, the contribution rate is 100%. In the case of type 3, the distance between the peaks is expressed by a luminance difference, and the contribution rate is 0% when the luminance difference is 0, and the contribution rate is 100% when the luminance difference is 100 or more. In the case of Type 4, the contribution rate is 0% when the cumulative frequency of low luminance is 25% or less, and the contribution rate is 100% when it is 75%.

  Further, the image may not be processed excessively by normalizing the total contribution rate to each type to 100%. For example, when the contribution rate of type 1 is 75%, the contribution rate of type 2 is 30%, and the contribution rate of type 4 is 45%, the processing of type 1 is 50%, the processing of type 2 is 20%, the type 4 Processing is performed at a rate of 30%.

  The image correction unit 4a corrects the input image based on the classification result of the backlight determination unit 3a, and outputs a corrected image. An example of correction processing when the input image is intended to be corrected to an image that is bright and has no gradation collapse is shown below.

  When the classification result of the backlight determination unit 3a is Type 1, since the histogram has the shape shown in FIG. 5A, correction is performed so as to widen only the low luminance gradation. For example, a gamma curve having a low luminance and a large inclination is applied to the entire screen. The larger the tilt, the more the corrected image changes compared to the input image. Therefore, the tilt can be used as the strength of the correction process. By changing this stepwise, the correction based on the contribution rate shown above is possible. Is possible.

  When the classification result of the backlight determination means 3a is type 2, the histogram has the shape shown in FIG. 5B, so that the low luminance gradation is lifted as a whole and the high luminance gradation is widened. Correct correction. For example, an image quality correction mapping curve indicated by 302 in FIG. 7 is applied. Further, the process of changing the gamma curve for each local region shown in the first embodiment may be performed.

  When the classification result of the backlight determination unit 3a is type 3, the histogram has the shape shown in FIG. 5C, so that the low luminance side peak is widened to the high luminance side, and the high luminance side peak is obtained. Correction is performed so as to widen to the low luminance side. This is also the same processing as in the case of type 2, and can be converted into an easy-to-view image with many intermediate gradation pixels.

  When the classification result of the backlight determination unit 3a is type 4, since the histogram has a shape with one peak at low luminance, correction is performed so as to spread it to the high luminance side. For example, on the contrary to the case of type 1, a gamma curve having a large high luminance gradient is applied to the entire screen. When the classification result is type 5, no correction process is performed.

  Note that the correction processing of the image correction unit 4a described above can be realized by changing the gamma curve applied to the entire image or locally or by adding a certain value to the image signal.

  As described above, according to the image processing apparatus of the present embodiment, the blackout in which the gradation on the low luminance side is crushed is not classified into two types of backlight and forward light, but the same backlight is used. The features of the image signal are classified more finely according to the shape of the histogram, such as those that exist, conversely, those that have whiteout where the gradation on the high luminance side is crushed, and those that lack intermediate gradation. This makes it possible to perform image correction suitable for each feature.

Further, according to the image processing apparatus of the present embodiment, the range of high luminance, intermediate luminance, and low luminance is designated, and the cumulative frequency of each luminance is calculated and used for determination. And the type 5 classification), it is possible to determine according to the optimum correction method.

  Further, according to the image processing apparatus of the present embodiment, when the ratio of the pixels in the intermediate luminance region also satisfies the condition that it is equal to or less than the predetermined value, it is determined that the backlight is backlit, so that the absolute value on the peak histogram The target position (class) can be taken into account, and there is an effect that the backlight can be judged to match the sensitivity characteristic of the human eye. In particular, the widths of the luminance ranges of the high luminance region, the intermediate luminance region, and the low luminance region are set in advance in the order of the intermediate luminance region, the low luminance region, and the high luminance region so as to match the sensitivity of the human eye. Thus, it is possible to make a backlight determination that better matches the sensitivity characteristics of the human eye.

DESCRIPTION OF SYMBOLS 1 Histogram calculation means, 2 Histogram feature extraction means, 3, 3a Backlight determination means, 3, 4a Image correction means, 21 Peak detection means, 22 Peak detection means,
23 feature calculation means, 100 brightness histogram, 113 low brightness area, 115 intermediate brightness area, 301, 302 image quality correction gamma curve.

Claims (11)

Histogram calculation means for creating a histogram of luminance signals from the input image signal;
Detecting a maximum value of frequency from the histogram, detecting a class region having a frequency equal to or higher than a predetermined ratio with respect to the maximum value as a peak around the class taking the maximum value, Histogram feature extraction means for calculating the interval between the peaks and the number of peaks as a feature of the histogram,
Back light determination means for performing back light determination as to whether or not the image signal is back light from the characteristics of the histogram;
An image processing apparatus comprising: an image correcting unit that corrects the image signal based on the result of the backlight determination. Histogram calculation means for creating a color difference signal or RGB signal histogram from the input image signal;
Detecting a maximum value of frequency from the histogram, detecting a class region having a frequency equal to or higher than a predetermined ratio with respect to the maximum value as a peak around the class taking the maximum value, Histogram feature extraction means for calculating the interval between the peaks and the number of peaks as a feature of the histogram,
Back light determination means for performing back light determination as to whether or not the image signal is back light from the characteristics of the histogram;
An image processing apparatus comprising: an image correcting unit that corrects the image signal based on the result of the backlight determination. Histogram feature extraction means, claim and calculates the kurtosis of the ridges comprising kurtosis and highest rank of the crest comprising a class of the lowest among the detected peak portion as a feature of the histogram 1 Or the image processing apparatus of 2. The histogram feature extraction means is provided with three luminance areas, ie, a high luminance area, an intermediate luminance area, and a low luminance area that do not overlap each other in advance, and the luminance value among the pixels of the image signal is the luminance value in each luminance area. the proportion of pixels having calculated as a feature of the histogram, the range of brightness of the intermediate luminance region, the image according to claim 1 or 3, characterized in that wider than the luminance range of the high luminance region and the low luminance region Processing equipment. Based on the characteristics of the histogram, the backlight determination unit determines whether the kurtosis of the peak including the lowest class is equal to or higher than a predetermined first predetermined value, and the kurtosis of the peak including the highest class in advance. The case where it is equal to or greater than a predetermined second predetermined value and the case where the number of peak portions is 2 and the interval between the peak portions is equal to or greater than a predetermined third predetermined value are determined to be backlight. The image processing apparatus according to any one of claims 1 to 4, wherein Based on the characteristics of the histogram, the backlight determination unit determines whether the kurtosis of the peak including the lowest class is equal to or higher than a predetermined first predetermined value, and the kurtosis of the peak including the highest class in advance. When the ratio is equal to or greater than a predetermined second predetermined value and the ratio of pixels in the intermediate luminance area is smaller than a predetermined fourth predetermined value, the number of peak portions is two, and the interval between peak portions is predetermined. The image processing apparatus according to claim 4, wherein a case where it is equal to or greater than a predetermined third predetermined value is determined to be backlight. The image correction means calculates a difference between the peak values of the peak portions in which the kurtosis of the peak portions in the luminance histogram is greater than or equal to a predetermined fifth predetermined value with respect to the image signal determined to be backlight by the backlight determination portion. The image processing apparatus according to claim 3 , wherein a correction curve that is a widening gamma curve is applied to the image. Class histogram, the luminance level, the image correcting means, claims 1 and 3, characterized in that the average luminance applies the correction curve to a localized area corresponding to the pixels corresponding to the luminance level of the crests 7 The image processing apparatus according to any one of the above. Backlight determining means image processing apparatus according to any one of claims 1 to 8, characterized in that to determine the two or more the number of types from the feature of the histogram. Histogram calculation means for creating a histogram of the luminance signal from the input image signal, and detecting a maximum value of the frequency from the histogram, and a predetermined value predetermined for the maximum value around a class that takes the maximum value Histogram feature extraction means for detecting a class region having a frequency equal to or higher than a ratio as a peak, and calculating the interval between the peaks and the number of peaks when there are a plurality of peaks as the characteristics of the histogram, and the histogram An image processing program for causing a computer to function as a backlight determination unit that determines whether the image signal is backlight or not, and an image correction unit that corrects the image signal based on the result of the backlight determination. A histogram calculation step for creating a histogram of a luminance signal from the input image signal, and detecting a local maximum value from the histogram, and surrounding a class that takes the local maximum value, a predetermined value determined in advance for the local maximum value A histogram feature extracting step of detecting a class region having a frequency equal to or higher than a ratio as a peak, and calculating the interval between the peaks and the number of peaks when there are a plurality of peaks as the features of the histogram;
A backlight determination step for performing backlight determination as to whether or not the image signal is backlight from the characteristics of the histogram;
An image processing method comprising: an image correction step of correcting the image signal based on the result of the backlight determination.

Images