JP2000188768A - Automatic gradation correction method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To realize an automatic gradation correction method that can conduct automatic gradation correction taking the skin color into account. SOLUTION: Underexposure pre-processing, overexposure pre-processing or linear pre-processing is conducted on the basis of a feature quantity of a luminance histogram and various curve data (steps 103-107). Except the case of overexposure, it is discriminated that a strong rear light is present and a rear light flag is set to '1' when a 2nd feature quantity is positive both in regions where an input luminance level is the highest and lowest (steps 108-114). After face part recognition processing to recognize the face area of an input picture, a gradation curve correction level is corrected (steps 110, 116). In the gradation curve correction level correction processing, the correction is made around a level obtained through skin color recognition in the case of rear light.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an automatic gradation correcting method, and more particularly to an automatic gradation correcting method for automatically correcting the gradation of a color image when an output color image of a digital camera, a video camera or the like is printed by a printer. It relates to a key correction method.

[0002]

2. Description of the Related Art FIG. 18 is a block diagram showing an example of a printing system using a color video printer. In the figure, a color image signal obtained by imaging with a digital camera 11 is directly supplied to a video printer 16 and printed, while being taken in a personal computer (hereinafter, referred to as a personal computer) 14 and a cathode ray tube of a display monitor. (CRT) 19 and the personal computer 14
It is processed by a central processing unit (CPU) 15 and then supplied to a video printer 16 for printing.

A color image signal obtained by imaging with a video camera 12 is also used in the digital camera 11.
Video printer 1 as well as the color image signal from
While being supplied to the printer 6 and printed, it is taken into the personal computer 14 and displayed on the CRT 19 of the display monitor, processed by the CPU 15 in the personal computer 14 and then supplied to the video printer 16 for printing. Further, the color image signal reproduced by the video deck 13 is supplied to the video printer 16 and printed.

As described above, the video printer 16 includes the digital camera 11, the video camera 12, and the VCR 1
3 to print the color image signal input directly from
The output color image signal of the digital camera 11 or the video camera 12 processed by the CPU 15 in the personal computer 14 can be printed.

By the way, usually, the digital camera 11,
When shooting with the video camera 12, automatic adjustment is performed so that the camera body optimizes exposure. However, an image photographed by them does not always reproduce a desired gradation in a printing system as shown in FIG. In particular, the brightness and contrast that were not bothersome because the video was scanned by the video camera 12 were moving images.
Printing as a still image on the video printer 16 often does not provide satisfactory results.

Therefore, in order to obtain a desired gradation in the video printer 16, it has been conventionally performed to perform gradation correction of a color image and then print. In the gradation correction of the color image, the CPU 17 in the video printer 16 analyzes the image data taken in the image memory 18 and automatically corrects the gradation before printing, or the CPU 15 of the personal computer 14 Processing is performed such as analyzing the image data taken in by 14 and automatically correcting the gradation before printing.

[0007]

However, in the conventional automatic gradation correction method, for example, the applicant of the present invention disclosed in Japanese Patent Application No.
As proposed in -264244, gradation correction is performed by grasping the feature amount of the luminance histogram, but correction is not particularly performed in consideration of skin color. In the conventional method,
Correction is made so that the gradation balance of the entire screen is good, so if the screen is largely empty and weak backlight, it will not be possible to judge the backlight, and even if the person's face is slightly dark, it will be corrected too much In addition, in the case of strong backlight, there is a problem that sufficient gradation correction cannot be performed because it is not known how bright the face is, even if the face is determined to be backlight.

[0008] The present invention has been made in view of the above points,
An object of the present invention is to provide an automatic gradation correction method capable of performing automatic gradation correction in consideration of skin color.

It is another object of the present invention to provide an automatic gradation correction method capable of performing automatic gradation correction that is resistant to backlight and has a good face luminance level.

[0010]

According to the present invention, in order to achieve the above object, a luminance histogram is created from color image data whose gradation is to be corrected, and an input luminance level axis is equally divided into (a) from the luminance histogram. Is an integer equal to or greater than 4) indicating the first characteristic amount indicating the ratio of the number of pixels in each region to all pixels, and the ratio of the number of pixels exceeding a certain limiter value in each region equally divided by a to all pixels. Second
And the third feature value indicating the ratio of the number of pixels in each area obtained by equally dividing the input brightness level axis to all the pixels from the brightness histogram, and calculating the third feature value, and storing the third feature value in the first storage unit. Step 1 and the first to third feature values stored in the first storage unit are fetched, and the third feature value of the lowest luminance area is larger than a preset lower limit value, and A second step of comparing whether the third feature value of the highest brightness region is smaller than a preset upper limit value, and a third feature value of the lowest brightness region by the second step. When a determination result is obtained that is larger than the lower limit set in advance, it is determined that the exposure is under,
Using the first feature amount, the slope information of the contrast is obtained,
The third step of performing under-preprocessing for obtaining under-intensity from the inclination information and the second step provide a determination result in which the third feature amount of the highest luminance side area is larger than a preset upper limit value. When it is determined that the exposure is over, the fourth step of performing the pre-over process for obtaining the inclination information of the contrast using the first feature amount and obtaining the over intensity from the inclination information, and the second step It is possible to obtain a determination result in which the third characteristic amount of the lowest luminance region is equal to or less than a preset lower limit value and the third characteristic amount of the highest luminance region is smaller than the preset upper limit value. A fifth step of obtaining contrast gradient information using the first feature amount and performing a linear pre-process of obtaining an intensity according to the second feature amount, and further, the sixth to eleventh steps. And Characterized in that it comprises.

In the sixth step, when it is determined that the exposure is not over in the second step, the second feature amount is a positive value in both the highest input luminance level area and the lowest input luminance level area. Sometimes it is determined that the backlight is strong, and the backlight flag is set to a predetermined value. The seventh step is to determine whether or not the input color image data is included in a region where the brightness, hue, and saturation are skin colors, and to determine whether the backlight flag is a predetermined value when the backlight flag is a predetermined value. In comparison, the determination is made by lowering the brightness level of the skin color area. In the eighth step, the area of each connected region is determined for the flesh-colored pixel detected in the seventh step, and a total of n (n is a predetermined number) of the connected regions in order from the largest area to the largest area. A connected area of (defined integer of 2 or more) is extracted as an area to be recognized, and each of the n areas is applied to a frame of a predetermined shape including the area therein, and the hair is placed at the top of the frame. Assuming a frame with an area,
For each of the n frames as a whole, a fourth feature value indicating the total number of skin color pixels, a fifth feature value indicating information of the frame in the vertical direction of the screen, and a sixth feature value indicating information of the frame in the horizontal direction of the screen. Then, the fourth to sixth feature values are analyzed, and one area having the highest probability of being a face area is determined as a face area.

In the ninth step, an average luminance of the face area determined in the eighth step is obtained, and the average luminance is compared with a predetermined target luminance range.
Based on the comparison result, the contrast inclination information and the intensity information created in the third, fourth, or fifth step are corrected to values predicted so that the average luminance falls within the target luminance range. In the tenth step, according to the intensity corrected in the ninth step, various curve data indicating various characteristic curves of the output gradation with respect to the input gradation are stored in the second storage unit. Curve data is selected, and the selected curve data is combined with the contrast gradient information corrected in the ninth step to generate gradation correction curve data, and stored in the third storage unit. The eleventh step above is
In the image memory, the gradation correction curve data from the third storage unit is reflected on the color image data to be corrected, read from the image memory in which the color image data to be corrected is stored. Store again.

According to the present invention, the luminance histogram of the input color image data is divided into equal parts a, the approximate shape of the curve is known, the slope information of the contrast is calculated according to the shape, and the gradation correction curve is calculated. Is corrected in consideration of the skin color to select the optimal curve data, and then synthesizes them to generate gradation correction curve data by software processing. Further, according to the present invention, it is possible to correct the gradation correction curve in consideration of the skin color in both the normal state and the strong backlight.

Further, in the present invention, when the backlight flag is a predetermined value, the probability value of one area having the highest probability of being an area showing a face is determined by a first predetermined value. When the area is equal to or larger than the lower limit, one area is determined to be a face area, and when the backlight flag is not a predetermined value,
A preset second count in which the total number of skin color pixels in one area having the highest probability of being a face area is equal to or greater than a predetermined empirical value, and the probability value in one area is greater than a first lower limit. One area is determined to be a face area only when the value is equal to or more than the lower limit of 2.

According to the present invention, when the backlight flag is a strong backlight having a predetermined value, the level center obtained by skin color recognition can be corrected. In other words, in the case of strong backlight, the conventional method does not have a sufficient amount of information for determining the level, so that the correction is not performed in consideration of the skin color.
In the present invention, it is possible to perform correction of the level center obtained by skin color recognition.

In the present invention, a relatively good result is normally obtained when the entire screen is viewed even at the level obtained by the conventional analysis method. However, the correction is performed in consideration of the luminance of the skin color. For this reason, the present invention is characterized in that a level between the level obtained by the conventional analysis method and the level obtained by skin color recognition is taken.

[0017]

Next, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a flowchart of one embodiment of a main part of an automatic gradation correction method according to the present invention.
FIG. 1 shows a block diagram of an embodiment of a correction device for realizing an automatic gradation correction method according to the present invention. The method of the present invention is shown in FIG.
In the printing system as shown in FIG. 8, the CPU 17 inside the video printer 16 analyzes the image data taken into the image memory 18 and automatically recognizes the flesh color. And automatically recognizes skin color by analyzing the image data obtained.Based on color recognition, image data that has been subjected to gradation correction so that the color reproduction of a person's face is good even in the case of backlight. The image is printed by the video printer 16.

In the automatic gradation correction processing of this embodiment,
In order to add skin color recognition to the conventional automatic gradation correction and to make the skin color recognition processing as short as possible, the original image is made to be able to recognize the skin color even in a dark state even before being corrected. In addition, it is necessary to correct the skin color in both normal and strong backlight, but usually it is necessary to devise to make use of the conventional correction method focusing on the overall balance, and in the case of strong backlight, Is a correction method centering on skin color.

This embodiment will be described first with reference to the block diagram of FIG. In FIG. 2, for example, a YUV input from an external video device such as a digital camera, a video camera, a personal computer, or a VCR.
Are stored in the image memory 21. In consideration of the processing speed, the image sampling processing unit 22 samples image data every g pixels (g> 0) from the image memory 21 and stores the sampled image data in the image sampling data storage unit 23. g may be a value such as 2 or 4 depending on the total number of pixels. The image sampling data storage unit 23 corresponds to the image memory 18 in FIG.
2 except for the image sampling data storage unit 23 and the image memory 21 is realized by software of the CPU 17.

The luminance histogram creating section 24 creates a luminance histogram from the digital color image data in the three-dimensional color space stored in the image sampling data storage section 23 and stores it in the luminance histogram storage section 25.
The brightness histogram feature value calculation unit 26 calculates a brightness histogram feature value from the brightness histogram stored in the brightness histogram storage unit 25, and stores the brightness histogram feature value in the brightness histogram feature value storage unit 27. There are three types of the above-described luminance histogram feature amounts to be stored.

First, a feature quantity indicating the ratio of the number of pixels in each region to all pixels obtained by equally dividing the data of the brightness histogram storage unit 25 by a on the input brightness level axis (where a> 3). Sep [0] to Sep [a-1].
The second is a feature amount indicating the ratio of all pixels over the number of pixels exceeding a certain limit value obtained by equally dividing the data of the brightness histogram storage unit 25 by a on the input brightness level axis (where a> 3). LimitSep [0]-Limit
Sep [a-1]. Thirdly, feature amounts L, M, and H indicating the ratio of the number of pixels in each region to all pixels obtained by dividing the data of the brightness histogram storage unit 25 into three on the input brightness level axis.

The overexposure / underexposure detection section 31 reads out the luminance histogram characteristic amounts stored in the luminance histogram characteristic amount storage section 27 and detects overexposure and underexposure from the luminance histogram characteristic amounts. Subsequently, the contrast gradient determination unit 32 determines contrast gradient information based on the luminance histogram feature stored in the luminance histogram feature storage 27 and the detection result of the over-exposure / under-detection unit 31. . The contrast data creating unit 35 creates contrast data based on the information on the contrast inclination determined by the contrast inclination determining unit 32.

The over / under intensity judging section 33 is determined by the luminance histogram feature stored in the luminance histogram feature storing section 27, the detection result of the over-exposure / under-detection section 31, and the contrast inclination determining section 29. FIG. 1
Are performed, the over-intensity and the under-intensity (Level) of the exposure are determined, and various curve data storage units 32 are determined in accordance with the intensities (Level).
Is selected from among various types of curve data stored in.

The strong backlight detection unit 28 stores LimitSep [0], [a−
1] and the information of over, under, and linear from the over- and under-exposure detection unit 31, and performs the judgment processing of steps 108 to 114 in FIG. 1 to determine the backlight flag refFlag to be 0 or 1.

The face part recognition processing section 29 receives the above-mentioned backlight flag refFlag, receives three-dimensional digital color image data from the image sampling data storage section 23, and performs face part recognition processing described later in step 115 of FIG. . The gradation correction curve correction unit 30 calculates the luminance average value information of the face area, the backlight flag refFlag, the contrast inclination,
Upon receiving the level intensity, a gradation correction curve level correction process described later in step 116 of FIG. 1 is executed. And over,
Under-intensity determination unit 33, contrast gradient determination unit 32
Modify the information in

As shown in step 117 of FIG. 1, the contrast straight line / curve data synthesizing section 36 compares the contrast straight line data created by the contrast data creating section 30 with the curve selected by the over / under intensity judging section 31. The data is combined to generate gradation correction curve data (118 in FIG. 1), which is stored in the gradation correction curve data storage unit 3.
7 is stored. Assuming that the variable i changes from 0 to 255, if the contrast straight line data is cont [i] and the curve data is curve [i], the synthesis result autog
am [i] is autogam [i] = curve [c
ont [i]].

The color difference signal overflow detecting section 38 uses the sampling data stored in the image sampling data storage section 23 and the gradation correction curve data stored in the gradation correction curve data storage section 37 to calculate the color difference. The number of pixels (Count) at which the signal overflows is detected, and the detection result is input to the gradation correction curve correction unit 39.

The tone correction curve correction section 39 includes an overflow pixel number input from the color difference signal overflow detection section 38, various curve data from the various curve data storage section 34, and a tone correction curve data storage section 37. In response to the linear level and the curve level, it is determined how much the intensity of overexposure (Level) should be reduced according to the number of overflow pixels, and the data in the gradation correction curve data storage unit 37 is corrected based on the determination result. As described above, the inclination information input to the contrast data creation unit 35 is corrected, and if correction is not possible, the correction curve data is further changed and input to the contrast straight line and curve data synthesis processing unit 36.

The image processing section 40 includes a gradation correction curve correcting section 3
The image data from the image memory 21 is corrected based on the gradation correction curve data in the gradation correction curve data storage unit 37 corrected by step 9, and the corrected image data is stored in the image memory 21 again. The image data subjected to the automatic gradation correction stored in the image memory 21 is read out and printed by the printer.

Next, the operation of the main part of this embodiment will be described in more detail with reference to flowcharts. When calculating the brightness histogram feature value in the brightness histogram feature value calculation unit 26, the input brightness level axis is divided into eight equal parts (a =
8) When the luminance histogram feature amount is set to Sep.
[0] to [7] and LimitSep [0] to [7] are prepared. Sep [] contains the ratio of the number of pixels in each area to the total number of pixels. Also, Limit
Sep [] contains the ratio of the number of pixels exceeding a certain limiter value to the total number of pixels. In this embodiment, the luminance histogram feature amounts L, M, and H are:
L = Sep [0] + Sep [1] + Sep [2], M =
Sep [3] + Sep [4], H = Sep [5] + Se
It is assumed that p [6] + Sep [7].

The various curve data stored in the various curve data storage unit 34 in FIG.
[5], * Down [1] to [5] are prepared. * U
p [] and * Down [] are YY 'look-up tables (LUTs). * Up [1] is L close to linear
It is assumed that the UT stores the value of the output Y ′ corresponding to the value of the input Y from 0 to 255. * Up []
As the number in [] increases, the correction strength of the LUT increases. Similarly, * Down [1] is the LUT closest to linearity, and as the number in [] of * Down [] increases, the correction strength of the LUT increases.

The over / under strength judging section 33
Fetches the luminance histogram characteristic amount from the luminance histogram characteristic amount storage unit 27 (step 101 in FIG. 1), and stores various curve data * Up from the various curve data storage unit.
[1] to [5], * Down [1] to [5] are fetched (step 102 in FIG. 1), and it is first determined whether or not the luminance histogram feature L is larger than a preset lower limit (underLimit). (Step 10 in FIG. 1)
3).

Since the luminance histogram characteristic amount L is a characteristic amount of the region on the lowest luminance side when the input luminance level axis is divided into three equal parts, if L> (lower limit value), it is determined that the exposure is underexposure and underexposure occurs. Perform pre-processing (Step 10 in FIG. 1)
4) If L ≦ (lower limit), it is determined whether or not the luminance histogram feature value H is larger than a preset upper limit (overLimit) (step 105 in FIG. 1).
Since the luminance histogram feature H is a feature of the region on the highest luminance side when the input luminance level axis is divided into three equal parts, H
If> (upper limit), it is determined that the exposure is over and pre-over processing is performed (step 106 in FIG. 1). Processing is performed (step 107 in FIG. 1).

The upper and lower limits are determined by conducting experiments on many images. For example, upper limit = 6
When 0 and the lower limit = 60, it is determined that L is under if the pixels are more than 60% biased, and it is judged that the pixels are over when H is more than 60% biased.

In the under pre-processing of step 104, the levels of the contrast straight line and the correction curve * Up [] are determined and combined based on the luminance histogram feature quantity in accordance with the flowcharts of FIGS. Tone correction curve. The basic strategy is to flatten the histogram.
The flattening of the histogram itself is known (Shin Nagao,
"Digital Image Processing", Modern Science, p177-p1
81, 1978).

First, the initial value of the level is set to, for example, "5" (step 201 in FIG. 3), and the variable s
um is set to "0" (step 20 in FIG. 3).
2) The variable i is set to "7" (step 20 in FIG. 3).
3) The luminance histogram feature amount Sep is set in the variable sum.
[I] is added (step 204 in FIG. 3), and it is determined whether or not the added value sum is greater than 0 (step 205 in FIG. 3). If sum ≦ 0, the value of i is set to 6 Similar addition and determination processing is performed. The above addition and determination processing is repeated while the value of i is reduced by 1 until sum> 0.

If sum> 0, the variable secthig
The value of h is set to 7-i, and the value of the variable sectlow is set to 0 (step 206 in FIG. 3). That is, first,
In consideration of the fact that the pixels are biased toward the lower luminance, an area where the pixels are 0% is searched from the luminance histogram feature amount Sep [7] having the highest luminance. The 0% area is set to Ycon-Yco in order not to assign a gradation to an area without pixels.
In the graph of n ', Ycon' is attached to 255. The variable second indicates the area to be attached to 255,
The variable “sectlow” indicates an area pasted to “0”.

If only 1% from the top is 0%, su
Since m becomes larger than 0 when i = 6, the variable sect
The high is 1 (= 7−6) section. It is to be noted that the reason why seclow = 0 is set in step 206 is that the probability of 0% counted from below in an under image is low and the process is simplified. The inclination of the contrast straight line is determined up to this point. The value to be compared with sum in step 205 may be other than 0. This value can be variably set by experience. If it is desired to increase the contrast correction strength, set this to a large value.

The variable sechigh is the result information of the contrast gradient determining section 29. If this value is large, it becomes a straight line that increases the output gradation with respect to the input gradation, and the under image is improved. Next, a variable Level indicating the strength of the under is represented by Level = Level-sect.
The contrast is subtracted from the level of the correction curve based on the arithmetic expression “high” (step 20 in FIG. 3).
7).

Next, a method using the accumulated luminance distribution is applied to determine the correction intensity. In the under pre-processing, since it is determined that the pixels are biased toward the lower luminance, the histogram shape of the region is used as a material for the determination. That is, first, Sep [0]> Sep [1]> S
It is determined whether the condition of ep [2] is satisfied (step 208 in FIG. 3). If the condition is not satisfied, it is determined that the level is weak, and the variable Level indicating the strength is set to Level-2 (FIG. 3). Step 209). Sep [0]> Sep
If [1]> Sep [2], then Sep [0] is S
Compare whether the value is greater than ep [1] * 1.5 (FIG. 3)
Step 210), if Sep [1] * 1.5 or less, set the variable Level indicating the intensity as a medium level to Level
-1 (step 211 in FIG. 3).

Also, Sep [0]> Sep [1]> Se
p [2] and Sep [0]> Sep [1] * 1.
In the case of 5, it is determined that the inclination is the maximum, and the variable Level indicating the intensity of the under is left as it is. The value "1.5" can be changed to one or more values by experience. When the value is increased, the correction is weak, and the ratio of being judged as the medium level is increased.

Next, paying attention to the lower luminance, it is determined whether or not the ratio of pixels exceeding the limiter value is larger than 0% (that is, LimitSep [0]> 0) (step 212 in FIG. 4). In this case, the level is kept as it is, otherwise, the value of Level is reduced by 1 (step 213 in FIG. 4).

As described above, as a result of subtraction from Level = 5, Level may become 0 or less. Since the image was originally determined to be under, it is corrected so that Level = 1 at least as the final correction. Therefore, it is determined whether Level is larger than 0 (see FIG. 4).
Step 214), if it is 0 or less, the value of Level is corrected to 1 (step 215 in FIG. 4).

Subsequently, since the contrast gradient is determined from the gradient information sechigh and sectLow determined by the contrast determining section 29, the contrast data of the input Ycon and the output Ycon 'is obtained, and * Cont is determined from j = 0 to j of Ycon. = 255 is stored (step 216 in FIG. 4). The equation is a linear equation as follows. a is the number of histogram divisions, which is “8” here.

X1 = (256 / a) * sectlow (provided that x1 = 255 when x1> 255) x2 = 256- (256 / a) * secthigh (provided that x2 = 255 when x2> 255) Ycon '[J] = 255 * (j-x1) / (x2-x
1) Thus, the under pre-processing is completed.

Next, the pre-over process in step 106 will be described in more detail with reference to the flowcharts of FIGS. In the case of over preprocessing, a straight line for contrast and a correction curve * based on the luminance histogram feature amount *
The major flow is to determine and combine the level of Down [].

First, the initial value of the level (Level) is set to, for example, "5" (step 301 in FIG. 5), and the variable s
The initial value of um is set to “0” (step 30 in FIG. 5).
2) The variable i is set to "0" (step 30 in FIG. 5).
3) The luminance histogram feature amount Sep is set in the variable sum.
[I] is added (step 304 in FIG. 5), and it is determined whether or not the added value sum is greater than 0 (step 305 in FIG. 5). If sum ≦ 0, the value of i is set to 2 Similar addition and determination processing is performed. The above addition and determination processing is repeated while incrementing the value of i by 1 until sum> 0.

When sum> 0, the variable setlow
Is set to i, and the value of the variable second is set to 0 (step 306 in FIG. 5). That is, first, in consideration of the fact that the pixels are biased toward the higher luminance, an area where the pixels are 0% is searched from Sep [0] having the lowest luminance. 0% to avoid assigning gradations to areas without pixels
In the area, Ycon 'is pasted to 0 in the Ycon-Ycon' graph.

It should be noted that in step 306, the second
= 0 is counted from the top in the over image and is 0
The probability of% is low to simplify the processing. The inclination of the contrast straight line is determined up to this point. The value to be compared with sum in step 305 may be other than 0. This value can be variably set by experience. If it is desired to increase the contrast correction strength, set this to a large value. Variable se
ctlow is the result information of the contrast gradient determination unit 29. If this value is large, it becomes a straight line that decreases the output gradation with respect to the input gradation, and the oversized image is improved.

Next, a variable Level indicating the overstrength is set.
1 is subtracted from the level of the correction curve by the amount of contrast based on an arithmetic expression of Level = Level-sectlow (step 307 in FIG. 5). Next, a method using a cumulative luminance distribution is applied to determine the correction intensity. In the pre-over processing, since it is determined that the pixels are biased toward the higher luminance, the histogram shape of the area is used as a material for the determination. That is, first, Sep
It is determined whether the condition [7]> Sep [6]> Sep [5] is satisfied (step 308 in FIG. 5). If the condition is not satisfied, it is determined that the level is weak, and the variable Le indicating the strength is determined.
The level is set to Level-2 (step 30 in FIG. 5).
9).

Sep [7]> Sep [6]> Sep
If [5], then Sep [7] is Sep [6] *
Compare whether it is greater than 1.5 (step 3 in FIG. 5).
10), if it is equal to or less than Sep [6] * 1.5, a variable Level indicating the intensity is set to Level-1 as a medium level (step 311 in FIG. 5). Also, Sep [7]> Se
p [6]> Sep [5], and Sep [7]> Se
In the case of p [6] * 1.5, it is determined that the inclination is the maximum, and the variable Level indicating the intensity of over is kept as it is. "1.
The value of 5 "can be changed to a value of 1 or more by experience. If the value is increased, the correction becomes weak, and the ratio of being judged as the medium level increases.

Next, paying attention to the higher luminance, it is determined whether or not the ratio of pixels exceeding the limiter value is larger than 0% (that is, LimitSep [7]> 0) (step 312 in FIG. 6). In this case, the level is kept as it is, otherwise, the value of Level is reduced by 1 (step 313 in FIG. 6). Next, Level
Is greater than 0 (step 31 in FIG. 6).
4) If the value is 0 or less, the value of Level is corrected to 1 (step 315 in FIG. 6).

Subsequently, since the contrast inclination is determined from the inclination information sechigh and sectLow determined by the contrast determination unit 29, the contrast data of the input Ycon and the output Ycon 'is obtained, and * Cont is determined from j = 0 to j of Ycon. = 255 is stored (step 316 in FIG. 6). The equation is the same linear equation as in the under pre-processing. With this, the pre-over process ends.

Next, the linear pre-processing of step 107 will be described in more detail with reference to the flowcharts of FIGS. In the linear pre-processing, a contrast straight line and a correction curve * U
The major flow is to determine and combine the levels of p [] or * Down [].

First, in order to determine the contrast, the upper and lower 0
Find the% area. That is, the data is added in the direction from Sep [7] to Sep [0] of the luminance histogram feature amount storage unit 27 to check which section from 7 to 0 or less and stores it in the variable sechigh (step 401 in FIG. 7). ~
405). Subsequently, the luminance histogram feature storage unit 27
Is added in the direction from Sep [0] to Sep [7], and it is checked how many sections from 0 to 0 or less.
It is stored in low (steps 406 to 410 in FIG. 7).
However, a value of 0 or less can be adjusted by experience. If it is desired to increase the contrast correction strength, set it higher.

Next, in order to determine the correction intensity, the data LimitSe in the luminance histogram feature storage 27 is stored.
Look at the value of p []. It is determined whether LimitSep [0]> 0 and LimitSep [7]> 0 (FIG. 8).
Step 411), if this condition is satisfied, it is determined that there is backlight. When it is determined that the subject is backlit, it is determined that the subject is under light, ignoring that a bright portion is crushed in order to simplify the processing. In the case of analyzing by the cumulative luminance distribution, a curve having an inflection point is often obtained in the case of such a histogram, but here, a curve of * Up [] is simply selected.

If it is determined that the value is under, the data LimitSe in the luminance histogram feature storage unit 27 is stored.
Looking only at p [0], when the value is small, the Level is low, and when the value is large, the Level is high (Step 41 in FIG. 8).
2). The level assignment may be adjusted by investigating many images. Next, second, second
The gradient of the contrast straight line * Cont is calculated from the low based on the same formula as the under process, and stored in the contrast straight line * Cont (step 413 in FIG. 8).

On the other hand, LimitSep [0] and Limi
If only LimitSep [0] of tSep [7] satisfies LimitSep [0]> 0 (steps 415 and 416 in FIG. 8), it is determined to be under and the same processing as steps 412 to 413 described above is performed. (Steps 417 to 418 in FIG. 8). Also, LimitSep [0]
And LimitSep [7], LimitSep
If only [7] has LimitSep [7]> 0 (steps 415 and 416 in FIG. 8), it is determined to be over and the data Limi in the luminance histogram feature amount storage unit 27 is determined.
Looking only at tSep [7], the level is lowered when the level is small, and the level is raised when the level is large (step 420 in FIG. 8). The level assignment may be adjusted by investigating many images.

Subsequently, sechigh, sectro
The gradient of the contrast straight line * Cont is calculated from w based on the same formula as the under process, and stored in the contrast straight line * Cont (step 421 in FIG. 8). Thus, the linear pre-processing at step 107 in FIG. 1 is completed.

When the above-described under pre-processing, over pre-processing or linear pre-processing is being performed, the strong backlight detection section 28 receives under-exposure information among over, under, and linear from the over-exposure and under-detection section 31. When this is done, LimitSep [0] and LimitSep input from the luminance histogram feature amount storage unit 27
It is determined whether both [7] are positive (step 108 in FIG. 1). If both are positive, it is determined that the backlight is strong and the backlight flag refFlag is set to “1” (step 1 in FIG. 1).
09) Otherwise, the backlight flag refFlag is set to "
0 "(step 110 in FIG. 1).

The strong backlight detection unit 28 sets the backlight flag refFlag to “0” when over-information is input from the over-exposure / under-detection unit 31 (FIG. 1).
Step 111), overexposure and underexposure detector 3
When the linear information is input from No. 1, the LimitSep input from the luminance histogram feature amount storage 27 is used.
It is determined whether both [0] and LimitSep [7] are positive (step 112 in FIG. 1). If both are positive, it is determined that the backlight is strong and the backlight flag refFlag is set to “
1 (step 113 in FIG. 1), otherwise the backlight flag refFlag is set to "0" (step 114 in FIG. 1), that is, the backlight flag refFlag.
Is "1" when the luminance of the face of the person in the input image is lower than normal compared to the luminance of the background.

Next, in the face portion recognition processing section 29, the above-mentioned backlight flag refFlag from the strong backlight detection section 28 is output.
Based on the digital image data from the image sampling data storage unit 23, a face portion recognition process is performed (step 115 in FIG. 1). The face portion recognition processing in step 115 will be described in more detail with reference to the flowcharts of FIGS. 9 to 13 and FIGS. 14 and 15.

As shown in FIG. 9, the face part recognition processing unit 29
First reads three-dimensional digital image data (YUV image data) in the YUV color space stored in the image sampling data storage unit 23 (step 50).
1), indicating that the image data is a skin color area, Y
Yi1 to Yi2, Ui1 in UV three-dimensional color space coordinates
~ Ui2, Vi1 ~ Vi2 (where i = 1,
2,. . . , M) are represented by ref
When Flag = 1, it is set to a low area, and refFlag is set.
When g = 0, Yi1 and Yi2 are set as standard regions (step 502 in FIG. 9).

Each of the above rectangular parallelepiped regions is a region which is determined in advance to be a skin color region. For example, when m = 3, the rectangular parallelepiped regions are represented as three rectangular parallelepipeds in the YUV three-dimensional space coordinates as shown in FIG. . In the figure, when i = 1, it is the area of the rectangular parallelepiped 41, when i = 2, it is the area of the rectangular parallelepiped 42, and when i = 3, it is the area of the rectangular parallelepiped 43.
Here, the above three rectangular parallelepipeds 41 to 43 are three in the Y-axis direction.
Y12 = Y21, Y22 = Y3
It is one.

Next, skin color detection is executed for all pixels of one screen of YUV image data (step 503). That is, it is determined whether or not a pixel YUV satisfies the m rectangular parallelepiped regions in the following conditional expression (step 50).
4).

Yi1 ≦ Y ≦ Yi2 and Ui1 ≦ U ≦
Ui2 and Vi1 ≦ V ≦ Vi2 (where i = 1, 2,..., M) Each of the above conditional expressions is a region which is determined in advance to be a flesh color region. The pixel YUV existing in the represented rectangular parallelepiped area is regarded as a flesh color area, and “1” is set to the corresponding pixel on the image sampling data storage (image memory) 23 (step 505).
Note that in the above conditional expression, it is not necessary to see all the gradations from Y11 to Ym2, m may be 2 or more and in any range, and only for Y, Y12 = Y21, Y
A condition that the region is a continuous region such as 22 = Y31 is also included. There is no limitation on U and V.

On the other hand, the pixel YUV that does not correspond to any of the m rectangular parallelepiped regions is regarded as not a flesh color region, and the image sampling data storage (image memory) 2
3 is set to "0" for the corresponding pixel (step 5).
06). In this way, the flesh color area on one screen of the input image is detected by the software processing.

Thereafter, the face portion recognition processing section 29 calculates the area (the number of pixels) of the connected area for the pixel recognized as the flesh color, and among the connected areas, a total of n pieces (n: A connected region of (a predetermined integer of 2 or more) is extracted as an area to be recognized, and subsequently, pixels having lightness, hue, and saturation of hair color and pixels not having such color are divided from the input color image data.

Subsequently, the face part recognition processing unit 29
Is applied to a frame of a predetermined shape including the area inside each of the n areas. For example, 1
When the positions of the left end, right end, upper end, and lower end of one area are known, for example, as shown in FIGS. 15A and 15B, it is fitted to one large frame so that the area is included inside. In FIGS. 15A and 15B, a circle I indicates a connected area of one skin area, but the shape of the connected area is not limited to this.

Then, the left end, right end,
The upper and lower edges are fitted to one frame, and that frame is
As shown in (A), the image is vertically divided into three parts, and as shown by II, one of the three divided areas is added to obtain a vertical length. That is, the length of (the length of the upper end and the lower end of the skin label) × 4/3 is the entire length. The added one area II assumes a hair area. The skin area below the hair area is the face area. The above frame is divided into four horizontal parts as shown in FIG. On the side, the length from the right end to the left end of the skin label is divided into four equal parts.

Next, the face part recognition processing section 29 obtains three feature values based on the above-mentioned frame. The first feature value is the total number of pixels, and the second feature value is the length of the frame in the vertical direction of the screen, the y coordinate of the left shoulder, and the inside of the divided frame obtained by dividing the vertical direction into four or eight. The vertical characteristic information is composed of the ratio of the number of hair pixels to the number of pixels, the ratio of the number of skin color pixels to the number of pixels in the divided frame, and the average luminance value of the flesh color present in the divided frame. The length of the frame in the horizontal direction of the screen, the x coordinate of the left shoulder, the ratio of the number of hair pixels to the number of pixels in the divided frame inside the divided frame divided into four or eight in the horizontal direction, and the skin to the number of pixels in the divided frame Is information in the horizontal direction, which is composed of the ratio of the number of pixels and the average luminance value of the flesh color present in the divided frame.

Next, the face portion recognition processing section 29 analyzes the feature values and determines the face area according to the flowcharts of FIGS. That is, the face part recognition processing unit 29
Reads out n label numbers corresponding to the n areas (step 601 in FIG. 10), reads out feature amounts of the n areas (step 602 in FIG. 10), and based on these, n The processing of steps 603 to 653 is repeated for each of the areas (step 603 in FIG. 10). However, the number of divided frames is four.

First, 0 is substituted for a variable p (step 604 in FIG. 10), and the first skin color total number of pixels maxCn [i], which is the first characteristic amount, is reduced to a smaller empirical value maxC1.
It is determined whether it is smaller (step 60 in FIG. 10).
5) If it is small, add 0 to the variable p (step 606 in FIG. 10), and if it is equal to or greater than maxC1, a large experience value ma
It is determined whether it is smaller than xC2 (step 607 in FIG. 10). If it is smaller, 1 is added to the variable p (step 608 in FIG. 10). If it is not smaller than maxC2, 2 is added to the variable p (FIG. 10 Step 609).

That is, the total number of skin color pixels in the area is
It is divided into high, medium and low, and the larger the point, the higher the point. It should be noted that adding 0 to the variable p and updating (substantially nothing is added) is a low-level point addition, p
Updating by adding 1 to p is also referred to as medium-level point addition, and updating by adding 2 to p is also referred to as high-level point addition.

Subsequently, an aspect ratio is obtained from the length hlength in the vertical information, which is the second characteristic amount, and the length wlength in the horizontal information, which is the third characteristic amount, in the above characteristic amounts. Is determined, the vertical length hlength is
Experience value hWper1 which is large in horizontal length wlength
It is determined whether or not the value is twice or more (hwlper1 is 1.0 or more) (step 610 in FIG. 10), and if it is, the variable p is added by 2 (step 611 in FIG. 10).
If less than, the experience value h that is smaller than the horizontal length wlength
It is determined whether or not the value is equal to or greater than wlper2 times (hwlper2 is 1.0 or more) (step 61 in FIG. 10).
2) If this is the case, the variable p is added by 1 (step 613 in FIG. 10), and if it is less, the variable p is not added (step 614 in FIG. 10). The reason why the empirical values hwlper1 and hwlper2 are set to 1.0 or more is to lower the value in that case because of the fact that there are few faces that are longer on the side including the hair.

Subsequently, the y coordinate h of the left shoulder in the second feature value
start and x coordinate wstar of the left shoulder in the third feature value
At t, the subject is normally divided into a higher position closer to the center and a lower position closer to the end from the nature of being normally photographed at the center (steps 615 to 619 in FIG. 11). That is, wstart> l
im0xs or hstart> lim0ys or xen
It is determined whether d <lim0xe or yend <lim0ye (step 615 in FIG. 11). If any of these inequalities is satisfied, it is determined that the distance is farther from the center, and the variable p is not added (FIG. 11). Step 6 of
16). Where lim0xs, lim0ys, lim
0xe and lim0ye are empirical values.

If all of the inequalities in step 615 are not satisfied, wstart> lim1xs or hst
It is determined whether art> lim1ys or xend <lim1xe or yend <lim1ye (step 617 in FIG. 11). If any of these inequalities is satisfied, it is determined that the distance is relatively far from the center, and only 1 is set for the variable p. When all of these inequalities are not satisfied (step 618 in FIG. 11), it is determined that the distance is close to the center, and the variable p is incremented by 2 (step 6 in FIG. 11).
19). Where lim1xs, lim1ys, lim
1xe and lim1ye are empirical values. Also, xen
d = xstart + length, yend = yst
art + hlength.

Next, of the ratio of the number of hair pixels to the number of pixels in the divided frame in the vertical information of the second feature amount, the ratio of the divided region II corresponding to the hair shown in FIG. Array hh
It is determined whether or not airC [3] is 0 (step 620 in FIG. 11). If it is 0, the value of the variable p is left as it is (FIG. 1).
Step 621 of 1; if not 0, experience value hhair
It is determined whether it is smaller than C1 (step 6 in FIG. 11).
22), if small, add 1 to variable p and update (FIG. 11)
Step 623), if the value is equal to or greater than hairC1, the variable p is incremented by 2 and updated (step 624 in FIG. 11).
In this way, the point (the value of the variable p) decreases as the number of pixels corresponding to the hair decreases.

Next, regarding the ratio of the number of skin pixels to the number of pixels in the division frame in the vertical information of the second feature amount,
If the number of skin pixels hfaceC [0] to hfaceC [2] in the lower three divided frames (blocks) shown in FIG.
And the three blocks hfaceC [0] to hfaceC [0] to h
If all the numbers of pixels in faceC [2] are the same value, it is determined that there is no roundness, the value of the variable p is reduced, and the number of skin pixels hfaceC of the uppermost block of the four blocks is further reduced.
If [3] is larger than the number of skin pixels in the other three blocks, the value of the variable p is reduced because the top block should be hair (steps 625 and 626 in FIG. 11).

None of the above, the number of skin pixels hfaceC of the lowest block among the number of skin pixels hfaceC [0] to hfaceC [2] of the lower three blocks
When [0] is the maximum, it is determined that the probability of a face is low, and the value of the variable p is updated to medium, that is, 1 is added (steps 627 and 628 in FIG. 11). If any of the above conditions is not satisfied, it is determined that the probability that the face is a face is high, and the value of the variable p is increased, that is, 2 is added and updated (step 62 in FIG. 11).
9).

Subsequently, in the vertical information of the second feature amount, of the skin color luminance average values with respect to the number of pixels in the divided frame,
The average luminance value hf of the lower three blocks shown in FIG.
Judgment is made based on aceY [0] to hfaceY [3], and the luminance average values hfaceY [0] to hfac of these blocks are determined.
If at least one of eY [3] is 0, the value of the variable p is not updated, and the difference between the maximum value and the minimum value of the luminance is smaller than the empirical value hfaceY1 due to the nature of various luminances on the skin. Also at this time, the value of the variable p is kept as it is, and the average luminance value hfaceY
Since [0] has the lowest probability, the value of the variable p is not updated in this case as well (steps 630 and 631 in FIG. 12).
When the condition of step 330 is not satisfied, it is determined that the probability of the face is high, and the value of the variable p is increased, that is, 2 is added and updated (step 632 in FIG. 12).

Next, of the horizontal information of the third feature, whairC [0] to whairC [3] of the number of hair pixels of the four divided frames (blocks) shown in FIG.
Of two or more blocks are determined to be 0 (FIG. 1
(Step 633 of FIG. 12) If the value is 0 for two blocks or more, the value of the variable p is left as it is (Step 634 of FIG. 12).
If not, if only the number of pixels in one block is 0 among wairC [0] to wairC [3], the variable p is incremented by 1 and updated (step 636 in FIG. 12), and the hairs of all the blocks are updated. If the number of pixels is not 0, the variable p is updated by adding 2 (step 637 in FIG. 12). In this way, the point (the value of the variable p) decreases as the number of pixels corresponding to the hair decreases.

Next, of the horizontal information of the third feature value, the number of skin pixels wfaceC [0] to wfaceC in each of the four divided frames (blocks) shown in FIG.
If any of [3] is 0, the value of the variable p is left as it is (point is lowered), and if all the pixels are the same, it is determined that there is no rounding, and the value of the variable p is changed. Leave as it is (decrease the point), and further, the number of pixels wfaceC [0] and wfaceC [3] of the skin at the right end or the left end
Is larger than the maximum value and the difference between the maximum value and the minimum value is equal to or more than the predetermined empirical value wfaceC1, it is determined that there is no roundness, and the value of the variable p is left as it is (step 638 in FIG. 12). 639).

Also, of the four divided frames (blocks) obtained by horizontal division, the number of skin pixels wfac of the leftmost block is shown.
eC [0] and the number of skin pixels wfaceC in the rightmost block
If any of [3] indicates the maximum value and the difference between the maximum value and the minimum value is less than the predetermined experience value wfaceC1, 1 is added to the value of the variable p and the point is set to the middle (step in FIG. 12). 640, 641). If the condition of step 340 is not satisfied, it is determined that the probability of the face area is high, and 2 is added to the value of the variable p to increase the point (step 642 in FIG. 12).

Next, of the horizontal information of the third feature value, the average luminance values wfaceY [0] to wfa of the skin colors of the four divided frames (blocks) shown in FIG.
If the average luminance value of the skin color of any one block of ceY [3] is 0, or if the difference between the maximum value and the minimum value is less than or equal to a predetermined empirical value wfaceY1, the roundness is reduced. It is determined that there is no variable, the value of the variable p is left as it is, and the point is lowered (step 643 in FIG. 12,
644). Consideration is given to the fact that various luminances (luminance differences equal to or more than a predetermined value) exist on the skin.

If the condition of step 343 is not satisfied, it is determined whether any of the average brightness wfaceY [0] of the leftmost skin and the average brightness wfaceY [3] of the rightmost skin is the maximum (FIG. 12). Step 645), if this condition is satisfied, 1 is added to the variable p to set the point to medium (step 646 in FIG. 12). If this condition is not satisfied, it is determined that the probability of the face area is high, and The point is increased by adding 2 (step 647 in FIG. 12).

Subsequently, the value of the variable p is substituted into the total point number point [i] of the label number i at that time (FIG. 1).
3 step 648). The number n of extracted areas is “5”
If the label number is 0 to 4, the above steps 604 to 647 are repeated five times to obtain the total point number point [0] to point [4].

Subsequently, the total point number point
The maximum value among [0] to point [4] is determined as the face area and stored in the register maxcount, and the label number corresponding to the area determined as the face area is stored in the register maxnum (step in FIG. 13). 649).
Next, it is determined whether or not the value of the backlight flag refFlag is “1” (step 650 in FIG. 13). If it is “1”, the total number of points maxco of the area determined to be the face area is determined.
It is determined whether or not unt is smaller than a predetermined lower limit value refprim (step 651 in FIG. 13). If it is smaller, it is determined that the face area has not been found (step 652 in FIG. 13).

In step 650, the backlight flag refFla
When the value of g is determined to be “0”, the total number of skin color pixels maxCn [max] in one area determined to be a face area
num] is smaller than a predetermined experience value maxCn1 (step 653 in FIG. 13).
When it is 1 or more, the total point number maxcount of the area determined as the face area is set to the preset lower limit value poimt
It is determined whether it is less than lim (step 654 in FIG. 13).

The total number of skin color pixels maxCn [maxnu
m] is less than the experience value maxCn1, or
The total number of points maxcou in the area determined to be the face area
If nt is less than the lower limit poimtlim, it is determined that no face area exists (step 65 in FIG. 13).
2). This is to reduce false recognition.

MaxCn [maxnum] ≧ maxCn
1 and maxcount ≧ poimtlim, or refFlag = 1 and maxcount
If ≧ refprim, the area corresponding to the label number stored in the register maxnum is determined as the face area (step 655 in FIG. 13). Step 65
Also when it is determined that maxcount ≧ refprimm in 1, the area corresponding to the label number stored in the register maxnum is determined as the face area (step 655 in FIG. 13). The lower limit refprim is an empirical value, p
By setting the value to be smaller than the point limit, the regulation is weaker than the determination when refFlag = 0, and the skin color recognition rate is increased even when the face area is dark.

When the face part recognition process described above with reference to FIGS. 9 to 15 is completed, the tone correction curve correction unit 30 performs a tone correction curve level correction process (step 116 in FIG. 1). This gradation correction curve level correction processing will be described in more detail with reference to the flowchart of FIG. First, the average luminance Yav of the previously determined face area
e (step 701 in FIG. 16), and subsequently the values of the variables seclow and sechigh obtained in the pre-processing and the level of UP or DOWN (step 702 in FIG. 16). As a value, the levels of sechigh and UP are set as positive values, and a value obtained by combining these is stored in the register curlev as a total level of preprocessing (FIG.
Step 703).

Subsequently, the average luminance Yave of the face area is calculated as follows:
The target luminances Ytag1 and Ytag2 set in advance are compared with each other in magnitude (steps 704 and 705 in FIG. 16).
When Ytag2 <Yave or Yave <Ytag1, the level lev at which Yave falls within the range of Ytag1 to Ytag2 is predicted (step 70 in FIG. 16).
6). For example, the steps 104, 106 or 107
What is the contrast gradient or curve level created in the above, the average luminance Yave of the face area can be reduced to the target luminance Ytag.
It is predicted whether it falls within the range of 1 to Ytag2.

Normally, the level falling within the range of Ytag1 to Ytag2 and the level of step 104, 106 or 10
It is predicted that the contrast gradient and the curve level are intermediate levels created in step 7. In the case of strong backlight, Ytag1 to Ytag
It is predicted that the level itself falls within the range of g2. It is possible to roughly estimate the amount by increasing the luminance by one level by experience.

Subsequently, the integrated level curle obtained in the same manner as the above-mentioned automatic gradation correction method proposed by the present applicant.
In order to compare v with the total level lev obtained centering on the flesh color, those difference values are substituted into a variable para (step 708 in FIG. 16). Next, it is determined whether the value of refFlag is “0” (step 70 in FIG. 16).
9) When refFlag = 0, it is normal time and par
After holding down by one level, such as plus 1 when a <0 and minus 1 when para> 0 (FIG. 16)
Step 710), seclow, s obtained in the preprocessing
The level of the value of "high", "UP" or "DOWN" is comprehensively corrected by the variable "para" (step 711 in FIG. 16). In this way, a correction curve that is not completely at the level of the skin color center but is about the middle of the conventional method can be obtained.

In step 709, refFlag =
When it is determined to be 1, it is a strong backlight, so that seclow, sec.
The level of “high”, “UP” or “DOWN” is comprehensively corrected using the value of the variable “para” as it is (FIG. 16).
Step 711). No correction is made when para = 0. Note that when Ytag2 ≧ Yave ≧ Ytag1, there is no need for correction, so the value of the variable para is set to 0 (step 707 in FIG. 16), and in this case, level
Is not modified. The tone correction curve correction unit 30 outputs the thus obtained seclow, sechigh,
The UP or DOWN level is output (step 712 in FIG. 16).

Based on the Level of the gradation correction curve thus corrected, the over / under intensity judging section 33 uses the curve data * Up [1] to [5] stored in the various curve data storage sections 32. * Up [Level] of Level in [] is selected as the correction curve data, and the correction curve data and the contrast straight line data stored in the * Cont are converted by the contrast straight line and curve data synthesis processing unit 26. The images are combined and stored as a gradation correction curve in the gradation correction curve data storage unit 37 (step 117 in FIG. 1).

After the gradation correction curve is determined as described above, the gradation correction curve correction section 39 corrects the gradation correction curve based on the image sampling data (Y, U, V). The operation of correcting the gradation correction curve will be described in detail with reference to the flowchart of FIG. An image is information composed of a three-dimensional signal of (Y, U, V). When the luminance signal Y becomes Y ′ / Y times as a result of gradation correction, the color difference signals U and V are generally the same. Multiply Y '/ Y times. The same applies to the present embodiment.

The tone correction curve correcting section 39 first stores the tone correction curve (Y, Y ') from the tone correction curve data storage section 37.
(Step 801 in FIG. 17)
If the color difference signals U and V are Y '/ Y, the number of overflows and the number of overflows are calculated in advance (step 802 in FIG. 17). The color difference signals U and V are from -128 to 12
7 range. Also, limit values 127 * Y / Y 'and -128 * Y / Y' for input luminance signal Y are calculated in advance. The luminance signal Y has a value of 0 to 255. With this preparation, the image sampling data (Y, U,
Check V).

Subsequently, the number of pixels in which U and V overflow is counted by the color difference signal overflow detection unit 38 while the number of sampling data is always the same, and a variable C is obtained.
(steps 803 and 80 in FIG. 17).
4). Subsequently, the above-mentioned variable Count and threshold value UVL
compare with the limit (Step 805 in FIG. 17).
If unt <UVLimit, the tone correction curve is used as it is (step 806 in FIG. 17). Otherwise, after lowering the Level according to the value of Count, * U
p [Level] and * Cont are combined to form a new correction curve (steps 807 and 808 in FIG. 17).

The UV Limit is adjusted by experience.
UVLimit is a value greater than 0. The larger the value, the greater the color difference signal collapse. The smaller the value, the less the color difference signal collapses. The rate at which the Level is lowered is also adjusted by experience.

For example, when Level is lowered by one level, C
When the number of “outs” drops from 100 to 5, or when “1000” drops to 50, the level is lowered by considering that one level is reduced by 1/20. If UVLimit is 6, the level is lowered by one when the initial Count is 100. 10
In the case of 00, the processing is to lower by two levels. The number of pixels in the image sampling data storage unit 23 is 19
In the case of 200 pixels, when Count> 1000, the pixel is unconditionally set to linear. 1000 ≧ Count> 10
Multiply by 0.25 during 0. When Count ≦ 100, multiply by 0.03. Lowers one level each time you ride.
Multiplication and level reduction are performed until Count becomes smaller than UVLimit.

When the data in the gradation correction curve data storage unit 37 is corrected, the level is lowered in accordance with the value of the Count.
First, determine the contrast line so that
Lower until cthight becomes 0, and still target Le
If the value does not decrease to the value of vel, the curve data having a lower level than the selected curve data is selected from the various curve data obtained from the various curve data storages 32. The curve data and the contrast straight line data are combined to generate correction curve data.

The generated correction curve data, that is, the corrected gradation correction curve 809 corrects the image data from the image memory 21, and the corrected image data is stored in the image memory 2.
1 is stored again. Since this image is three-dimensional (Y, U, V), a correction curve determined from only the Y component can be corrected to obtain a well-balanced image.

The present invention is not limited to the above embodiment. For example, the input brightness level axis can be divided into n equal parts of 4 or more, and the correction level can be set in m steps of 1 or more. In the above embodiment, the processing of the blocks from the image sampling processing unit 22 to the image processing unit 40 has been described as being performed by the software of the CPU 17 in the video printer 16 in FIG.
5 may be performed. Or,
The brightness histogram feature value calculation unit 26, the brightness histogram feature value storage unit 27, the strong backlight detection unit 28, the face part recognition processing unit 29, the gradation correction curve correction unit 30, the overexposure and underdetection unit 31, the contrast gradient in FIG. Decision part 3
2. Each processing of the contrast data creation unit 35, the over / under intensity determination unit 33, various curve data storage units 34, the contrast straight line and curve data synthesis processing unit 36, the color difference signal overflow detection unit 38, and the gradation correction curve correction unit 39 May be performed by software of the CPU 17 in the video printer 16, and the other blocks may be configured by hardware.

[0106]

As described above, according to the present invention,
Divide the luminance histogram of the input color image data into equal parts a, know the approximate shape of the curve, calculate the slope information of the contrast according to the shape, and modify the gradation correction curve in consideration of the skin color. After selecting the optimal curve data by performing this, they are combined to generate gradation correction curve data by software processing, so that it is more natural and flesh color than when using the accumulated luminance distribution as it is. A gradation with an improved recognition rate can be obtained.

Further, according to the present invention, the tone correction curve is corrected in consideration of the skin color in both the normal state and the strong backlight, so that the face is not determined to be a strict decision in the normal decision. Faces can be recognized even in dark places, and in the case of strong backlighting, the correction is performed at the level center obtained by skin color recognition. Tone correction can be performed.

Further, according to the present invention, since a sophisticated technique such as a neural network is not used at the time of judgment, a complicated calculation is not required, and the cost can be reduced and the speed can be increased. Further, since the processing is software processing, it is possible to easily cope with a change in image data.

[Brief description of the drawings]

FIG. 1 is a flowchart of a main part of an embodiment of the method of the present invention.

FIG. 2 is a block diagram of an embodiment of a correction device for realizing an automatic gradation correction method according to the present invention.

FIG. 3 is a flowchart (part 1) for explaining an example of an under pre-process in FIG. 1;

FIG. 4 is a flowchart (part 2) for explaining an example of the under pre-processing of FIG. 1;

FIG. 5 is a flowchart (part 1) for explaining an example of the pre-over process of FIG. 1;

FIG. 6 is a flowchart (part 2) for explaining an example of the pre-over process of FIG. 1;

FIG. 7 is a flowchart (part 1) for explaining an example of the linear pre-processing of FIG. 1;

FIG. 8 is a flowchart (part 2) for explaining an example of the linear pre-processing of FIG. 1;

FIG. 9 is a flowchart showing details of a skin color pixel recognition process in the face portion recognition process in FIG. 1;

FIG. 10 is a flowchart (part 1) showing details of a feature value analysis process in the face portion recognition process in FIG. 1;

FIG. 11 is a flowchart (part 2) showing details of a feature value analysis process in the face portion recognition process in FIG. 1;

FIG. 12 is a flowchart (part 3) showing details of a feature amount analysis process in the face portion recognition process in FIG. 1;

FIG. 13 is a flowchart (part 4) showing details of a feature value analysis process in the face portion recognition process in FIG. 1;

14 is an explanatory diagram of an example of a rectangular parallelepiped region in the face portion recognition processing in FIG.

FIG. 15 is an explanatory diagram of a vertical division and a horizontal division in the face part recognition processing in FIG. 1;

FIG. 16 is a flowchart illustrating details of a tone correction curve level correction process in FIG. 1;

FIG. 17 is a flowchart illustrating a tone correction curve correction process according to an embodiment of the present invention.

FIG. 18 is a block diagram illustrating an example of a printing system using a color video printer.

[Description of Signs] 14 Personal Computer (PC) 15, 17 Central Processing Unit (CPU) 16 Video Printer 18, 21 Image Memory 22 Image Sampling Processing Unit 23 Image Sampling Data Storage Unit 24 Luminance Histogram Creation Unit 25 Luminance Histogram Storage Unit 26 Brightness histogram feature value calculation unit 27 Brightness histogram feature value storage unit 28 Strong backlight detection unit 29 Face part recognition processing unit 30 Tone correction curve correction unit 31 Overexposure / under detection unit 32 Contrast inclination determination unit 33 Over / under intensity judgment unit 34 Various Curve Data Storage Unit 35 Contrast Data Creation Unit 36 Contrast Straight Line and Curve Data Synthesis Unit 37 Tone Correction Curve Data Storage Unit 38 Color Difference Signal Overflow Detection Unit 39 Tone Correction Curve Correction Unit 40 Image Processing Unit 104 under preprocessing step 106 over preprocessing step 107 linear preprocessing step 115 face part recognition processing step 116 gradation correction curve level correction processing step

 ────────────────────────────────────────────────── ─── Continued on the front page (72) Inventor Kenji Tanaka 3-12 Moriyacho, Kanagawa-ku, Yokohama-shi, Kanagawa Japan F-term (reference) 5C055 AA14 BA03 BA06 BA08 CA07 EA02 EA04 HA19 HA37 5C066 AA01 AA05 BA13 CA08 CA17 EA11 EB02 EC06 EF02 GA02 GA05 HA04 JA01 KD04 KE07 KP05 5C077 LL19 MP08 NN03 PP15 PP28 PP31 PP37 PP46 PP52 PP53 PP61 PP68 PQ19 PQ22 PQ23 TT09

Claims (3)

[Claims] 1. A luminance histogram is created from color image data whose gradation is to be corrected, and the input luminance level axis is equally divided by a (a is an integer of 4 or more) from the luminance histogram. A first feature value indicating a ratio to pixels, a second feature value indicating a ratio of the number of pixels exceeding a certain limiter value to all pixels in each of the a equally divided areas, and an input brightness level from the brightness histogram. A first step of calculating a third feature amount indicating a ratio of the number of pixels of each area divided into three equal parts to all pixels, and storing the calculated third feature amount in a first storage unit; The first to third feature values stored in the first and third feature values are taken in, and the third feature value in the lowest brightness region is larger than a preset lower limit value, and the third feature value in the highest brightness region is 3 features A second step of comparing and determining whether the amount is smaller than a preset upper limit value; and the second step, wherein the third feature value of the lowest luminance side region is larger than a preset lower limit value. When a certain determination result is obtained, it is determined that the exposure is underexposure, the inclination information of the contrast is obtained using the first feature amount, and an under preprocessing for obtaining the under intensity from the inclination information is performed. In the second step, when a determination result that the third feature amount of the highest luminance side region is larger than a preset upper limit value is obtained, it is determined that overexposure is performed, and the first feature is determined. A fourth step of performing pre-over processing for obtaining over-intensity from the slope information by obtaining the slope information of the contrast by using the amount, and the third feature of the lowest luminance area by the second step. When the determination result that the amount of collection is equal to or less than a preset lower limit value and the third feature amount of the highest luminance side area is smaller than the preset upper limit value is obtained, the first feature is obtained. A fifth step of performing linear pre-processing for obtaining the intensity of the contrast in accordance with the second characteristic amount while obtaining the inclination information of the contrast using the amount, and when it is determined that the exposure is not over in the second step A sixth step of determining that the backlight is strong backlight and setting a backlight flag to a predetermined value when the second feature value is a positive value in both the highest region and the lowest region of the input luminance level; It is determined whether or not the color image data is included in a region where the brightness, hue, and saturation are skin colors, and when the backlight flag is a predetermined value, the region that is the skin color is smaller than when the backlight flag is not the predetermined value. Brightness level And determining the area of each connected region with respect to the flesh-colored pixel detected in the seventh step, and among the connected regions, a total of n (from the largest area to the largest area) (n is a predetermined integer of 2 or more) is extracted as an area to be recognized, and each of the n areas is applied to a frame of a predetermined shape including the area therein, and Assuming a frame in which a hair area is added to the upper part, a fourth number indicating the total number of skin color pixels for each of the n frames.
, A fifth feature value indicating information of the frame in the vertical direction of the screen, and a sixth feature value indicating information of the frame in the horizontal direction of the screen. Eighth step of analyzing a feature amount and determining one area having the highest probability of being a face area as a face area; and calculating an average luminance of the face area determined in the eighth step. A comparison is made as to whether or not the average luminance falls within a preset target luminance range. Based on the comparison result, the contrast inclination information and the intensity information created in the third, fourth, or fifth step are compared with the average luminance. A ninth step of correcting the luminance to a value predicted so that the luminance falls within the target luminance range, and various characteristic curves of the output gradation with respect to the input gradation according to the intensity corrected in the ninth step. The various curve data shown are stored in advance. Curve data is selected from the second storage unit, and the selected curve data is combined with the gradient information of the contrast corrected in the ninth step to generate gradation correction curve data. A tenth step of storing the color image data to be corrected in the storage unit; and reading the color image data to be corrected from the image memory in which the color image data to be corrected is stored from the third storage unit. An eleventh step of reflecting the gradation correction curve data and storing the data again in the image memory. 2. The method according to claim 1, wherein the seventh step comprises:
V of the color image data in the three-dimensional color space of Y
In a m-shaped (m is an integer of 2 or more) rectangular parallelepiped area, which indicates a preset skin color area in the three-dimensional coordinate space of the axes, U-axis, and V-axis and has the same luminance boundary value between adjacent rectangular parallelepiped areas 2. The automatic gradation correction method according to claim 1, wherein the pixels included in the image data are obtained as pixels whose brightness, hue, and saturation are skin colors. 3. The eighth step, wherein when the backlight flag is the predetermined value, the probability value of one area having the highest probability of being an area indicating the face is determined by a first predetermined value.
When the area is equal to or more than the lower limit value, the one area is determined to be a face area, and when the backlight flag is not the predetermined value,
The total number of skin color pixels in one area having the highest probability of being the area indicating the face is equal to or greater than a predetermined empirical value, and the probability value of the one area is larger than the first lower limit. 2. The automatic gradation correction method according to claim 1, wherein the one area is determined as a face area only when the area is equal to or more than a set second lower limit.