JP2004192129A - Method and device for extracting facial area - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a method for extracting a facial area by which the facial area can be extracted with precision under various photographing situations in consideration of an error in determining a photographing light source, and also to provide a device therefor. <P>SOLUTION: Image data acquired with a digital camera is analyzed so as to determine the photographing light source of the image based on the color information, etc. (step S114). Lightnesses obtained under the respective light sources are assumed by the characteristic of an artificial light source such as an electric bulb, or a fluorescent lamp, or a natural light source such as outdoors in shade, or fine weather, so that accuracy in determining the light sources is determined by the luminance of a subject (step S116). A flesh color table for detecting the optimum flesh color is automatically selected based on information by determined light source and subject luminance information (step S118). An area having the hue of the flesh color in the image is detected through the use of the flesh color table, and, then, the acquired flesh color area is divided by color saturation. Shape recognition is performed about each flesh color area divided by color saturation, so that the final facial area is extracted. <P>COPYRIGHT: (C)2004,JPO&NCIPI

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a face area extraction method and apparatus, and more particularly to a method and apparatus for extracting an area corresponding to a human face existing in a color image acquired by a digital camera or the like.
[0002]
[Prior art]
The most noticeable part when appreciating a portrait is the face of the person. There has been proposed a technique for automatically detecting a region corresponding to a human face in an image so that the human face is reproduced with appropriate brightness and color (Patent Document 1). According to the method disclosed in Patent Document 1, a face color distribution model is selected according to the illumination condition at the time of shooting, and face extraction is performed.
[0003]
[Patent Document 1]
JP 2001-167273 A
[0004]
[Problems to be solved by the invention]
However, Patent Document 1 only mentions that the camera photographing conditions (strobe ON / OFF, white balance, whiro balance mode, aperture, shutter speed, AGC, AE, gamma, pedestal level) are used for determining the illumination condition. It does not disclose what is actually the problem and how to set the facial color distribution model.
[0005]
The auto white balance (AWB) function installed in a general digital camera automatically determines the light source type from the captured image data, but it is difficult to accurately grasp the lighting conditions automatically, and lighting conditions to some extent It is necessary to allow a judgment error. In consideration of the detection variation of the photographic light source, it is necessary to extract a face with a wide range of skin color hues. This is also true for the data after the AWB result.
[0006]
That is, in the AWB process, it is impossible to correctly determine the type of light source, so the white balance correction amount is weakened to some extent with respect to the estimated light source. As a result, the light source color remains uncorrected. That is, white does not become white, and as a result, the skin color varies depending on the illumination light source. Therefore, when a face is extracted with a wide range of skin color hues, there is a problem that the frequency of erroneous detection of objects other than the face increases.
[0007]
The present invention has been made in view of such circumstances, and provides a face area extraction method and apparatus capable of accurately extracting a face area under various shooting conditions in consideration of detection variation of an illumination light source. For the purpose.
[0008]
[Means for Solving the Problems]
In order to achieve the above object, a face area extracting method according to the present invention is a method for extracting an area corresponding to a human face from an image, and determining a light source at the time of photographing based on image data And a accuracy determination step for determining the accuracy of light source determination in the light source determination step, and a skin color condition that defines a skin color hue range necessary for skin color detection is obtained from the light source information obtained from the light source determination step and the accuracy determination step. A skin color condition setting step that is set according to the accuracy information; and a skin color region detection step that detects a region having a skin color hue from within the image in accordance with the skin color condition set in the skin color condition setting step. To do.
[0009]
According to the present invention, image data is analyzed, and a photographing light source of the image is estimated from the color information and the like. Since it is difficult to determine a photographing light source completely automatically, the skin color condition is variably set so as to allow an error in light source determination in consideration of the accuracy (accuracy) of the estimated light source. The skin color condition defines the condition of the hue to be detected as the skin color, and this skin color condition is optimized according to the light source. Thereby, compared with the conventional method which applies the skin color condition common to all the light sources, the false detection of a skin color can be reduced and a face extraction with high precision is attained.
[0010]
If the skin color condition is represented by a region (area or volume) in a certain color coordinate system, the size, center position, or shape of the skin color condition region is changed according to the estimated photographing light source and its accuracy. If the accuracy of the light source determination is high, the skin color condition area can be set relatively small, so that the skin color detection accuracy is improved.
[0011]
A face region extraction method according to an aspect of the present invention includes a subject brightness detection step of detecting subject brightness, and the accuracy determination step determines the accuracy of the light source determination according to the subject brightness detected in the subject brightness detection step. It is characterized by determining.
[0012]
The subject brightness (brightness) obtained under each light source can be assumed from the characteristics of artificial light sources such as light bulbs (tungsten) and fluorescent lamps, or natural light sources such as outdoor shade and sunny weather. The accuracy of light source determination can be determined.
[0013]
According to another aspect of the present invention, the skin color region detection step includes a saturation division step of further dividing the skin color portion detected based on the skin color condition by saturation.
[0014]
Further, there is an aspect including a face area determining step of determining a face area based on the shape of the skin color area for each saturation divided in the saturation dividing step.
[0015]
An area having a flesh-colored hue (skin-color area) in an image is further divided into areas according to saturation, and the shape and size thereof are recognized, thereby facilitating determination of a face area.
[0016]
In order to identify a true face area from among the skin color areas (facial area candidates) subjected to saturation division, it is preferable to narrow down the face area based on the shape of the area. For example, a person's face is considered to be approximately circular or elliptical, and an aspect ratio for determining whether or not the face is a face region can be determined. Those that are extremely different from the specified aspect ratio (extremely long and narrow) for each candidate for the face area are excluded as non-faces, and those that are close to the specified aspect ratio are determined to be faces.
[0017]
In order to provide an apparatus that embodies the above method invention, a face area extraction apparatus according to the present invention is an apparatus that extracts an area corresponding to a human face from an image, and is a light source at the time of photographing based on image data A light source determination unit that determines the accuracy of light source determination by the light source determination unit, a light source information obtained from the light source determination unit and a skin color condition that defines a hue range of a skin color necessary for skin color detection and the light source information Skin color condition setting means for setting according to the accuracy information obtained from the accuracy determining means, skin color area detecting means for detecting an area having a skin color hue from the image in accordance with the skin color condition set by the skin color condition setting means, It is characterized by including.
[0018]
The face area extraction apparatus of the present invention can be incorporated in a signal processing unit of an electronic photographing apparatus (electronic camera) such as a digital camera or a video camera, and reproduces or displays or prints out image data recorded by the electronic camera. It can be incorporated into an image processing apparatus or the like.
[0019]
The face area extraction apparatus of the present invention can be realized by a computer, and a program for realizing the steps of the above-described face area extraction method by a computer is stored on a CD-ROM, a magnetic disk, or other recording medium. The program can be recorded and provided to a third party through a recording medium, or the program download service can be provided through a communication line such as the Internet.
[0020]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, preferred embodiments of a face area extracting method and apparatus according to the present invention will be described in detail with reference to the accompanying drawings.
[0021]
FIG. 1 is a block diagram showing a configuration of an electronic camera according to an embodiment of the present invention. This camera 10 is a digital camera that converts an optical image of a subject into digital image data and records it on a recording medium 12, and the face region of the present invention is part of a signal processing means that processes an image signal obtained by photographing. An extraction device is used.
[0022]
The overall operation of the camera 10 is centrally controlled by a central processing unit (CPU) 14 built in the camera. The CPU 14 functions as a control means for controlling the camera system according to a predetermined program, and various types such as automatic exposure (AE) calculation, automatic focus adjustment (AF) calculation, auto white balance (AWB) control, face area extraction calculation, etc. It functions as a calculation means for performing a calculation.
[0023]
The CPU 14 is connected to a ROM 20 and a memory (RAM) 22 via a bus 16. The ROM 20 stores a program executed by the CPU 14 and various data necessary for control. The memory 22 is used as a program development area and a calculation work area for the CPU 14, and is also used as a temporary storage area for image data.
[0024]
Further, an EEPROM 24 is connected to the CPU 14. The EEPROM 24 is a non-volatile storage means for storing a skin color determination table (skin color table) necessary for face area extraction processing, data necessary for control of AE, AF, AWB, etc., or customization information set by the user. Yes, the stored contents are retained even when the power is turned off. The CPU 14 performs calculations and the like with reference to data in the EEPROM 24 as necessary. The ROM 20 may be non-rewritable or may be rewritable like an EEPROM.
[0025]
The camera 10 is provided with an operation unit 30 for a user to input various commands. The operation unit 30 includes various operation units such as a macro button 31, a shutter button 32, and a zoom switch 33.
[0026]
The macro button 31 is an operation means for setting (ON) / releasing (OFF) a macro mode suitable for short-distance shooting. Macro mode allows you to take close-up photos with a relatively shallow depth of field and a beautifully blurred background. When the camera 10 is set to the macro mode by pressing the macro button 31, focus control suitable for short-distance shooting is performed, and shooting is possible within a subject distance range of about 20 cm to 80 cm.
[0027]
The shutter button 32 is an operation means for inputting an instruction to start photographing, and is configured by a two-stroke switch having an S1 switch that is turned on when half-pressed and an S2 switch that is turned on when fully pressed. When S1 is on, AE and AF processing are performed, and when S2 is on, recording exposure is performed. The zoom switch 33 is an operation means for changing the photographing magnification and the reproduction magnification.
[0028]
Although not shown, the operation unit 30 includes a mode selection unit for switching between the shooting mode and the playback mode, and an optimum operation mode (continuous shooting mode, auto shooting mode, manual shooting mode, portrait mode) according to the shooting purpose. Shooting mode setting means for setting a landscape mode, night view mode, etc., a menu button for displaying a menu screen on the LCD monitor 40, a cross button for selecting a desired item from the menu screen (cursor moving operation means), and confirmation of the selected item In addition, an operation button such as an OK button for instructing execution of processing, an erasing of a desired target such as a selection item, cancellation of instruction contents, or a cancel button for inputting a command for returning to the previous operation state is also included. The operation unit 30 is not limited to a push-type switch member, dial member, lever switch, or the like, but also includes a unit realized by a user interface that selects a desired item from a menu screen. It is.
[0029]
A signal from the operation unit 30 is input to the CPU 14. The CPU 14 controls each circuit of the camera 10 based on an input signal from the operation unit 30. For example, lens driving control, photographing operation control, image processing control, image data recording / reproduction control, display control of the liquid crystal monitor 40, and the like. I do.
[0030]
The liquid crystal monitor 40 can be used as an electronic viewfinder for checking the angle of view at the time of shooting, and is used as a means for reproducing and displaying a recorded image. The liquid crystal monitor 40 is also used as a user interface display screen, and displays information such as menu information, selection items, and setting contents as necessary. Instead of a liquid crystal display, other types of display devices (display means) such as an organic EL can be used.
[0031]
Next, the shooting function of the camera 10 will be described.
[0032]
The camera 10 includes a photographing lens 42 as a photographing optical system and a CCD solid-state imaging device (hereinafter referred to as a CCD) 44. Instead of the CCD 44, other types of image pickup devices such as MOS solid-state image pickup devices can be used. The photographic lens 42 is composed of an electric zoom lens. Although a detailed optical configuration is not shown in the drawing, a magnifying lens group and a correction lens group that mainly provide an effect of changing magnification (variable focal length), and focus adjustment. And a contributing focus lens.
[0033]
When the photographer operates the zoom switch 33, a control signal is output from the CPU 14 to the zoom drive unit 46 in accordance with the switch operation. The zoom drive unit 46 is an electric drive unit including a motor (zoom motor) serving as a power source and a drive circuit thereof. The motor driving circuit of the zoom driving unit 46 generates a lens driving signal based on the control signal from the CPU 14 and supplies the lens driving signal to the zoom motor. Thus, the zoom motor operates by the motor drive voltage output from the motor drive circuit, and the zoom lens group and the correction lens group in the photographic lens 42 move back and forth along the optical axis, so that the focal length of the photographic lens 42 is increased. (Optical zoom magnification) is changed.
[0034]
In this example, it is assumed that the focal length of the photographing lens 42 can be varied in 10 steps within the zoom operation range from the wide (wide angle) end to the tele (telephoto) end. The photographer can select a desired focal length according to the shooting purpose and perform shooting.
[0035]
The zoom position (corresponding to the focal length) of the photographic lens 42 is detected by the zoom position detection sensor 48 and the detection signal is notified to the CPU 14. The CPU 14 can grasp the current zoom position (that is, focal length) from the signal from the zoom position detection sensor 48. The zoom position detection sensor 48 may be a circuit that generates a pulse by the rotation of a zoom motor or the like, or may have a configuration in which a position detection encode plate is disposed on the outer periphery of the lens barrel. At that time, there is no particular limitation.
[0036]
The light that has passed through the photographing lens 42 is incident on the CCD 44 after the amount of light is adjusted through a diaphragm mechanism (not shown). A large number of photosensors (light receiving elements) are arranged in a plane on the light receiving surface of the CCD 44, and primary color filters of red (R), green (G), and blue (B) are arranged in a predetermined arrangement corresponding to each photosensor. Arranged in a structure (Bayer, G stripe, etc.).
[0037]
The subject image formed on the light receiving surface of the CCD 44 is converted into a signal charge of an amount corresponding to the amount of incident light by each photosensor. The CCD 44 has an electronic shutter function that controls the charge accumulation time (shutter speed) of each photosensor according to the timing of the shutter gate pulse.
[0038]
The signal charges accumulated in the photosensors of the CCD 44 are sequentially read out as voltage signals (image signals) corresponding to the signal charges based on the pulses given from the CCD driver 50. The image signal output from the CCD 44 is sent to the analog processing unit 52. The analog processing unit 52 is a preceding processing unit including a CDS (correlated double sampling) circuit and a gain adjustment circuit. In this analog processing unit 52, sampling processing and color separation processing are performed on each of the R, G, and B color signals, The signal level of each color signal is adjusted (pre-white balance processing).
[0039]
The image signal output from the analog processing unit 52 is converted into a digital signal by the A / D converter 54 and then stored in the memory 22 via the signal processing unit 56. The image data stored in the memory 22 at this time is an A / D conversion output of the image signal output from the CCD 44 recorded as it is (unprocessed), and is subjected to signal processing such as gamma conversion and synchronization. This is image data that is not known (hereinafter referred to as CCD RAW data). However, “raw data” does not exclude any signal processing. For example, an image obtained by performing defective pixel correction processing that interpolates defective pixel data of an image sensor. Data and the like are included in the concept of CCD RAW data in that they are not expanded into a general-purpose format.
[0040]
A timing generator (TG) 58 provides timing signals to the CCD driver 50, the analog processing unit 52, and the A / D converter 54 in accordance with instructions from the CPU 14, and the circuits are synchronized by this timing signal. .
[0041]
The signal processing unit 56 is a digital signal processing block that also serves as a memory controller that controls reading and writing of the memory 22. The signal processing unit 56 calculates the color of each point by interpolating a spatial shift of the color signal associated with the color filter array of the single-chip CCD and an auto calculation unit that performs AE / AF / AWB processing. Processing circuit), a white balance circuit, a gamma conversion circuit, a luminance / color difference signal generation circuit, a contour correction circuit, a contrast correction circuit, and the like, and processes image signals using the memory 22 in accordance with commands from the CPU 14 To do.
[0042]
The CCD RAW data stored in the memory 22 is sent to the signal processing unit 56 via the bus 16. The image data input to the signal processing unit 56 is subjected to predetermined signal processing such as white balance adjustment processing, gamma conversion processing, conversion processing to luminance signals (Y signals) and color difference signals (Cr, Cb signals) (YC processing). Is stored in the memory 22.
[0043]
When the captured image is output to the monitor, the image data is read from the memory 22 and transferred to the display circuit 60. The image data sent to the display circuit 60 is converted into a predetermined display signal (for example, an NTSC color composite video signal) and then output to the liquid crystal monitor 40. The image data in the memory 22 is periodically rewritten by the image signal output from the CCD 44, and the video signal generated from the image data is supplied to the liquid crystal monitor 40, so that the video being captured (through image) can be obtained. It is displayed on the liquid crystal monitor 40 in real time. The photographer can check the angle of view (composition) from the video (so-called through movie) displayed on the liquid crystal monitor 40.
[0044]
When the photographer determines the angle of view and presses the shutter button 32, the CPU 14 detects this, performs AE processing and AF processing in response to half-pressing of the shutter button 32 (S1 ON), and fully presses the shutter button 32. In response to (S2 = ON), CCD exposure and readout control for capturing an image for recording is started.
[0045]
The AF control in the camera 10 is, for example, contrast AF that moves the focus lens (a moving lens that contributes to focus adjustment among the lens optical systems constituting the photographing lens 42) so that the high-frequency component of the G signal of the video signal is maximized. Applies. That is, the AF calculation unit is a high-pass filter that passes only a high-frequency component of the G signal, an absolute value processing unit, and an AF area that cuts out a signal in a focus target area set in advance in the screen (for example, the center of the screen). An extraction unit and an integration unit that integrates absolute value data in the AF area are configured.
[0046]
The integrated value data obtained by the AF calculation unit is notified to the CPU 14. The CPU 14 calculates a focus evaluation value (AF evaluation value) at a plurality of AF detection points while moving the focus lens by controlling the focus driving unit 62 including the AF motor, and calculates the AF calculated at each AF detection point. From the evaluation value, the lens position where the value is maximized is determined as the focus position. Then, the focus driving unit 62 is controlled to move the focus lens to the obtained in-focus position.
[0047]
In relation to the AE control, the AE calculation unit includes a circuit that divides one screen into a plurality of areas (for example, 8 × 8) and integrates the RGB signals for each divided area, and provides the integrated value to the CPU 14. To do. The CPU 14 performs weighted addition based on the integrated value obtained from the AE calculation unit, detects the brightness of the subject (subject brightness), and calculates an exposure value (shooting EV value) suitable for shooting.
[0048]
Next, a method for calculating the photographing EV value will be described.
[0049]
One screen of the photographed image is divided into a plurality of areas (for example, 64 blocks of 8 × 8), and the EV value (EVi) of each divided area is calculated based on the integrated value obtained from the R, G, and B signals for each divided area. Ask. Subsequently, as shown in FIG. 2, the EV value of each divided area is weighted corresponding to the shooting mode, and the EV ′ value of the entire screen is calculated by the following equation.
[0050]
EV ′ = log ₂ {Σ (W _i × 2 ^EVi ) / ΣW _i }
However, i: 0 to 63 (subscript indicating an 8 × 8 divided area)
W _i : Weighting factor for each divided area corresponding to the shooting mode, that is, when the shooting mode is auto / person mode, the center-weighted metering method is used as shown in the weighting coefficient of FIG. In this case, as shown in FIG. 2 (B), the photometry method is obtained by reducing the weighting of the divided areas located on the outermost periphery. In the night view mode, the average photometry method is used as shown in FIG. 2 (C). .
[0051]
The EV ′ value calculated as described above is further subjected to exposure correction ΔEV corresponding to the shooting mode as shown in the following equation to obtain the shooting EV value.
[0052]
## EQU2 ## EV = EV′−ΔEV
Note that ΔEV is, for example, ΔEV = 0 in the portrait mode, and ΔEV = 0.3 in the landscape mode and the night view mode.
[0053]
The aperture value and shutter speed at the time of shooting are determined based on the shooting EV value obtained as described above.
[0054]
Then, when the shutter button 32 is fully pressed, the aperture mechanism is driven through an aperture drive unit (not shown) so that the determined aperture value is obtained, and charge is accumulated by the electronic shutter so that the determined shutter speed is achieved. Control the time.
[0055]
The image data taken in response to the shutter button 32 being fully pressed (S2 = ON) is subjected to YC processing and other predetermined signal processing in the signal processing unit 56 shown in FIG. In accordance with the compression format (for example, JPEG method). The compressed image data is recorded on the recording medium 12 via the media interface unit 66. The compression format is not limited to JPEG, and MPEG and other methods may be adopted.
[0056]
As a means for storing image data, various media such as a semiconductor memory card represented by SmartMedia (trademark), compact flash (trademark), a magnetic disk, an optical disk, and a magneto-optical disk can be used. Further, the recording medium (internal memory) built in the camera 10 is not limited to the removable medium.
[0057]
When the playback mode is selected by the mode selection means, the last image file (last recorded file) recorded on the recording medium 12 is read. The image file data read from the recording medium 12 is decompressed by the compression / decompression circuit 64 and output to the liquid crystal monitor 40 via the display circuit 60.
[0058]
By operating the cross button during single-frame playback in the playback mode, it is possible to advance the frame in the forward direction or in the reverse direction, and the next file after the frame advance is read from the recording medium 12, and the image is processed in the same manner as described above. Is played.
[0059]
FIG. 3 is a principal block diagram related to face extraction processing in the camera 10 of this example. 3 that are the same as those described in FIG. 1 are denoted by the same reference numerals, and description thereof is omitted.
[0060]
In FIG. 3, an integration circuit 70 and a white balance circuit 72 in the upper stage are processing systems used for normal auto white balance processing. Further, the integration circuit 74, white balance circuit 76, and synchronization circuit 78 shown in the lower part are processing units (hereinafter referred to as a preprocessing system) for performing preprocessing of the face extraction algorithm.
[0061]
The CCD RAW data output from the A / D converter 18 when the shutter button 32 is fully pressed is stored in the memory 22 and sent from the memory 22 to the integrating circuit 70. The integrating circuit 70 includes a circuit that divides one screen into a plurality of areas (for example, 8 × 8 64 blocks) and calculates an average integrated value for each color of the RGB signal for each area, and the calculation result is a reference. It is sent to the white balance circuit 72 assuming a light source (daylight). The signal whose gain has been adjusted by the white balance circuit 72 is provided to the CPU 14.
[0062]
The CPU 14 obtains an integrated value of R, an integrated value of B, and an integrated value of G, obtains a ratio of R / G and B / G, and takes these R / G and B / G values and a shooting EV by AE calculation. Based on the value information (subject luminance LV value = EV value), scene discrimination (light source type discrimination) is performed, and signal processing is performed according to a predetermined white balance adjustment value (setting that leaves the light source atmosphere) suitable for the scene. The amplifier gains of the white balance circuit 80 and the pre-processing white balance circuit 76 in the unit 56 are controlled to correct the signals of the respective color channels. As a method for discriminating a light source type and a white balance control method for leaving an atmosphere of a light source, a method disclosed in Japanese Patent Application Laid-Open No. 2000-224608 can be used.
[0063]
The CCD RAW data acquired in response to S2 = ON is sent from the memory 22 to the signal processing unit 56 and the pre-processing system integration circuit 74 for face extraction. The preprocessing integration circuit 74 divides one screen into, for example, 130 × 190 areas, and calculates an integrated value for each area. The calculation result of the integration circuit 74 is sent to the white balance circuit 76, where white balance processing reflecting AWB is performed. Data with AWB applied is sent to the synchronization circuit 78, where data of the same number of pixels (3-plane data) is generated for each of R, G, and B.
[0064]
Based on the RGB three-plane data generated by the synchronization circuit 78, the CPU 14 executes skin color detection, saturation division, face candidate extraction, and face area specification processing based on the shape of the face candidate area. Details of the face extraction algorithm will be described later.
[0065]
On the other hand, the CCD RAW data sent from the memory 22 to the signal processing unit 56 is subjected to processing reflecting AWB by the white balance circuit 80 and then sent to the gamma conversion circuit 82. The gamma conversion circuit 82 changes the input / output characteristics so that the white balance adjusted RGB signal has a desired gamma characteristic, and outputs it to the luminance / color difference signal generation circuit 84.
[0066]
The luminance / chrominance signal generation circuit 84 creates a luminance signal Y and chroma signals Cr, Cb from the gamma-corrected R, G, B signals. The luminance signal Y and chroma signals Cr and Cb (YC signal) are compressed by the compression / decompression circuit 64 in a predetermined format such as JPEG and recorded on the recording medium 12.
[0067]
Next, the face extraction function installed in the camera 10 according to the present embodiment will be described.
[0068]
FIG. 4 is a flowchart showing a sequence for determining a skin color table that defines a hue range to be extracted as a skin color. As shown in the figure, AE and AF processing is executed by confirming S1 = ON of the shutter button 32 (step S110), CCDRAW data is taken in response to S2 = ON, and this is temporarily recorded in the memory 22 ( Step S112).
[0069]
Thereafter, the light source type is determined based on the acquired CCD RAW data (step S114). Also, information on the subject brightness obtained by the AE process in step S110 is acquired (step S116), and a skin color table is selected from the light source information obtained in step S114 and the subject brightness information obtained in step S116 (step S118).
[0070]
The photographing light source is determined by the following method.
[0071]
First, the luminance information of the subject is obtained by weighted addition from a photometric value based on S1 = ON of the shutter button 32.
[0072]
The color information is handled by applying a daylight white balance to the captured image and standardizing the white balance at the daylight. At the time of daylight, R / G and B / G are in the form of white balance, so R / G = B / G = 1.
[0073]
On the other hand, since tungsten is red, R / G is large and B / G is small. On the other hand, since the shade has a high color temperature, the R / G is small and the B / G is large. In this way, the light source color can be determined using a color system having R / G and B / G as coordinate axes.
[0074]
FIG. 5 shows an example of detecting the light source color. This figure is a color coordinate system with the origin at (R / G, B / G) = (1, 1). The shade detection frame, the daylight color fluorescent light detection frame, the daylight detection frame, the day white fluorescent light detection frame, and the tungsten detection frame shown in the figure respectively define the color distribution ranges of the light source types. Such detection frame data is stored in the camera 10.
[0075]
When determining the light source, one screen of the captured image is divided into a plurality of areas (for example, 8 × 8), and an average integrated value for each color of the R, G, and B signals is obtained for each divided area. A ratio R / G with the integrated value of the G signal and a ratio B / G between the integrated value of the B signal and the integrated value of the G signal are obtained.
[0076]
Thus, (R / G, B / G) obtained for each divided area is compared with each detection frame in FIG. 5 to determine the number of divided areas that fall within each detection frame. Are determined to be the light source of the scene.
[0077]
By the way, for example, since a fluorescent lamp scene tends to include green, both R / G and B / G are smaller than “1”. However, since the outdoor green scene also shows the same result (R / G and B / G are both values smaller than “1”), it is difficult to separate the light source color and the object color from only the color information. Therefore, in the present embodiment, light source determination is performed using not only color information but also subject luminance information.
[0078]
That is, since it is unlikely that a fluorescent lamp is extremely bright, green that is brighter than a predetermined brightness is estimated not as a fluorescent lamp but as an outdoor leaf green. Reflecting such light source determination, the skin color table is also set from outside.
[0079]
FIG. 6 is a chart that defines the accuracy of light source determination used for selecting the skin color table. According to the figure, the level of accuracy of the light source is ranked according to the subject brightness (expressed by LV value). Rank A indicates that the accuracy of the light source is high, and rank B indicates that the accuracy of the light source is low.
[0080]
For example, if the subject luminance is lower than 11 LV with respect to the light source determined to be “day white fluorescent lamp”, the light source (day white fluorescent lamp) has a high degree of accuracy, and if the subject luminance is 11 LV or higher, the light source The accuracy of is low. However, when it is determined as “clear (daylight)”, the ranks of the accuracy are not classified and the entire area is not changed.
[0081]
The camera 10 includes a skin color table corresponding to the distribution of hues included in the image and the subject brightness (LV value), and performs control for automatically switching the skin color table according to the light source discrimination information (hue, brightness). Is going.
[0082]
FIG. 7 is a diagram illustrating a skin color table (for A rank) set for each light source type. In the coordinate system where the horizontal axis is R / G and the vertical axis is B / G, the skin color table selected at the rank A of each light source is set as a range (skin color extraction area) surrounded by a rectangular frame as shown in the figure. Has been.
[0083]
FIG. 8 is an example of a skin color table selected at the time of rank B of the daylight white fluorescent lamp. As shown in the figure, the skin color table (shown by a solid line) 90 of the day white fluorescent lamp B rank has a larger area than the skin color table 91 (shown by a dashed line) of the day white fluorescent lamp A rank. The range includes a flesh color table 91 of daylight white fluorescent lamp A rank and a clear flesh color table 92 (illustrated by dotted lines) 92.
[0084]
FIG. 9 is an example of a skin color table selected when the light bulb (tungsten) is ranked B. As shown in the drawing, the skin color table 94 (shown by a solid line) of the light bulb B rank has a larger area than the skin color table 95 (shown by a two-dot chain line) 95 of the light bulb A rank. The range includes a skin color table 95 and a clear skin color table (illustrated by dotted lines) 92.
[0085]
Although the B-rank skin color table is not shown for other light sources, the B-rank skin color table is more similar to the A-rank skin color table for other light sources (excluding daylight) as in the examples described in FIGS. Even a large range is set.
[0086]
7 to 9 exemplify a rectangular skin color table, the shape of the skin color table is not limited to a rectangle, and various shapes can be designed.
[0087]
A plurality of skin color tables are prepared in the camera 10 corresponding to the light source type and the accuracy of the determination, and the skin color table is automatically selected according to the automatically determined photographing light source and the accuracy, and is within the range of the skin color table. The entered hue point is detected as “skin color”.
[0088]
FIG. 10 is a flowchart showing a face extraction sequence.
[0089]
First, through S1 = ON and S2 = ON of the shutter button 32, CCD RAW data is recorded in the memory 22 (step S210). Then, processing by the preprocessing system (74, 76, 78) described in FIG. 3 is performed, gain correction processing by auto white balance is performed on the recorded CCD RAW data (step S212 in FIG. 10), and integration processing (step S214). ) And a synchronization process are performed (step S216). In this way, RGB three plane data is generated.
[0090]
Next, the CPU 14 acquires a skin color table from the EEPROM 24 (step S218). At this time, as described with reference to FIGS. 4 to 9, the skin color table is selected according to the light source type and its accuracy.
[0091]
Thereafter, the CPU 14 detects the hue in the skin color table based on the RGB three-plane data (linear system data before applying gamma) acquired from the preprocessing system synchronization circuit 78 (step in FIG. 10). S220).
[0092]
For example, when a range surrounded by a rectangular frame indicated by reference numeral 100 in FIG. 11 is set as a skin color extraction area (corresponding to a skin color table), a part within this range is determined as a skin color.
[0093]
Furthermore, the skin color extraction area 100 is further divided into a plurality of areas depending on the saturation. In FIG. 11, “saturation” is represented by a distance from the origin O, and the saturation increases as the distance from the origin O increases. In the example of the figure, the skin color detection area 100 is divided into six regions by a concentric boundary line (illustrated by a broken line) centered on the origin.
[0094]
After detecting the skin color area from the RGB three-surface data acquired from the preprocessing system synchronization circuit 78, the CPU 14 further divides the detected skin color area based on the saturation (step S222 in FIG. 10). .
[0095]
FIG. 12 shows an example in which the skin color detected area is further divided by saturation. Different objects in the image have different saturations, and generally the skin color of a person's face tends to be higher in saturation than white objects or trees such as desks under a tungsten light source. Therefore, it becomes easy to discriminate whether it is a face region or a region other than the face by finely dividing the skin color detected region according to the saturation and grasping the shape of the region.
[0096]
After performing the area division based on the saturation in step S222 in FIG. 10, the process proceeds to step S224. In step S224, the shape is further detected for each region from those divided by saturation (face region candidates), and the face region is identified from the shape.
[0097]
That is, when the aspect ratio specified by the model shape (ellipse or circle) appropriate for the face is compared with the shape of the face area candidate, the aspect ratio of the detected shape is significantly different from the specified value. , It is determined that the region is other than the face.
[0098]
As for the aspect ratio of the face area candidate, for example, as shown in FIG. 13, H (vertical value) and W (horizontal value) are obtained from the vertical and horizontal histograms for the face area candidate 102, and the ratio H / W is calculated. A face shape is determined when the aspect ratio value (H / W) is within a certain range (a shape close to an ellipse or a circle).
[0099]
Face candidates are finally narrowed down by such shape recognition, and an area portion having a shape having a predetermined aspect ratio is extracted as a “face”.
[0100]
The face area extraction result is used for brightness correction, white balance correction, color correction for bringing the skin color closer to the best color (target value), red-eye correction, and the like.
[0101]
[Modification]
In the above-described embodiment, the light source color is estimated from the hue distribution included in the image, and the accuracy is determined by the subject luminance value (LV value). However, in the implementation of the present invention, the accuracy can be determined from the hue distribution. It is. For example, when determining the light source color described with reference to the drawing, the accuracy of the light source may be determined based on the frequency (number) of divided areas included in each detection frame. Of course, the accuracy may be determined by combining the subject luminance value and the hue distribution.
[0102]
In the above embodiment, the color system of (R / G, B / G) has been described. However, in the implementation of the present invention, the color space is not particularly limited, and the RGB system, CIE L * a * b, CIE L * u * v and other color spaces may be used.
[0103]
In the above-described embodiment, the digital camera is exemplified. However, the scope of application of the present invention is not limited to this, and other electronic imaging functions such as a mobile phone with camera, a PDA with camera, and a mobile personal computer with camera are provided. The present invention can also be applied to information equipment. In this case, the imaging unit may be a detachable (external type) that is separable from a main body such as a mobile phone. Furthermore, the present invention can also be applied to an image reproducing apparatus that reproduces and displays image data recorded by the above-described device with an electronic imaging function, or a printing apparatus that prints out.
[0104]
【The invention's effect】
According to the present invention, the skin color condition can be optimized according to the type and accuracy of the photographic light source automatically determined from the image data, so that the skin color detection accuracy can be improved. Further, the error of light source detection can be covered by changing the size, center position or shape of the skin color condition area according to the accuracy of the light source type determination. Thereby, accurate face extraction is possible.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of an electronic camera according to an embodiment of the present invention. FIG. 2 is a diagram used for explaining a photometry method for each photographing mode. FIG. 3 is used for face extraction processing in the camera of this example. Fig. 4 is a flowchart showing a skin color table determination sequence. Fig. 5 is a detection frame showing a range of color distribution of light source types. Fig. 6 is a diagram of a light source used for selecting a skin color table. Chart showing accuracy [Fig. 7] Fig. 8 is an example of a flesh color table (A rank) set for each light source type. [Fig. 8] An example of the flesh color table selected at the B rank of daylight white fluorescent lamps. 9 is a diagram showing an example of a skin color table selected when the light bulb (tungsten) is ranked B. FIG. 10 is a flowchart showing a face extraction sequence. FIG. 11 is an example of a skin color extraction area (skin color table) and saturation division. Illustration 12] Description of Figure [code used for explaining the method of obtaining the Figure 13 the aspect ratio of the shape of the face region candidate showing an example of divided by further skin color detected area chroma]
DESCRIPTION OF SYMBOLS 10 ... Camera, 14 ... CPU, 24 ... EEPROM, 42 ... Shooting lens, 44 ... CCD, 70 ... Integration circuit, 72 ... White balance circuit, 74 ... Integration circuit, 76 ... White balance circuit, 78 ... Synchronization circuit, 90 ... Natural white fluorescent light B rank skin color table, 91 ... Natural white fluorescent light A rank skin color table, 92 ... Sunny skin color table, 94 ... Light bulb B rank skin color table, 95 ... Light bulb A rank skin color table

Claims (5)

A method for extracting an area corresponding to a human face from an image,
A light source determination step of determining a light source at the time of shooting based on image data;
An accuracy determination step for determining the accuracy of light source determination by the light source determination step;
A skin color condition setting step for setting a skin color condition for defining a skin color hue range necessary for skin color detection according to the light source information obtained from the light source determination step and the accuracy information obtained from the accuracy determination step;
A skin color region detection step of detecting a region having a skin color hue from within the image in accordance with the skin color condition set in the skin color condition setting step;
A facial region extraction method characterized by comprising: Including a subject brightness detection step of detecting subject brightness,
The face area extraction method according to claim 1, wherein the accuracy determination step determines the accuracy of the light source determination according to the subject luminance detected in the subject luminance detection step. The face area extraction method according to claim 1, wherein the skin color area detecting step includes a saturation division step of further dividing a skin color portion detected based on the skin color condition according to saturation. 4. The face area extraction method according to claim 3, further comprising a face area determination step of determining a face area based on the shape of the skin color area for each saturation divided in the saturation division step. An apparatus for extracting an area corresponding to a human face from an image,
Light source determination means for determining a light source at the time of shooting based on image data;
Accuracy determination means for determining the accuracy of light source determination by the light source determination means;
Skin color condition setting means for setting a skin color condition for determining a skin color hue range necessary for skin color detection according to light source information obtained from the light source determination means and accuracy information obtained from the accuracy determination means;
Skin color area detecting means for detecting an area having a skin color hue from within the image in accordance with the skin color condition set by the skin color condition setting means;
A face area extracting apparatus comprising:

Images