JP2005294917A - Digital camera for extracting face - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a digital camera for properly extracting a face by excluding an object whose hue is similar to that of the face. <P>SOLUTION: The digital camera 10 images a field in which a layout auxiliary frame is set so as to include people, and generates an image signal 102. Further, a system control section 20 detects an image denoting a skin color region from an image signal 126 on the basis of a skin color table and further excludes other skin color regions than that of the layout auxiliary frame from the skin color region image. The skin color region part in the layout auxiliary frame of the skin color region image left in this way is divided by each saturation, and a face object of the people is calculated depending on the shape of each divided region to extract the face region. <P>COPYRIGHT: (C)2006,JPO&NCIPI

Description

  The present invention relates to a digital camera that performs face extraction for extracting an area estimated to correspond to a human face in an image.

  The most noticeable part when viewing portraits is the person's face. For example, when an original image recorded on a negative film or the like is printed on a copy material such as photographic paper, the color of the person's face is It is desirable to extract a region corresponding to a person's face in the original image so as to be printed in an appropriate color, and to determine the exposure amount based on the color and density of the extracted region.

  Therefore, conventionally, in the face area extraction method, an area estimated to correspond to a human face is extracted from the color original image by extracting skin color data. In the face area extraction method described in Patent Document 1, as a result of performing photometry by dividing a large number of locations on the color original image into a plurality of component colors, the face area extraction method is composed of pixels that are located in the skin color area and classified as non-edge portions, and An area surrounded by pixels determined to be edge portions can be extracted as a face candidate area.

JP-A-9-101579

  By the way, a problem in the face extraction of a digital camera is an object having a hue similar to the skin color. For example, a tree or the ground can be considered. In addition, an object under a tungsten light source is not able to achieve a complete white balance (WB) due to the robustness of automatic white balance (AWB), and the light source color remains. For example, a background such as a white wall appears to be a skin color when exposed to tungsten light. When trying to completely correct this situation, the damage at the time of misjudgment becomes large. Therefore, a digital camera is usually set to leave an atmosphere of a tungsten light source.

  In the face region extraction method described in Patent Document 1, when an object having a hue similar to that of the face as described above is photographed, there may be a case where the edge of the face does not become a closed curve due to light intensity or hue, and erroneous detection is detected. There are things to do.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a digital camera that eliminates such drawbacks of the prior art and appropriately extracts a face by excluding an object having a hue similar to that of the face.

  In order to solve the above-described problems, the present invention provides a digital camera including an imaging unit that captures an image of a scene and forms an image signal representing the scene, and a storage unit that temporarily stores the image signal. Including a setting means for setting a composition auxiliary frame indicating the composition to the operator according to the object scene, and a face area extracting means for extracting a face area from an image indicated by the image signal. A skin color table that includes a plurality of color data that can be visually determined as skin color, and the face area extraction unit acquires pre-extraction processing by acquiring the image signal from the storage unit as an extraction processing image; Based on the skin color table, it is determined whether each pixel data in the extraction processing image is a skin color, and a skin color region detecting means for detecting a skin color region image composed of the pixel data determined to be a skin color The skin color area image includes an excluding means for excluding the skin color area outside the composition auxiliary frame, and a dividing means for dividing the skin color area image into a plurality of divided areas. Each of them is characterized by determining whether or not it corresponds to a face and extracting a face region.

  According to the present invention, the digital camera can extract the face by narrowing down the face position by using the composition auxiliary frame. As a result, the influence of skin color similar colors such as the ground or the background at the time of tungsten can be reduced, and photographing with more accurate face extraction can be performed. Furthermore, appropriate white balance adjustment can be performed based on the face extraction result.

  Next, embodiments of a digital camera according to the present invention will be described in detail with reference to the accompanying drawings. Referring to FIG. 1, in the embodiment of the digital camera 10 according to the present invention, an image signal picked up by an image pickup unit 12 is preprocessed by a preprocessing unit 14 and an analog / digital (A / D) converter 16 and stored in a memory 18. Temporarily accumulating, the system control unit 20 processes the image signal using the composition auxiliary frame information to perform face extraction, and the WB adjustment unit 22 controls the white balance adjustment according to the face extraction result. Note that portions not directly related to understanding the present invention are not shown and redundant description is avoided.

  In this embodiment, as shown in FIGS. 6 to 9, the composition auxiliary frame is displayed on the display 40 of the digital camera 10 by changing the type according to the shooting mode, and has the optimum composition according to the object scene. It has the best framing function shown to the operator. The composition auxiliary frame may be switched between display and non-display by the operator.

  In this embodiment, the composition auxiliary frame may be set by selecting one of several types of frames. For example, in the landscape mode or the night view mode, a lattice-shaped frame as shown in FIGS. 10 and 11 is selected. Is done. If the shooting mode is portrait mode, that is, portrait shooting, portrait shooting frames as shown in FIGS. 12 and 13 or vertical shooting as shown in FIGS. A portrait frame of the type is selected. In the present invention, since the human face area is extracted from the image, the portrait frames shown in FIGS. 12 to 15 are used, and the operator applies the object to be photographed in the innermost frame of these frames. Good.

  12 and 14 show two frames that divide an image into three stages, and the composition auxiliary frames in FIGS. 13 and 15 show three frames that divide an image into four stages. 12 to 15, the composition assistance frame may be displayed as one composition assistance frame having a plurality of frames, or a plurality of frames may be switched and displayed.

  Although a specific configuration is not illustrated, the imaging unit 12 includes an imaging surface that forms one screen of a captured image, and an object field image formed on the imaging surface via an optical system is converted into an analog electric signal 102. Output after photoelectric conversion. The imaging unit 12 of the present embodiment may be any image sensor such as a charge coupled device (CCD) or a metal oxide semiconductor (MOS).

  The imaging surface of the imaging unit 12 is formed by arranging a plurality of light receiving elements. Each light receiving element is an optical sensor that photoelectrically converts incident light into an electrical signal. For example, a photodiode is used. In addition, each light receiving element is provided with a color filter that separates the incident light, and by the arrangement of this color filter, incident light that has been color-separated to have attributes of the respective colors, for example, the three primary colors RBG is incident. Then, an analog electrical signal 102 corresponding to the color is output.

  The preprocessing unit 14 has a function of performing analog signal processing on the analog electrical signal 102. For example, the preprocessing unit 14 performs correlated double sampling (CDS) processing to reduce noise, and amplifies the signal by gain control amplifier (Gain Controlled Amplifier: GCA) processing. Further, the processed analog image signal 104 is output to the A / D converter 16.

  The A / D converter 16 has a function of quantizing the signal level of the input analog image signal 104 with a predetermined quantization level and converting it into a digital image signal. In this embodiment, the memory 18 and the WB adjustment unit 22 In addition, digital image signals 106 and 108 are output, respectively. The memory 18 has a function of temporarily storing the digital image signal 106.

  The system control unit 20 is a control function unit that controls and controls the overall operation of the camera. In addition, the system control unit 20 has a skin color table including a plurality of color data that can be visually determined as skin color, and several types of composition auxiliary frames represented by image signals and the like, and can store the composition auxiliary frame settings. it can. In the present embodiment, in particular, an image signal is acquired as an extraction processing image from the memory 18, and face extraction is performed using a composition auxiliary frame. For example, the white balance adjustment of the WB adjustment unit 22 is performed using the face extraction result. Control. Since this face extraction process has an advantageous effect in person shooting, it is preferable to perform this process when, for example, the shooting mode is the person mode, or when the composition assistance frame is for portraits.

  Further, the system control unit 20 in the present embodiment includes a summation unit 30, an AWB adjustment unit 32, and a synchronization processing unit 34 to perform preprocessing for face extraction, and a central processing unit (CPU) The face is extracted including 36, and the WB adjustment unit 22 is controlled according to the extraction result.

  The accumulating unit 30 processes the digital image signal 120 extracted from the memory 18 to generate an evaluation value for AWB adjustment.

  The AWB adjustment unit 32 has a function of performing AWB adjustment on the image signal 122 supplied from the integration unit 30 using the evaluation value obtained by the integration unit 30.

  In the image processing unit 124 for which the AWB adjustment has been performed, the synchronization processing unit 34 surrounds the pixel data obtained from each light receiving element with a gap position without the light receiving element, that is, the pixel data at the virtual light receiving element or the virtual pixel. Interpolation is generated for any of the three primary colors RGB from the pixel data. By this interpolation generation, plain image data of the three primary colors RGB can be obtained simultaneously.

  In this embodiment, the CPU 36 has a function of extracting a face region from the image indicated by the supplied image signal 126, generating a control signal 110 according to the extraction result, and controlling the WB adjustment unit 22. . In the face area extraction, an image showing the skin color area is detected from the image signal 126 based on the skin color table, and the skin color area outside the composition auxiliary frame is excluded from the skin color area image. The skin color area portion of the in-frame skin color area image remaining in this manner is divided for each saturation, and face candidates are calculated from the shapes of the divided areas to extract face areas.

  Further, the CPU 36 generates an evaluation value such as a white balance gain for WB adjustment based on a luminance signal indicating the brightness of the face area, for example, using weighting according to the composition assist frame, and generates WB. A control signal 110 for controlling the adjustment unit 22 is supplied to the WB adjustment unit 22.

  In the present embodiment, the WB adjustment unit 22 is controlled by the control signal 110 supplied from the system control unit 20 with respect to the input digital image signal 108, and appropriately adjusts the white balance so that the digital image signal 112 is output. Further, the WB adjustment unit 22 may read the digital image signal 108 from the memory 18.

  The gamma (γ) conversion unit 24 has a function of performing gamma correction on the input digital image signal 112 in accordance with, for example, the gradation characteristics of the imaging unit 12, and outputs a corrected image signal 114.

  The Y / C processing unit 26 converts the image signal represented by the three primary colors RGB into a signal composed of the luminance signal Y and the color difference signals RY and BY with respect to the input digital image signal 114, Y / C processing is performed to output the image signal.

  The compression / decompression unit 28 has a function of compressing the input digital image signal 116 in accordance with, for example, JPEG (Joint Photographic Cording Experts Group) standard using orthogonal transform and decompressing the original signal. The compression / decompression unit 28 outputs the compressed signal to an information recording medium (not shown) such as a package containing a rotating recording body such as a memory card on which a semiconductor memory is mounted or a magneto-optical disk, and stores it. Good.

  Next, the operation of the digital camera 10 will be described with reference to the flowchart of FIG. The operator operates the camera to set a portrait frame as a composition assistance frame, and an image of the scene including a person or the like is captured by the imaging unit 12.

  The imaged analog electrical signal 102 is supplied to the preprocessing unit 14 and subjected to analog signal processing. The analog image signal 104 subjected to the analog signal processing is supplied to the A / D converter 16 and subjected to analog / digital conversion. The converted digital image signal 106 is temporarily stored in the memory 18 (step 502).

  Next, in the system control unit 20, the image signal 120 is read from the memory 18, and is preprocessed by the integration unit 30, the AWB adjustment unit 32, and the synchronization processing unit 34 (step 504). At this time, the integration unit 30 calculates an evaluation value for AWB adjustment based on the image signal 120, and supplies the image signal 122 and the evaluation value to the AWB adjustment unit 32. In the AWB adjustment unit 32, the image signal 122 is AWB-adjusted using the evaluation value, and the AWB-adjusted image signal 124 is supplied to the synchronization processing unit 34. In the synchronization processing unit 34, the three primary colors RGB are simultaneously generated for the image signal 124, and the generated image signal 126 is supplied to the CPU 36.

  The CPU 36 first obtains a skin color table (step 506). For example, as shown in FIG. 2, when the color that reproduces the object scene image is represented in a diagram with R / G on the horizontal axis and B / G on the vertical axis, the range that can be visually judged as skin color is The skin color extraction area 202 is assumed. In the present embodiment, the skin color table includes a plurality of color data within a range surrounded by the area 202 as a table.

  Here, using this skin color table, a skin color region image is detected from the image signal 126 (step 508). At this time, it is determined whether the image data of each pixel in the image signal 126 corresponds to any of a plurality of color data included in the skin color table, that is, whether it is a skin color. As a result, when the skin color is determined, the image data is left in the image as a skin color region. When the skin data is not determined, the image data is excluded from the image. In this way, the skin color area image is detected.

  Further, the CPU 36 acquires a composition assistance frame (step 510). This composition auxiliary frame may be set by the operator immediately before imaging.

  In this embodiment, as shown in FIG. 3, when the composition auxiliary frame 304 is superimposed on the skin color region image 302, the skin color region located outside the frame is highly likely not to be a face region. It is excluded from the skin color area image as a candidate excluded (step 512).

  Next, the CPU 36 determines the skin color and saturation of each pixel of the image in the composition assistance frame 304, and performs region division based on the saturation (step 514). For example, when the color graphs indicated by R / G and B / G according to FIG. 2 are divided by circles 210, 212, 214, 216 and 218 centered on the origin, each of the division ranges becomes an outer circle. The saturation increases, that is, the saturation increases as R / G or B / G increases. In this embodiment, the skin color table is divided into a predetermined number of saturations. For example, as shown in FIG. 2, when the skin color extraction area 202 is divided into 6 for each saturation, as shown in FIG. The image in 304 is divided into regions for each saturation. In this embodiment, this region division is performed after excluding the skin color region outside the frame, but may be performed after this exclusion.

  In the present invention, when the image in the composition assistance frame is divided, the edge of each pixel may be detected and divided for each boundary of the edge. At this time, the edge may be detected using a high pass filter (HPF) or the like.

  Thus, in order to distinguish between the face and the skin color similar part other than the face, it is determined whether or not each shape corresponds to the face, and the face area is appropriately extracted. (Step 516).

  In this embodiment, as a result of the face extraction, for example, face area candidates 406, 408, 410, and 412 are extracted from the skin color area image 402 using the composition auxiliary frame 404 as shown in FIG. At this time, the system control unit 20 can calculate an evaluation value for white balance adjustment in the WB adjustment unit 22 based on a luminance signal indicating the brightness of these face candidates.

  The brightness of the face candidates 406, 408, 410, and 412 is indicated by Y1, Y2, Y3, and Y4, respectively, and may be obtained from the image data. On the other hand, the weights of the face candidates 406, 408, 410, and 412 are indicated by W1, W2, W3, and W4, respectively, and these are adjusted so as to change inside and outside the composition assisting frame 404. For example, the weights of the face candidates 406, 408, and 410 located in the frame 404 are equal to each other, and the face candidate 412 that is out of the frame 404 is more likely to be a non-face than the others, so it is smaller than the weights of the other face candidates. Thus, the relationship of W1 = W2 = W3> W4 is established.

In this embodiment, the brightness Y _i of the candidate i of the extracted face region, by using the weighting W _i determined according to the composition auxiliary frame, the brightness _{Y = ΣY i W i / ΣW} i of the image 402 Can be calculated. Furthermore, an evaluation value such as a white balance gain can be generated using this brightness Y and supplied to the WB adjustment unit 22 by the control signal 110. For example, a value indicating a magnification such as 1.2 times may be applied to the white balance gain.

  When the composition auxiliary frame has a plurality of frames at the same time, the weight adjustment may be made smaller as the frame is on the outer side. In this embodiment, the weight adjustment is performed for each frame. However, for example, it may be performed according to brightness, saturation, or the level of color data.

  In the WB adjustment unit 22, the image signal 108 is subjected to appropriate white balance adjustment according to the supplied control signal 110. Further, the adjusted image signal 112 is supplied to the γ conversion unit 24, the Y / C processing unit 26, and the compression / decompression unit 28 for signal processing.

It is a block diagram which shows one Example of the digital camera which concerns on this invention. In the digital camera of the Example shown in FIG. 1, about the color which reproduces a scene image, it is a figure which shows the relationship between R / G and B / G, and a skin color extraction area. FIG. 2 is a schematic diagram showing saturation division for a composition auxiliary frame in a captured image in the digital camera of the embodiment shown in FIG. 1. FIG. 2 is a schematic diagram showing a face candidate area for a composition assist frame in a captured image in the digital camera of the embodiment shown in FIG. 1. It is a flowchart explaining the operation | movement procedure of the digital camera which performs face extraction of this invention. It is a figure which shows the auto mode of imaging | photography mode in the digital camera of the Example shown in FIG. FIG. 2 is a diagram illustrating a portrait mode of a photographing mode in the digital camera of the embodiment illustrated in FIG. 1. FIG. 2 is a diagram illustrating a landscape mode of a photographing mode in the digital camera of the embodiment illustrated in FIG. 1. FIG. 2 is a diagram illustrating a night scene mode of a photographing mode in the digital camera of the embodiment illustrated in FIG. 1. FIG. 2 is a schematic diagram of a composition auxiliary frame having a lattice shape in the digital camera of the embodiment shown in FIG. 1. FIG. 2 is a schematic diagram of a composition auxiliary frame having a lattice shape in the digital camera of the embodiment shown in FIG. 1. FIG. 2 is a schematic diagram of a composition auxiliary frame for landscape shooting portrait in the digital camera of the embodiment shown in FIG. 1. FIG. 2 is a schematic diagram of a composition auxiliary frame for landscape shooting portrait in the digital camera of the embodiment shown in FIG. 1. FIG. 2 is a schematic diagram of a composition auxiliary frame for portrait shooting portrait in the digital camera of the embodiment shown in FIG. 1. FIG. 2 is a schematic diagram of a composition auxiliary frame for portrait shooting portrait in the digital camera of the embodiment shown in FIG. 1.

Explanation of symbols

10 Digital camera
12 Imaging unit
14 Pre-processing section
16 A / D converter
18 memory
20 System controller
22 White balance adjustment section
24 Gamma correction section
26 Y / C processing section
28 Compression / decompression unit
30 Integration unit
32 Automatic white balance adjustment section
34 Synchronization processing section
36 Central processing unit

Claims (8)

Imaging means for imaging the scene and forming an image signal representing the scene;
In a digital camera including storage means for temporarily storing the image signal, the camera includes a setting means for setting a composition auxiliary frame indicating a composition corresponding to the object scene to an operator
A face area extracting means for extracting a face area from an image indicated by the image signal,
The face area extracting unit has a skin color table including a plurality of color data that can be visually determined as skin color, and the face area extracting unit includes:
Pre-processing means for acquiring the image signal from the storage means as an extraction processing image and performing pre-extraction processing;
Skin color area detecting means for determining whether each pixel data in the extraction processing image is a skin color based on the skin color table and detecting a skin color area image composed of the pixel data determined to be a skin color;
Excluding means for excluding a skin color area outside the composition auxiliary frame in the skin color area image;
Dividing means for dividing the skin color area image into a plurality of divided areas;
A digital camera which extracts a face area by determining whether or not each of the plurality of divided areas corresponds to a face. The camera according to claim 1, wherein the skin color table divides the plurality of color data into a predetermined number of saturations,
The digital camera according to claim 1, wherein the dividing unit divides the skin color region image for each of the predetermined number of saturations.   3. The camera according to claim 1, wherein the dividing unit includes an edge detecting unit that detects an edge of each pixel of the skin color region image, and divides the skin color region image by a boundary of the edge. Digital camera.   4. The camera according to claim 1, wherein the preprocessing unit calculates a first evaluation value for automatic white balance adjustment, and uses the first evaluation value for the extracted processed image. A digital camera that performs automatic white balance adjustment and further performs synchronization processing. The camera according to any one of claims 1 to 4, wherein the camera calculates a second evaluation value for white balance adjustment based on the face area extracted by the face area extraction means;
A digital camera comprising: white balance adjustment means for performing white balance adjustment on the image signal using a second evaluation value. 6. The camera according to claim 5, wherein the calculating means uses the brightness Y _i of the face area i extracted by the face area extracting means and a weight W _i determined based on the composition assistance frame. A digital camera that calculates brightness Y = ΣY _i W _i / ΣW _i of an extracted processed image, and calculates a second evaluation value using the brightness Y.   7. The digital camera according to claim 6, wherein the second evaluation value is a white balance gain.   8. The digital camera according to claim 1, wherein the composition auxiliary frame has a plurality of frames at the same time.

Images