JP2009123081A - Face detection method and photographing apparatus - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a face area highly precisely and speedily with respect to a face detection method and a photographing apparatus. <P>SOLUTION: After image data to be output by a solid-state image sensor is fetched (step S1) and reduced image data is generated (step S2), pattern matching processing between a face pattern and the reduced image data is performed to extract a face area candidate having correlation of a prescribed value or larger with the face pattern (step S3). When a face area candidate is extracted (YES determination at step S4), a face area which is most likely to be a human face is specified by skin color determination processing from respective extracted face candidate areas (step S5) and, with respect to the respective extracted face candidate areas, pattern matching processing with a face component pattern is executed to detect a face component area from the face candidate area (step S6). On the basis of a relative position relationship among the respective extracted face component areas, it is determined whether a face area specified by the skin color determination processing is a real face area (step S7). <P>COPYRIGHT: (C)2009,JPO&INPIT

Description

  The present invention relates to a face detection method and a photographing apparatus that detect a face area of a person present in a captured image.

  In recent years, digital video cameras, digital still cameras, and other photographic devices have focus processing that focuses on a person's face when a person is the subject, and exposure adjustment processing and white balance correction processing that makes the face appear beautiful. Some of them are equipped with a function for automatically performing an image, and it is possible to easily shoot an image with appropriate focus, color and brightness on the face of a person.

  For example, in a digital still camera, a face area in an image is extracted by analyzing captured image data sequentially taken from a solid-state image sensor before a still image is captured, and the extracted face area is focused and the face is Automatic exposure adjustment and white balance correction for areas.

  It is necessary to detect the face area from the captured image with high accuracy. If a face area is mistakenly detected, focus processing is performed on the face that is not the face, so an image that is not focused on the human face (so-called out-of-focus image) is captured, resulting in poor exposure or white balance. This is because a person's face becomes an unnatural color.

  For this reason, the detection of a face area generally involves a first step of extracting a face area candidate (face candidate area) from an image and determining whether or not the extracted face candidate area is likely to be a human face. And the second step. For example, the first step is performed by the pattern matching method, and the second step is performed by the skin color detection method.

  Pattern matching method means that various face patterns such as human face pattern, side view pattern, adult face pattern, infant face pattern, etc. are stored as templates in the memory of the imaging device, In this method, the correlation between these templates and each partial area in the captured image is obtained, and a partial area having a high correlation is extracted as a face candidate area.

  The skin color detection method divides a captured image of one screen into, for example, 8 × 8 = 64 areas, and each divided area based on accumulated data of R (red), G (green), and B (blue) for each divided area. Is obtained, and it is determined that a face is present in a divided area where the hue is a skin color.

In relation to the face detection technique described above, a method for detecting facial parts such as eyes, nose and mouth attached to a person's face from the extracted face candidate regions is known (for example, see Patent Document 1). ). Thus, by detecting a face part from the face candidate area, it is possible to more reliably prevent erroneous detection of an area other than the face as a face area, and an improvement in detection accuracy is expected. Further, the size of the face area can be determined more accurately by detecting the face part.
JP 2006-228061 A

  However, in order to improve the detection accuracy of face detection, if face component detection processing is performed in addition to the above processing, the calculation processing becomes longer, and therefore one frame period (for example, 1/30 second) of a moving image There arises a time problem such as whether or not the face detection process ends. If the face detection process is completed by the next frame and the face area is not specified, naturally, the process (focus process, exposure adjustment process, white balance correction process, etc.) for the face area cannot be executed. If the face detection process does not end within one frame period, the face detection process must be performed over a period of two frames or more, and the tracking performance of the face area is degraded.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a face detection method and a photographing apparatus that can detect a face area from one frame image at high speed and with high accuracy.

  In order to achieve the above object, a face detection method according to the present invention is a face detection method for detecting a human face area present in a captured image output from a solid-state image sensor. From the face candidate area detection step that executes a pattern matching process and extracts a face area candidate having a correlation greater than a predetermined value with the face pattern from the captured image, and from each face candidate area extracted in the face candidate area detection step A narrowing process step for identifying a face area that is likely to be a human face, and a pattern matching process with a face part pattern are performed on each face candidate area extracted in the face candidate area detection step, and a face is identified from the face candidate area. Based on the relative positional relationship between each face part area detected in the face part area detecting step and the face part area detecting step for detecting the part area, the aperture Identified face region viewed processing step is characterized in that and a determination step of determining whether a true face region.

  The narrowing-down processing step preferably examines the hue of the face candidate area extracted in the face candidate area detection step and specifies a probable face area as a human face depending on whether or not the hue is a skin color.

  Further, an imaging device of the present invention performs a pattern matching process between a face pattern and the captured image in an imaging device that detects a human face area present in a captured image output from a solid-state image sensor, Face candidate area detecting means for extracting face area candidates having a correlation greater than a predetermined value with the face pattern from the captured image, and face areas that are likely to be human faces from each face candidate area extracted by the face candidate area detecting means And a face part area for detecting a face part area from the face candidate area by executing a pattern matching process with the face part pattern for each face candidate area extracted by the face candidate area detecting means. Whether the face area specified by the narrowing-down processing means is a true face area based on the relative positional relationship between the detection means and each face part area detected by the face part area detection means Determining means for determining, characterized by comprising a.

  It is preferable that the narrowing-down processing unit examines the hue of each face candidate region extracted by the face candidate region detection unit and specifies a probable facial region as a human face depending on whether the hue is a skin color.

  According to the present invention, since the face part area detection process is executed in parallel with the narrowing-down process, the face area can be detected at high speed and with high accuracy.

  In FIG. 1, a photographing apparatus 10 according to an embodiment of the present invention includes a photographing lens 11 that takes in incident light from an object field and forms an image, and a CCD image in which a light receiving surface is disposed on the image forming surface of the incident light. A solid-state imaging device 12 such as a sensor or a CMOS image sensor, and an analog front-end processor (AFE) 13 that performs correlated double sampling processing, gain adjustment processing, and analog / digital conversion processing on captured image data output from the solid-state imaging device 12 Is provided.

  Further, the photographing apparatus 10 includes a signal processing unit 14 and a face detection processing unit 15 that capture captured image data output from the AFE 13 in parallel, and an image data storage memory 16 connected to the signal processing unit 14, and includes face detection. The face information detected by the processing unit 15 is output to the signal processing unit 14.

  The signal processing unit 14 performs offset processing, gamma correction processing, RGB / YC conversion processing, and the like on the captured image data, and further outputs the image data compressed to a JPEG image to a recording medium (not shown). It is stored or displayed as a through image on a display unit (not shown) mounted on the rear surface of the photographing apparatus.

  The signal processing unit 14 includes a main control circuit (CPU) 17 that controls each unit, and performs exposure adjustment and white balance correction based on face information (face area coordinate data) sent from the face detection processing unit 15. Etc. to instruct the taking lens 11 to control the focus position.

  The face detection processing unit 15 includes a pre-processing unit 18 that generates reduced image data by thinning out the captured image data captured from the AFE 13, a reduced image data storage memory 19 that stores the reduced image data, and a prepared in advance. A template data storage memory 20 for storing template data of various face patterns and face part patterns, a face area detection unit 21 for detecting a face area that looks like a human face from the reduced image data, and a face part from the face area. A face part detection unit 22 to detect, and a determination unit 23 that determines a true face area from the detection results of the face area and the face part, and the detected true face area coordinate data as face information to the signal processing unit 14 Notice.

  As shown in FIG. 2, the face area detection unit 21 includes a pattern matching unit 21a that performs pattern matching processing using reduced image data and face pattern template data, and a face candidate area detected by the pattern matching unit 21a. It is comprised from the skin color determination part 21b which determines whether a hue is a skin color.

  The pattern matching unit 21a uses the reduced image data stored in the reduced image data storage memory 19 as image data (gray image data) represented by a luminance value, and detects an area where the correlation value with the template data is greater than a predetermined threshold value. Extract as a candidate area. Specifically, the template (face pattern) is shifted by one pixel in each image data, and the correlation value r between the template and the partial image overlapping it at each position is calculated based on the following equation (1).

In the expression (1), the pixel value of the template is represented by g, the pixel value of the partial image overlapping the template is represented by f, and the sum of the pixel values in those regions is represented by (Σg), (Σf), each pixel The sum of squares of the values is (Σg ² ), (Σf ² ), the sum of the products of the pixel values of the corresponding pixels of the template T and the corresponding partial image is (Σfg), and the number of pixels of the template is n. The correlation value r takes a value in a range of 0 ≦ r ≦ 1, and shows a value closer to 1 in a region having a higher degree of matching with the template.

  The skin color determination unit 21b divides the reduced image data stored in the reduced image data storage memory 19 into one-screen captured image data into, for example, 8 × 8 = 64 divided regions, and R, G, and R for each divided region. By calculating the integrated value of each of the B color signals, the hue of each divided area is obtained, and the face candidate area is determined by determining whether the hue of the face candidate area extracted by the pattern matching unit 21a is a skin color. The face candidate area that is likely to be a human face is specified as the face area, and the result is input to the determination unit 23.

  The facial part detection unit 22 uses the template data of facial part patterns such as eyes, nose and mouth stored in the template data storage memory 20, and when the processing of the pattern matching unit 21a of the facial region detection unit 21 is completed. Pattern matching is performed on each obtained face candidate area, a face part area is detected from each face candidate area, and the result is input to the determination unit 23. That is, the face part region detection process by the face part detection unit 22 is performed in parallel with the determination process by the skin color determination unit 21b. Note that the pattern matching by the face part detection unit 22 is performed by the same method as described above based on the above formula (1).

  The determination unit 23 determines whether or not a face component region is detected in the face region input from the skin color determination unit 21b from the detection information of the face component region input from the face component detection unit 22, and the face component region Is detected, it is determined whether or not the face area is a true face area based on the relative positional relationship between the face part areas (such as the relative position of the nose and mouth with respect to the positions of both eyes).

  Next, the procedure of the face detection process performed by the face detection processing unit 15 will be described with reference to the flowchart shown in FIG. From the solid-state imaging device 12, captured image data is sequentially output in the order of a first frame image, a second frame image, a third frame image,.

  In step S1, the face detection processing unit 15 first captures the captured image data of the first frame from the AFE 13, and in step S2, the preprocessing unit 18 generates reduced image data. The reduction ratio of the reduced image data is set to a value that allows the captured image data composed of several million pixels to be stored in the reduced image data storage memory 19 having a relatively small capacity.

  In the next step S3 (face candidate region detection step), the pattern matching unit 21a of the face region detection unit 21 extracts a face pattern template from the template data storage memory 20 and performs pattern matching processing. In this pattern matching processing, as shown in FIG. 4, the template 31 calculated by the above equation (1) and the partial image in the reduced image data 30 are scanned while the face pattern template 31 is scanned on the reduced image data 30. It is sequentially determined whether or not the correlation value r is greater than or equal to a predetermined threshold value, a partial image having a correlation value r greater than or equal to the threshold value is defined as a face candidate area, and the coordinates of the face candidate area on the reduced image data 30 Ask for.

  When the scanning of the template 31 on the reduced image data 30 is completed, a template 32 obtained by enlarging (or reducing) the template 31 by a predetermined magnification is generated, and the enlarged template 32 is scanned on the reduced image data 30. The pattern matching process is performed again.

  When there are a plurality of face pattern templates prepared in advance, the pattern matching process described above is repeated using all face pattern templates, and coordinate data indicating the obtained face candidate areas is stored.

  In the example shown in FIG. 4, the face candidate area is extracted in the area surrounded by the rectangular frames 33a and 33b. The face candidate area 33b is a face image area of a true person, but the face candidate area 33a is an erroneously extracted tree pattern as a face image.

  In the next step S4, it is determined whether or not a face candidate area has been detected in step S3. If no face candidate area has been detected, the process proceeds to step S8 to be described later, indicating that no face candidate area has been detected. Is notified to the signal processing unit 14.

  In the example shown in FIG. 4, since the face candidate areas 33a and 33b exist, it is determined in step S4 that the face candidate area has been detected. In this case, the process proceeds to step S5 and step S6. Steps S5 and S6 are performed in parallel based on the information (coordinate data) of the detected face candidate regions 33a and 33b.

  In step S5 (squeezing process step), the skin color determination unit 21b performs a skin color detection process. The skin color determination unit 21b calculates hue data from each divided area of the reduced image data 30 (for example, 8 × 8 = 64 divided areas), examines the hue of the face area candidates 33a and 33b, and the face whose hue is skin color The candidate area is specified as the face area. In the example shown in FIG. 4, the face area candidate 33a is an image of a tree and is green, so it is not specified as a face area, and the face area candidate 33b is a face image of a person and has a skin color. , Identified as a face region.

  In step S6 (facial part region detection step), the facial part detection unit 22 extracts a facial part pattern template from the template data storage memory 20 and performs pattern matching processing. In this pattern matching process, as shown in FIG. 5, while scanning the face part pattern template 34 on the face candidate areas 33a and 33b, the template 34 calculated by the above equation (1) and the face candidate areas 33a, 33b, It is sequentially determined whether or not the correlation value r with the partial image in 33b is equal to or greater than a predetermined threshold value, and a partial image having the correlation value r equal to or greater than the threshold value is set as a facial part region, Find the coordinates above.

  When the scanning of the template 34 on the face candidate areas 33a and 33b is completed, a template (not shown) obtained by enlarging (or reducing) the template 31 by a predetermined magnification is generated, and the enlarged template is used as the face candidate area. The pattern matching process is performed again by scanning on 33a and 33b.

  When there are a plurality of face part pattern templates prepared in advance, the above pattern matching process is repeated using all face part pattern templates, and coordinate data indicating the obtained face part regions is stored.

  In the example shown in FIG. 5, the face part area is not detected from the face candidate area 33a, and the face part area is detected from the face candidate area 33b. Rectangular frames 35 to 38 illustrated in FIG. 6 indicate face component areas detected from the face candidate area 33b. The face part areas 35 and 36 are eye image areas, the face part area 37 is a nose image area, and the face part area 38 is a mouth image area.

  In step S7 (determination step), the determination unit 23 determines whether or not a facial part is detected in the face region (in the example shown in FIG. 4, the face region candidate 33b) input from the skin color determination unit 21b. When a face part is detected, the relative position relationship of the face part region (in the example shown in FIG. 6, the face part regions 35 to 38) (relative to the distance between the eyes and the distance to the nose, etc.) It is determined whether or not the face area is a true face area.

  In the next step S8, the determination unit 23 notifies the detected coordinate data of the true face area to the signal processing unit 14 as face information. If no true face area is detected, the signal processing unit 14 is notified of information indicating that no face area has been detected.

  The face detection processing unit 15 repeatedly executes the above steps S1 to 8 for each frame period. The signal processing unit 14 performs focus processing, exposure amount adjustment, white balance correction processing, and the like based on the face information notified from the face detection processing unit 15 every frame period. As a result, an image in which the focus, color, and brightness are optimized on the face of a person can be easily taken.

  As described above, according to the present embodiment, the face by the face component detection unit 22 is performed in parallel with the skin color determination process of the second step performed after the face region detection unit 21 extracts the face candidate region as the first step. Since the component area detection process is executed, the face area can be detected from one frame image at high speed and with high accuracy.

  In the above embodiment, the face detection is repeatedly executed every frame period. However, after face detection is performed in the first frame period, the detected face area is used as a template in the subsequent frame periods. It is also preferable that the face area is tracked by performing pattern matching. In the present invention, since the face area is specified by detecting the face parts, the position and size of the face area used as the template can be determined accurately, and the background other than the face is not included in the template, and the tracking performance Will improve.

  In the above embodiment, after the face candidate area is detected, the likelihood of the face candidate area is narrowed down by the skin color detection process. However, other narrowing processes may be used instead of the skin color detection process.

  In the above embodiment, one face candidate area is detected for the face of the same person. However, depending on the type of face pattern and pattern matching method used, the face candidate area for the face of the same person is used. May be detected in duplicate (where the position or size is slightly shifted). In this case, after detecting the face candidate area, it is necessary to perform a process (duplicate detection determination process) to combine a plurality of face candidate areas detected in duplicate as one face area. At the time of processing), the face part region detection processing by the face part detection unit 22 may be performed in parallel.

  It is also preferable to apply the imaging device of the above-described embodiment to a monitoring camera used for a door phone system or the like. A device that saves an image only when a person approaches, such as a door phone system, accurately captures the necessary subject (person's face) by reducing face mis-extraction and improving the human face detection rate. It becomes possible.

It is a functional block diagram which shows the structure of the imaging device which concerns on one Embodiment of this invention. It is a functional block diagram which shows the structure of a face area | region detection part. It is a flowchart which shows the process sequence which a face detection process part performs. It is explanatory drawing of the pattern matching process which used the face pattern as the template. It is explanatory drawing of the pattern matching process which used the face component pattern as the template. It is the figure which illustrated the face part area | region detected from the face candidate area | region.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Image pick-up device 12 Solid-state image sensor 14 Signal processing part 15 Face detection process part 16 Image data storage memory 18 Pre-processing part 19 Reduced image data storage memory 20 Template data storage memory 21 Face area detection part 21a Pattern matching part 21b Skin color determination part 22 Face part detection unit 23 determination unit

Claims (4)

In a face detection method for detecting a face area of a person present in a captured image output from a solid-state image sensor,
A face candidate area detection step of executing a pattern matching process between the face pattern and the captured image, and extracting a face area candidate having a correlation of a predetermined value or more with the face pattern from the captured image;
A narrowing processing step for identifying a face area that is likely to be a human face from each face candidate area extracted in the face candidate area detection step;
For each face candidate area extracted in the face candidate area detection step, perform a pattern matching process with a face part pattern, and detect a face part area from the face candidate area,
A determination step of determining whether or not the face region specified in the narrowing-down processing step is a true face region based on the relative positional relationship of each face component region detected in the face component region detection step;
A face detection method comprising:   In the narrowing-down process step, the hue of the face candidate area extracted in the face candidate area detection step is examined, and a probable face area as a person's face is specified based on whether or not the hue is a skin color. The face detection method according to claim 1. In an imaging device that detects a human face area present in a captured image output from a solid-state imaging device,
A face candidate area detecting means for executing a pattern matching process between a face pattern and the captured image, and extracting a face area candidate having a correlation of a predetermined value or more with the face pattern from the captured image;
Narrowing processing means for identifying a face area that is likely to be a human face from each face candidate area extracted by the face candidate area detection means;
For each face candidate area extracted by the face candidate area detection means, a face part area detection means for executing a pattern matching process with a face part pattern and detecting a face part area from the face candidate area;
Determining means for determining whether the face area specified by the narrowing-down processing means is a true face area based on the relative positional relationship of each face part area detected by the face part area detecting means;
An imaging apparatus comprising:   The narrowing-down processing means examines the hue of each face candidate area extracted by the face candidate area detection means, and identifies a probable face area as a human face depending on whether or not the hue is a skin color. The imaging device according to claim 3.

Images