JP2007257322A - Face portion detecting method and its device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a face portion detecting method and its device capable of improving detection accuracy of ears by positively determining existence or nonexistence of the ears from profile lines on both widthwise sides of a face of a person to be detected. <P>SOLUTION: The face of the person to be detected is shot from the front to detect a profile of the face of the person to be detected from the shot image. Then a portion wherein the profile of the face becomes discontinued in a width direction is detected, and an area surrounded by the detected discontinuous points is considered as an ear area. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a face part detection method and apparatus capable of detecting the face direction from the face contour of a person to be detected.

  2. Description of the Related Art Conventionally, in a vehicle such as a vehicle, a driver's (detected person) 's face is imaged by a TV camera (imaging means), and the captured face image is image-processed to obtain facial eyes, ears, noses, and the like. 2. Description of the Related Art There is known a face feature amount detection apparatus that detects a face portion as a feature amount of a driver's face by obtaining the face portion. With this feature detection apparatus, it is possible to grasp the driver state such as the face orientation and arousal based on the detected face part.

As such a face feature amount detection apparatus, there is also known one that performs image processing using the outermost portion in the width direction of a face image captured by a TV camera as an ear (for example, see Patent Document 1). ).
JP 2004-78749 A

  By the way, in such a facial feature amount detection device, when the driver is facing the front of the TV camera, the left and right ears of the face image can be detected. It is conceivable to detect the driver's face direction from such an ear position and grasp the driver state.

  However, when the driver's face is directed laterally with respect to the TV camera, it is conceivable that a false detection is performed on one or both ears.

  Therefore, the present invention provides a face part detection method and apparatus capable of improving the detection accuracy of the ear by surely determining the presence or absence of the ear from the contour lines on both sides of the face of the detection subject. It is for the purpose.

  In order to achieve this object, the present invention provides a face part detection method and apparatus for detecting a face part from a certain region on an image, and images the face of the person to be detected from the front, After detecting the contour of the face of the person to be detected, a portion where the contour of the face is discontinuous in the width direction is detected, and an area surrounded by the detected discontinuous point is used as an ear region And

  According to these inventions, the presence / absence of an ear can be reliably determined from the contour lines on both sides in the width direction of the face of the detected person, and the detection accuracy of the ear can be improved.

Hereinafter, embodiments of the present invention will be described with reference to the drawings.
<First Embodiment>
[System block diagram]
FIG. 1 is a block diagram showing a configuration of a first embodiment of a face part detection apparatus to which the present invention is applied. As shown in the figure, the face part detection device includes a face image capturing unit CL1 that captures an image of a face of a detection target person (for example, a driver of a vehicle) and outputs face image data. It has face part detection means CL2 of the person to be detected imaged by means CL1.

  The face image capturing means CL1 continuously captures the face of the person to be detected in time series and outputs a video signal of a moving image.

  The face part detection means CL2 is detected by the face area detection means CL3 that processes the face image data output from the face image imaging means CL1 and detects the area where the face has moved in the image, and the face area detection means CL3. Contour detecting means CL4 for detecting the contour of the driver's face, and discontinuous point detecting means CL5 for detecting a location where the contour of the detection subject's face detected by the contour detecting means CL4 is discontinuous in the width direction; The ear region setting means CL6 for setting the ear region from the discontinuous points detected by the discontinuous point detecting means CL5 is provided.

  The contour detecting means CL4 includes a face width center line defining means CL41 and a density change point detecting means CL42.

  The face width center line defining means CL41 includes a left and right symmetric face part detecting means CL411 for detecting a face part that is present at a substantially symmetrical position on the left and right sides of the face such as the eye or eyebrow from the face image picked up by the face image pickup means CL1. ing. The face width center line defining means CL41 is a vertical line segment between face parts that connects face parts that exist at substantially symmetrical positions on the left and right sides of the face such as the eyes or eyebrows detected by the left and right symmetrical face part detection means CL411. The bisector is defined as the face width center line.

  The density change point detecting means CL42 changes the density value from the white direction to the black direction by looking at the change in the image density from the face width center line defined by the face width center line defining means CL41 outward in the face width direction. Find a spot. In addition, the contour detection means CL4 sequentially finds density change points by the density change point detection means CL42 in the face height direction, and sets a set of continuous density change points as the face contour.

  Further, the discontinuous point detecting means CL5 includes a contour cutting point detecting means CL51 for detecting a cutting point where the contour line is broken when the contour line is traced in one direction in the height direction of the face, and a contour cutting point detecting means CL51. Other contour detection means CL52 for detecting the presence or absence of another contour extending in the height direction of the face at a position separated in the face width direction around the detected contour cut point. Moreover, when the contour line is detected by the other contour detection unit CL52, the discontinuous point detection unit CL5 sets the contour cutting point detected by the contour cutting point detection unit CL51 as a discontinuous point.

  The ear region setting unit CL6 sets the region surrounded by the discontinuous points detected by the discontinuous point detection unit CL5 as the ear region, and sets the center of gravity of the ear region as the position of the ear of the face.

This device can be used to detect the contour of a driver's face such as an automobile or a railway vehicle, but will be described in the case where it is applied to a driver of an automobile in all the embodiments described below. .
[Device layout]
FIG. 2 is a layout diagram of the device of the present invention. The camera 1 serving as the face image capturing means CL1 has a face Df of a driver D, who is a person to be detected (a person to be detected), on the instrument of the car. It is installed at a position where it can be imaged from the side) and photographs the driver's face Df.

  The camera 1 is installed so as to be able to capture an image from a substantially lower front side of the person to be detected, so that the left and right movements of the face of the person to be detected are equally captured in both the right and left, and the state of the structure of the face Is to accurately capture

  As the camera 1, a camera that can capture a moving image including an image sensor such as a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) is used. In this embodiment, a CCD having, for example, 640 pixels in the horizontal direction (X-axis direction) and 480 pixels in the vertical direction (Y-axis direction) is used as the camera 1. The camera 1 captures an input image to the microcomputer 2. The input image captured by the camera 1 is input as image data to the microcomputer 2 installed inside the vehicle body, such as the back side of the instrument.

The microcomputer 2 is programmed with the face part detecting means CL2 shown in FIG. The microcomputer 2 executes the following processing according to the programming of the facial part detection means CL2.
[System-wide processing]
Next, the processing status of the system will be described. FIG. 3 is a flowchart showing the overall processing flow of the system. First, when the key cylinder is operated from the ignition key (IG) to the start position by an ignition key (not shown) and the engine is operated, the processing shown in FIG. 3 is started. When this process is started, an image including the face Df of the driver D who is the detection target person is detected by the camera 1 in the face image capturing process step S1 (hereinafter, “step S” is simply referred to as “S”). An image is taken and input as image data to the microcomputer 2, and the process proceeds to S2.

  In the face area detection process of S2, the face area is detected from the image captured in the imaging process S1. Note that a known method can be used for the face area detection process, and a detailed description thereof will be omitted.

  In S3, it is determined whether or not the face area has been detected based on the detection result of the face area detection process in S2. If the face area can be detected in S2, YES is determined in S3, and the process proceeds to the face width centerline defining process in S4. If the face area cannot be detected in S2, NO in S3, the process returns to the face image capturing process in S1, and the face area detection process in S2 is repeated until the face area can be detected.

  In the face width center line defining process in S4, the face width center line L of the face area detected in the face area detecting process in S2 is obtained as shown in FIGS. 4 to 7, and the image density change point detecting process in S5 is performed. Transition. 4 and 6 show the state of facing the front, and FIGS. 5 and 7 show the state of looking at the left side. 4 and 6, the left and right ears are visible, and in FIGS. 5 and 7, only the right ear is visible.

  By the way, as a method for obtaining the face width center line L in S4, for example, there is the following method.

  First, as shown in FIG. 4 and FIG. 5, as the first method, it can be determined that the face image 3 detected in S <b> 2 is positioned substantially at the center of the image 4 captured by the camera 1. In the case where the face image 3 detected in S2 is substantially in the entire image 4 captured by the camera 1, the face width center line defining means CL41 sets the center of the image 4 in the left-right direction to the face width of the face image 3. There is a method of using the center line. This face width center line setting method is based on the idea that the driver (driver) may fix the face width center line because the face position hardly moves while driving.

  As a second method, as shown in FIGS. 5 and 6, the left and right eyes 5L and 5R of the driver are detected, and the left and right eyes 5L pass through the center O between the left and right eyes 5L and 5R. , 5R, a line perpendicular to the line 6 connecting 5Rc and 5Rc is used as the face width center line L by the face width center line defining means CL41. As a method for obtaining the eye position, a conventionally known image processing method can be used, and thus detailed description thereof is omitted.

  In the image density change point detection process of S5, the density change point detection means CL42 detects the point where the density value changes from the white direction to the black direction by looking at the change in the image density from the face width center line L to the outside in the face width direction. Then, the process proceeds to the cutting point detection process of S6. For example, when looking at the amount of light reflected from the face on the horizontal line 7 slightly below the eyes 5L and 5R shown in FIG. 8A, the amount of reflected light changes as shown in FIG. 8B. In FIG. 8B, the range of x1 to x2 indicates the change in brightness of the right hair part, the range of x2 to x3 indicates the change of brightness of the face part, and x3 to x4 indicate the change in brightness on the left side. Shows changes in hair brightness.

  Accordingly, density change points x2 and x3 where the density value changes from the white direction to the black direction are detected. Such detection is performed at predetermined intervals over the entire vertical direction, and the density change points x2 and x3 are sequentially detected in the height direction of the face as shown in FIG. 9, and the detected height direction is detected. The set of density change points x2 and x3 is connected in the vertical direction to form a face outline, and the process proceeds to the face outline cut point detection process in S6. In FIG. 10, the left face contours FL1, FL2, FL3 and the right face contours FR1, FR2, FR3 are detected. Further, in FIG. 11, the left face outline FL1 and the right face outline FR1, FR2, FR3 are detected. It should be noted that the density change points Ep, Np, Mp in the eyes, nose, mouth, etc. shown in FIG. 9 are not gathered so as to be lined up in the height direction (vertical direction). It is determined that it is not x2 or x3 (noise).

  In the cutting point detection process of S6, the cutting point where the contour line is broken when the contour line is traced in one direction in the height direction of the face is detected by the contour cutting point detection means CL51, and the process proceeds to the other contour detection process of S7. To do. In FIG. 12, cutting points Lc11, LC12, Lc21, LC22, Lc31, Lc32 of the contour lines FL1, FL2, FL3, and cutting points Rc11, Rc12, Rc21, Rc22, Rc31, Rc32 of the contour lines FR1, FR2, FR3 are detected. Further, in FIG. 13, the cut points Lc11 and Lc12 of the left contour line FL1 are detected, and the cut points Rc11, Rc12, Rc21, Rc22, Rc31 and Rc32 of the right face contour lines FR1, FR2 and FR3 are detected.

  In the other contour detection process in S7, the contour lines FL1, FL2, FL3 and the contour lines FR1, FR2, FR3 are detected as other contours by the other contour line detection means CL52, and the discontinuous point detection process in S8 is performed. Transition.

  An example of this other contour detection process will be described with reference to contour lines FR1, FR2, FR3 on the left side of the face as shown in FIG. 14, for example. In FIG. 14, an exploration area 8 having, for example, 30 to 50 pixels on the left and right and 5 to 10 pixels on the upper and lower sides is set around the cutting point Rc12 of the outline FR1. Then, the other contour detection means CL52 determines whether or not there are contour lines FR1 and FR2 deviated in the horizontal direction (left and right) in the exploration area 8, and if there are deviated contour lines FR1 and FR2, The line FR2 is detected as another outline with respect to the outline FR1.

  As shown in FIG. 15, the search area 8 is set at the cutting points Rc11, Rc21, Rc22, Rc31, Rc32, respectively, and detects that the contour lines FR1, FR2, FR3 are other contours. Similarly, the contour lines FL1, FL2, and FL3 can be detected as other contours.

  In the discontinuous point detection process of S8, the contour lines FR1, FR2, FR3 detected in S7 are other contour lines, and therefore, the cutting points Rc12, Rc21, Rc22, Rc31 are discontinuous points of the contour lines FR1, FR2, FR3. Detect as. That is, the above-described search area 8 is also used for detecting the discontinuous points. For example, in FIG. 14, when there are contour lines FR1 and FR2 that are separated in the horizontal direction (left and right) in the exploration area 8, the contour lines FR1 and FR2 are not continuous. It is detected as a discontinuous point with respect to the line FR2. The cutting points Lc12, Lc21, Lc22, Lc31 and the cutting points Rc21, Rc22, Rc31 can be detected as discontinuous points in the same manner. As a result, the cutting points Rc11 and Rc32 in FIGS. 12 to 15 can be detected as not being discontinuous points, and the cutting points Lc11 and Lc12 in FIG. 13 can be detected as having no discontinuous points.

  Similarly, in S8, the contour lines FL1, FL2, FL3 and the contour lines FR1, FR2, FR3 are other contour lines in S7, so that the discontinuous point detecting means CL5 detects the cutting points Lc12, Lc21, Lc22 detected in S6. , Lc31 are detected as discontinuous points of the contour lines FL1, FL2, FL3, and the process proceeds to the ear region detection process of S9. As a result, the cut points Lc11 and Lc32 in FIG. 12 can be detected as not being discontinuous points, and the cut points Lc11 and Lc12 in FIG. 13 can be detected as not being discontinuous points.

  In the ear region detection process of S9, the ear region setting means CL6 sets the region surrounded by the discontinuous points as the ear region and proceeds to the ear position setting process of S10. That is, in FIG. 12, the region surrounded by the cut points Lc12, Lc21, Lc22, and Lc31 that are discontinuous points is the left ear region EL, and is surrounded by the cut points Rc12, Rc21, Lc22, and Rc31 that are discontinuous points. This region is defined as the right ear region ER (see FIG. 16). In FIG. 13, a region surrounded by the cutting points Rc12, Rc21, Rc22, and Rc31, which are discontinuous points, is a right ear region ER.

  In the ear position setting process in step S10, the ear region setting means CL6 sets the center of gravity position Lg of the ear region EL (see FIG. 12) as the left ear position, and the ear region setting means CL6 sets the center of gravity position Rg of the ear region ER to the right. Shift to step S11.

  In S11 (see FIGS. 12 and 13), it is determined whether or not the ears are detected on the left and right of the captured driver's face image (whether there are two), and the ears are detected on the left and right. If YES, the process returns to S1 and the contour detection process is repeated. If no ear is detected on either the left or right, the process proceeds to S12.

  In S12, it is assumed that the face is greatly turned in the horizontal direction, the driver is informed of driving advice, and the process proceeds to Step S13.

  In this step S13, it is determined whether or not the engine is stopped by turning the ignition key from the IG position (ignition position) to the ACC position (accessory position). If not, the process returns to S1 to perform contour detection processing. Repeat and stop if stopped.

  9, 10, and 12 show the state of facing the front, and FIGS. 11 and 13 show the state of looking at the left side. 9, 10, and 12, the left and right ears are visible, and in FIGS. 11 and 13, only the right ear is visible.

Moreover, although the example mentioned above demonstrated the case where the left-right direction center of the image 4 was made into the face width centerline of the face image 3, it is not necessarily limited to this. For example, if the camera 1 is installed in the vicinity of the center console and the camera 1 captures the interior of the vehicle with a wide-angle lens and images the situation inside the vehicle as well as the driver, the face width is set at an offset location in the image. A center line may be set.
<Second Embodiment>
In the second embodiment, two adjacent image frames are extracted from a moving image continuously output in time series from the face image capturing means CL1, and a difference image is extracted from the two extracted image frames. The face contour is generated based on the pixel density of the difference image.
[System block diagram]
FIG. 17 is a block diagram showing the configuration of the second embodiment of the face part detecting apparatus to which the present invention is applied. As shown in the figure, the face part detection device includes a face image capturing unit CL1 that captures an image of a face of a detection target person (for example, a driver of a vehicle) and outputs face image data. It has face part detection means CL2a of the person to be detected imaged by means CL1.

  The face image capturing means CL1 continuously captures the face of the person to be detected in time series, outputs a video signal of the moving image, and inputs the video signal of the moving image to the face part detecting means CL2a.

  The face part detecting means CL2a is detected by the face area detecting means CL3 for processing the face image data output from the face image capturing means CL1 and detecting the area where the face has moved in the image, and the face area detecting means CL3. Contour detecting means CL4a for detecting the contour of the driver's face, and discontinuous point detecting means CL5 for detecting a location where the contour of the face of the person to be detected detected by the contour detecting means CL4a is discontinuous in the width direction. I have.

  The contour detection unit CL4a includes a difference image generation unit CL7 that generates (detects) a contour line of the driver's face imaged by the face image imaging unit CL1, and a difference image generated by the difference image generation unit CL7. Binarized image processing means CL8 for generating a binarized image from the image and vertical contour line detecting means CL9 for detecting a line extending in the vertical direction from the binarized image.

The difference image generation means CL7 extracts two adjacent image frames from the moving image continuously output in time series from the face image imaging means CL1, and generates a difference image from the extracted two image frames. Then, the pixel density of the difference image is stored in a memory or the like (not shown). The generation of the difference image is sequentially executed in time series, and a plurality of the difference images generated in sequence are stored in a frame memory (not shown) while being sequentially updated in time series.
[Device layout]
Since the arrangement of the devices is the same as in the first embodiment, illustration is omitted. In the embodiment of the present invention, the face part detecting means CL2a shown in FIG. 17 is programmed in the microcomputer 2 shown in FIG. The microcomputer 2 executes the following processing according to the programming of the facial part detection means CL2a.
[System-wide processing]
Next, the processing status of the system will be described. FIG. 18 is a flowchart showing the overall processing flow of the system. First, when the key cylinder is operated from the ignition key (IG) to the start position by an ignition key (not shown) and the engine is operated, the processing shown in FIG. 3 is started. When this processing is started, the face image of the detection target person is picked up by the camera 1 of FIG. 2 in the face image pickup processing step S20 (hereinafter, “step S” is simply expressed as “S”), and the microcomputer 2 is input as image data, and the process proceeds to S21.

  In the face area detection process of S21, the face area is detected from the image (for example, change in contrast or movement in the image) captured in the face image capturing process S20, and the process proceeds to S22.

  In S22, it is determined whether the face area has been detected based on the detection result of the face area detection process in S21. If the face area can be detected in S21, YES in S22, and the process proceeds to the differential image processing in S23. If the face area cannot be detected in S21, NO in S22, the process returns to the face image capturing process in S20, and the face area detection process in S21 is repeated until the face area can be detected.

  In the differential image processing of S23, two adjacent image frames are extracted from the moving image continuously output in time series from the camera 1 of FIG. 2, and the differential image generation means is extracted from the extracted two image frames. A difference image is generated by CL7, the pixel density of the difference image is stored in a frame memory (not shown), and the process proceeds to S24. That is, the image of the previous frame in FIG. 19A and the image of the current frame in FIG. 19B are extracted, and a difference image between the two extracted image frames is generated as shown in FIG. Migrate to

  The generation of the difference image is sequentially executed for image frames that are close in time series, and a plurality of the sequentially generated difference images are stored in the frame memory in time series. At this time, the plurality of difference images stored in the time series in the frame memory are sequentially updated so that the plurality of difference images stored in the frame memory are sequentially in a new time series.

  In S24, the difference image generated by the difference image generation means CL7 is binarized by the binarized image processing means CL8 to generate a binarized image as shown in FIG. The process proceeds to contour detection processing.

  In the contour detection process of S25, the vertical contour detection means CL9 is FL1 which is the left vertical line of the face continuous in the height direction from the binarized image of FIG. 20 [same as FIG. 19 (d)]. , FL2 and FL3 are detected as the left contour line of the face, FR1, FR2 and FR3 which are the right vertical lines of the face continuous in the height direction are detected as the right contour line of the face, and the cut point detection in S26 Transition to processing.

  At this time, since the vertical line 10 detected in the eye and the vertical line 11 detected in the nose are short, it is determined that they are not changing points constituting the contour of the face. Further, since the vertical line 12 between the hair and the background is substantially continuous with the vertical line of the ear portion, it is not used as the face outline.

  In the cutting point detection process in S26, the cutting point where the contour line is broken when the contour line is traced in one direction in the height direction of the face is detected by the contour cutting point detection means CL51, and the process proceeds to another contour detection process in S27. To do. In FIG. 12, cutting points Lc11, LC12, Lc21, LC22, Lc31, Lc32 of the contour lines FL1, FL2, FL3, and cutting points Rc11, Rc12, Rc21, Rc22, Rc31, Rc32 of the contour lines FR1, FR2, FR3 are detected. Further, in FIG. 13, the cut points Lc11 and Lc12 of the left contour line FL1 are detected, and the cut points Rc11, Rc12, Rc21, Rc22, Rc31 and Rc32 of the right face contour lines FR1, FR2 and FR3 are detected.

  In the other contour detection process in S27, the contour lines FL1, FL2, FL3 and the contour lines FR1, FR2, FR3 are detected as other contours by the other contour line detection means CL52, and the discontinuous point detection process in S28 is performed. Transition.

  An example of this other contour detection process will be described with reference to contour lines FR1, FR2, FR3 on the left side of the face as shown in FIG. 14, for example. In FIG. 14, an exploration area 8 having, for example, 30 to 50 pixels on the left and right and 5 to 10 pixels on the upper and lower sides is set around the cutting point Rc12 of the outline FR1. Then, the other contour detection means CL52 determines whether or not there are contour lines FR1 and FR2 deviated in the horizontal direction (left and right) in the exploration area 8, and if there are deviated contour lines FR1 and FR2, The line FR2 is detected as another outline with respect to the outline FR1.

  As shown in FIG. 15, the search area 8 is set at the cutting points Rc11, Rc21, Rc22, Rc31, Rc32, respectively, and detects that the contour lines FR1, FR2, FR3 are other contours. Similarly, the contour lines FL1, FL2, and FL3 can be detected as other contours.

  In the discontinuous point detection process of S28, since the contour lines FR1, FR2, FR3 detected in S27 are the other contour lines, the cutting points Rc12, Rc21, Rc22, Rc31 are discontinuous points of the contour lines FR1, FR2, FR3. Detect as. That is, the above-described search area 8 is also used for detecting the discontinuous points. For example, in FIG. 14, when there are contour lines FR1 and FR2 that are separated in the horizontal direction (left and right) in the exploration area 8, the contour lines FR1 and FR2 are not continuous. It is detected as a discontinuous point with respect to the line FR2. The cutting points Lc12, Lc21, Lc22, Lc31 and the cutting points Rc21, Rc22, Rc31 can be detected as discontinuous points in the same manner. As a result, the cutting points Rc11 and Rc32 in FIGS. 12 to 15 can be detected as not discontinuous points, and the cutting points Lc11 and Lc12 in FIG. 13 can be detected as not discontinuous points.

  Similarly, in S28, the contour lines FL1, FL2, FL3 and contour lines FR1, FR2, FR3 are other contour lines in S27, so that the discontinuous point detecting means CL5 detects the cutting points Lc12, Lc21, Lc22 detected in S26. , Lc31 are detected as discontinuous points of the contour lines FL1, FL2, and FL3, and the process proceeds to the ear region detection process of S29. As a result, the cutting points Lc11 and Lc32 can be detected as not being discontinuous points.

  In the ear region detection process in S29, the region surrounded by the discontinuous points is set as the ear region, and the process proceeds to the ear position setting process in S30. That is, in FIG. 12, the region surrounded by the cut points Lc12, Lc21, Lc22, and Lc31 that are discontinuous points is the left ear region EL, and is surrounded by the cut points Rc12, Rc21, Lc22, and Rc31 that are discontinuous points. This region is defined as the right ear region ER (see FIG. 16). In FIG. 13, a region surrounded by the cutting points Rc12, Rc21, Rc22, and Rc31, which are discontinuous points, is a right ear region ER.

  In the ear position setting process in step S30, the ear region setting unit CL6 sets the center of gravity position Lg of the ear region EL (see FIG. 12) as the left ear position, and the ear region setting unit CL6 sets the center of gravity position Rg of the ear region ER to the right. Shift to step S31.

  In S31 (see FIGS. 12 and 13), it is determined whether or not ears are detected on the left and right of the captured driver's face image (whether there are two), and the ears are detected on the left and right. If YES, the process returns to S20 and repeats the contour detection process. If no ear is detected on either the left or right, the process proceeds to S32.

  In S32, it is assumed that the face is greatly directed in the horizontal direction, the driver is notified of driving advice and the like, and the process returns to S20 to repeat the contour detection process.

  In this S32, it is assumed that the face is greatly facing in the horizontal direction, the driver is informed of driving advice, and the process proceeds to step S33.

  In step S33, it is determined whether or not the engine is stopped by turning the ignition key from the IG position (ignition position) to the ACC position (accessory position). If not, the process returns to S20 to perform the contour detection process. Repeat and stop if stopped.

  9, 10, and 12 show the state of facing the front, and FIGS. 11 and 13 show the state of looking at the left side. 9, 10, and 12, the left and right ears are visible, and in FIGS. 11 and 13, only the right ear is visible.

  As described above, the face part detection method according to the embodiment of the present invention captures the face of the person to be detected from the front, detects the contour of the face of the person to be detected from the captured image, and then A portion where the contour of the face is discontinuous in the width direction is detected, and a region surrounded by the detected discontinuous point is set as a region.

  That is, in this face part detection method, the face of the person to be detected is picked up by the image pickup means (face image acquisition means CL1, camera 1) from the front, and is picked up by this image pickup means (face image acquisition means CL1, camera 1). After detecting the contour of the face of the person to be detected from the captured image by the contour detecting means (CL4, CL4a), the discontinuous point detecting means CL5 detects the location where the contour of the face is discontinuous in the width direction. The region surrounded by the discontinuous points is set as the ear region by the ear region setting means CL6.

  According to this method, since the presence of the ear can be reliably obtained from the contour lines on both sides in the width direction of the face (Df) of the person (D) to be detected, the detection accuracy of the ear can be improved.

In addition, the face part detection device according to the embodiment of the present invention includes an imaging means (face image imaging means CL1, camera 1) for imaging the face (Df) of the detected person (driver D) from the front, and the imaging means. Contour detection means (CL4, CL4a) for detecting the contour of the face of the detected person (D) from (face image imaging means CL1, camera 1);
Discontinuous point detection means CL5 for detecting a point where the contour of the face (Df) of the detected person (D) detected by the contour detection means (CL4, CL4a) is discontinuous in the width direction; and the discontinuity And an ear region setting unit CL6 that uses the region surrounded by the discontinuous points detected by the point detection unit CL5 as an ear region (EL, ER).

  According to this configuration, since the presence of the ear can be reliably obtained from the contour lines on both sides in the width direction of the face (Df) of the person to be detected (D), the detection accuracy of the ear can be improved.

  In addition, the discontinuous point detection means CL5 of the face part detection device according to the embodiment of the present invention has contour lines (FL1 to FL3) in one direction in the height direction of the face (DF) of the person (D) to be detected. Contour cutting point detection means CL51 for detecting contour cutting points (cutting points Lc12, Lc21, Lc22, Lc31, cutting points Rc12, Rc21, Lc22, Rc31) where the contour line is cut off when tracking FR1 to FR3); The height direction of the face at a position separated in the face width direction around the contour cutting points (cutting points Lc12, Lc21, Lc22, Lc31, cutting points Rc12, Rc21, Lc22, Rc31) detected by the cutting point detection means CL51 And another contour line detecting means CL52 for detecting the presence or absence of another contour line extending in a straight line. Moreover, the discontinuous point detecting means CL5 makes the contour cut point detected by the contour cut point detecting means CL51 a discontinuous point when a contour line is detected by the other contour line detecting means CL52. It has become.

  According to this configuration, when contour lines extending in the height direction are detected on both sides in the width direction of the face (Df) of the person (D) to be detected, that is, on both sides of the face image 3, a plurality of lines are provided on each side in the face width direction. It is possible to detect whether or not the contour lines extending in the height direction are other contour lines. In addition, when there are contour lines extending in the plurality of height directions on each side in the face width direction as other contour lines, discontinuous points between the contour lines can be detected from the plurality of contour lines and the contour cutting point. And the part enclosed by this discontinuous point can be made into an ear | edge area | region.

  Further, the contour detecting means (CL4, CL4a) of the face part detecting device according to the embodiment of the present invention is a face width centerline defining means CL41 for defining the face width centerline L of the face of the detected person (D). And a density change for finding a point where the density value changes from the white direction to the black direction by looking at the change in image density from the face width center line defined by the face width center line defining means CL41 outward in the face width direction. Point detection means CL42. In addition, the contour detection means (CL4, CL4a) sequentially finds density change points by the density change point detection means CL42 in the height direction of the face of the person (D) to be detected, and successive density change points. Is a contour of the face (Df) of the person (D) to be detected.

  According to this configuration, it is possible to easily detect the contour line extending in the height direction on both sides of the face width direction of the person to be detected (D).

  Furthermore, the face width center line defining means CL41 of the face part detecting device according to the embodiment of the present invention is configured so that the center line of the image captured by the image capturing means is the face width center line.

  According to this configuration, the definition of the center line of the face image is simple and easy, so that the image processing time can be shortened compared to the case where the center line of the face image is obtained from the face image. In this configuration, when the person to be detected is a driver (driver), the position of the face hardly moves when driving, so that the face image is approximately the center of the image captured by the imaging means or the image approximately. This is because the face width center line may be fixed to the center in the width direction of the image when it is entirely present.

  In addition, the face width centerline defining means CL41 of the face part detecting device according to the embodiment of the present invention detects the detected object such as an eye or an eyebrow from an image picked up by the image pickup means (face image pickup means CL1, camera 1). A bilaterally symmetric face part detecting means CL411 is provided for detecting a face part that exists at a substantially symmetrical position on the left and right of the person's face. In addition, the face width center line defining means CL41 detects the face part that exists in a substantially symmetrical position on the left or right of the face of the detected person such as the eye or the eyebrow detected by the left-right symmetric face part detection means CL411. The perpendicular bisector between the connected face part line segments is defined as a face width center line L.

  According to this configuration, since a symmetrical face part located on the center side in the face width direction of the face image is detected, the center of the face width can be obtained even when one of the ear images is not in the face image. The line L can be reliably obtained, and the contour line extending in the height direction on both sides in the face width direction can be obtained.

  Further, the contour detection means CL4a of the face part detection device according to the embodiment of the present invention is a difference image from a plurality of images taken by the imaging means (face image imaging means CL1, camera 1) at different times in time series. Is provided with difference image generation means CL7. In addition, the contour detection means extracts the face contour by performing image processing on the difference image generated by the difference image generation means CL7. According to this configuration, the contour lines of a plurality of difference images can be detected in time series.

  Furthermore, the imaging means (face image imaging means CL1, camera 1) of the face part detection device according to the embodiment of the present invention continuously captures moving images in time series, and the difference image generation means CL7 The difference image is generated from two image frames that are close in series. According to this configuration, the contour lines of a plurality of difference images can be detected in a continuous time series.

  In addition, the difference image generation means of the face part detection device according to the embodiment of the present invention is configured to apply a pixel density of the difference image generated from two original images (image frames) to a plurality of images close in time series. Accumulate. According to this configuration, the pixel density of a plurality of difference images can be accumulated in a continuous time series, so that a continuous contour change can be detected by sequentially updating the accumulated difference images. Can do. As a result, a change in the direction of the face in the left-right direction or the up-down direction can be continuously detected as the time changes, and the movement of the face of the person to be detected can be continuously monitored.

It is a block diagram which shows the structure of the face part detection apparatus which concerns on the 1st Embodiment of this invention. It is explanatory drawing which shows arrangement | positioning at the time of mounting the face part detection apparatus of FIG. 1 in a vehicle. It is a flowchart which shows the processing operation of the face part detection apparatus which concerns on 1st Embodiment. FIG. 3 is an explanatory diagram showing the relationship between the face width center line and an example of a face image with the face width center line defined by the face width center line defining means shown in FIG. 1 as the center in the width direction of the image. FIG. 5 is an explanatory diagram showing the relationship between the face width center line and another example of the face image, with the face width center line defined by the face width center line defining means shown in FIG. 1 as the center in the width direction of the image; FIG. 2 is an explanatory diagram showing a face width center line defined by the face width center line defining means shown in FIG. 1 from the left and right eyes and displaying the relationship between the face width center line and an example of a face image. It is explanatory drawing which shows the other relationship between this face width centerline and a face image while calculating | requiring and displaying the face width centerline prescribed | regulated by the face width centerline definition means shown in FIG. (A) is explanatory drawing of the detection position of the face height direction for detecting the change point of face image density, (b) is an image density change which shows the change of the image density of the face width direction in the detection position of (a). FIG. It is explanatory drawing which shows the collection of the density change points detected by the density change point detection means of FIG. It is explanatory drawing which shows an example of the outline calculated | required by connecting the collection of the density | concentration change points of FIG. It is explanatory drawing which shows the other example of the outline calculated | required by connecting the collection of the density | concentration change points of FIG. It is explanatory drawing which shows the ear | edge area | region and ear position calculated | required from the outline of FIG. It is explanatory drawing which shows the ear | edge area | region and ear position calculated | required from the outline of FIG. It is explanatory drawing for calculating | requiring one of the discontinuous points of an outline from the some outline of a face. It is explanatory drawing for calculating | requiring the discontinuous point of another outline using the method of calculating | requiring the discontinuous point of FIG. It is explanatory drawing for calculating | requiring an ear | edge area | region and an ear | edge position from the discontinuous point calculated | required like FIG. It is a block diagram which shows the structure of the face part detection apparatus which concerns on the 2nd Embodiment of this invention. It is a flowchart which shows the processing operation of the face part detection apparatus which concerns on 2nd Embodiment. It is explanatory drawing for producing | generating a difference image by the difference image generation means of FIG. It is explanatory drawing for calculating | requiring the outline of a height direction from the difference image calculated | required in FIG. It is explanatory drawing which shows the relationship between the outline of a height direction calculated | required in FIG. 19, and a face image.

Explanation of symbols

1 ... Camera (imaging means)
CL1 ... face image imaging means (imaging means)
CL4 ... contour detection means CL41 ... face width center line defining means CL411 ... left and right symmetrical face part detection means CL42 ... density change point detection means CL4a ... contour detection means CL5 ... discontinuous point detection means CL51 ... contour cut point detection means CL52 ... other Contour line detection means CL6 ... ear region setting means CL7 ... difference image generation means D ... driver (detected person)
Df ... face EL, ER ... ear regions FL1 to FL3 ... contour lines FR1 to FR3 ... contour lines Lc12, Lc21, Lc22, Lc31 ... cutting points (contour cutting points)
Rc12, Rc21, Lc22, Rc31 ... cutting point (contour cutting point)

Claims (9)

A face part detection method for detecting a face part from a certain area on an image,
After the face of the person to be detected is imaged from the front and the contour of the face of the person to be detected is detected from the captured image, a portion where the contour of the face is discontinuous in the width direction is detected and detected. An area surrounded by discontinuous points is used as an ear area. Imaging means for imaging the face of the person to be detected from the front;
Contour detecting means for detecting the face contour of the detected person from the imaging means;
Discontinuous point detecting means for detecting a location where the contour of the face of the detected person detected by the contour detecting means is discontinuous in the width direction;
A face part detecting apparatus comprising: an ear region setting unit that uses an area surrounded by discontinuous points detected by the discontinuous point detecting unit as an ear region.   3. The face part detecting device according to claim 2, wherein the discontinuous point detecting means detects a contour cut point where a contour line is broken when the contour line is traced in one direction in a height direction of the face of the detected person. Contour cutting point detecting means for detecting the presence or absence of another contour line extending in the face height direction at a position separated in the face width direction around the contour cutting point detected by the contour cutting point detecting means A face detection unit, wherein a contour cut point detected by the contour cut point detection unit is used as a discontinuous point when a contour line is detected by the other contour line detection unit. apparatus.   3. The face part detecting apparatus according to claim 2, wherein the contour detecting means is defined by a face width center line defining means for defining a center line of the face width of the detected person and the face width center line defining means. Density change point detection means for finding a point where the density value changes from the white direction to the black direction by looking at the change in image density from the face width center line outward in the face width direction, and the face of the person to be detected An apparatus for detecting a part of a face, wherein a density change point is found by the density change point detection means sequentially in the height direction, and a set of continuous density change points is used as the face contour of the detected person.   5. The face part detection apparatus according to claim 4, wherein the face width center line defining means sets a center line of an image taken by the image pickup means as a face width center line.   5. The face part detecting device according to claim 4, wherein the face width center line defining means is present at a position that is substantially symmetrical to the left and right of the face of the person to be detected, such as an eye or an eyebrow, from an image captured by the imaging means. The face having a left and right symmetrical face part detecting means for detecting a face part, and present at a position substantially symmetrical to the left and right of the face of the detected person such as the eye or the eyebrow detected by the left and right symmetrical face part detecting means. A face part detecting apparatus characterized in that a vertical bisector of line segments between face parts connecting parts is defined as a face width center line.   The face part detection device according to claim 2, wherein the contour detection unit includes a difference image generation unit that generates a difference image from a plurality of images captured by the imaging unit at different times on a time series, and the difference An apparatus for detecting a facial region, wherein a facial contour line is extracted by subjecting the difference image generated by an image generating means to image processing.   8. The face part detection apparatus according to claim 7, wherein the imaging unit captures a moving image continuously in time series, and the difference image generation unit calculates the difference from two adjacent image frames in time series. An apparatus for detecting a facial part, characterized by generating an image. 9. The face part detection apparatus according to claim 8, wherein the difference image generation means accumulates the pixel density of the difference image generated from two original images for a plurality of adjacent images in time series. Face part detection device.