JP5396298B2 - Face orientation detection device - Google Patents

Description

  The present invention relates to a face orientation detection device.

  Conventionally, for example, vertical edge detection is performed on a face image of a human face to detect provisional contour positions on the left and right of the face, and a plurality of candidate positions for the contour position are determined based on pixels around the provisional contour position. An apparatus is known that extracts contour positions at the left and right ends by statistical processing on candidate positions (see, for example, Patent Document 1).

JP 2007-139535 A

  By the way, according to the apparatus according to the prior art, when detecting the left and right ends of the face, first, vertical edge detection is performed on the face image. However, for example, when a part of the face is covered by a person's clothes or hands, a vertical edge is detected on the face, and the vertical edge is different from the vertical edges corresponding to the left and right edges of the face. Is erroneously detected, which causes a problem that the contour positions at the left and right ends cannot be accurately extracted.

  The present invention has been made in view of the above circumstances, and provides a face orientation detection device capable of accurately detecting the left and right face edge positions of a human face and improving the detection accuracy of the face orientation. Objective.

  In order to solve the above-described problems and achieve the object, the face orientation detection device according to the first aspect of the present invention is an imaging means that images a human face and outputs a face image (for example, in the embodiment) An occupant camera 12) and position detection means for detecting the face center position and the face edge position of the person based on the face image output from the imaging means and outputting a detection result (for example, the face center in the embodiment) A position detection unit 22 and a face edge position detection unit 24) and a face direction detection unit (for example, a face in the embodiment) that detects the face direction of the person based on the detection result output from the position detection unit. A face direction detection device including a direction detection unit 25), wherein a search line setting for setting a plurality of search lines crossing the human face included in the face image in the face image output from the imaging unit Means (eg in the embodiment A search line setting unit (23), wherein the position detection means detects a plurality of face edge positions based on the plurality of search lines set by the search line setting means and outputs detection results, The detection means detects a plurality of face orientation directions based on the plurality of detection results output from the position detection means, and detects a median value of the plurality of face orientation directions as the face orientation direction of the person. .

  Furthermore, in the face orientation detection device according to the second aspect of the present invention, the position detection means has a length of the search line within a predetermined range among the plurality of search lines set by the search line setting means. And the left end of the search line is on the left side of the face center position, and the right end of the search line is on the right side of the face center position. A plurality of face edge positions are detected.

  According to the face direction detection device according to the first aspect of the present invention, the median values of the plurality of face direction directions detected based on the face edge positions detected for each of the plurality of search lines crossing the human face are obtained. Since the direction of the human face is ultimately set, the detection accuracy and the reliability of the detection result are improved compared to the case where the face direction is detected only from a specific part of the face or the left and right end positions of the face, for example. be able to.

  Furthermore, according to the face direction detection device according to the second aspect of the present invention, the length is within a predetermined range, the left end is on the left side of the face center position, and the right end is on the right side of the face center position among the plurality of search lines. By excluding search lines other than the search line, it is possible to improve the detection accuracy of the face edge position detected for each of the plurality of search lines.

It is a block diagram of the face direction detection apparatus which concerns on embodiment of this invention. It is a figure which shows the several search line in the face image which concerns on embodiment of this invention. It is a figure which shows an example of angle (theta) which the face direction direction makes | forms with respect to the front direction of the driver | operator who seated on the driver's seat which concerns on embodiment of this invention. It is a flowchart which shows operation | movement of the face direction detection apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process of the face edge position detection of each search line which concerns on embodiment of this invention. It is a flowchart which shows the process of the brightness | luminance map creation on the search line which concerns on embodiment of this invention. It is a flowchart which shows the process of the face left and right edge detection which concerns on embodiment of this invention. It is a flowchart which shows the process of the face left and right edge detection which concerns on embodiment of this invention.

Hereinafter, an embodiment of a face direction detection device of the present invention will be described with reference to the accompanying drawings.
As shown in FIG. 1, for example, the face orientation detection device 10 according to the present exemplary embodiment includes an imaging control device 11, an occupant camera 12, a light source control device 13, an imaging light source 14, and a processing device 15. The processing device 15 further includes a binocular detection unit 21, a face center position detection unit 22, a search line setting unit 23, a face edge position detection unit 24, and a face direction detection unit 25. Has been.

The occupant camera 12 controlled by the imaging control device 11 is disposed, for example, on an instrument panel in the passenger compartment, and includes at least the face of the driver seated in the driver's seat of the vehicle in the imaging region as an imaging target. The face image including the driver's face can be output in the region or the infrared region.
The imaging light source 14 controlled by the light source control device 13 is disposed, for example, at a position shifted from the occupant camera 12 on the instrument panel in the passenger compartment to the left and right, and faces at least the driver seated in the driver's seat of the vehicle. Light such as visible light or infrared light can be irradiated.

  The binocular detection unit 21 of the processing device 15 performs recognition processing such as feature amount calculation and shape determination using, for example, a driver's eyeball as a detection target on the face image output from the occupant camera 12, for example. The position of the eye (for example, the center position of the iris, the center position of the Purkinje image, which is an infrared reflection image on the cornea surface, the center position of the eyeball, etc.) is detected.

  For example, as shown in FIG. 2A, the face center position detection unit 22 calculates the centers of the left and right eye positions EL and ER of the driver detected by the binocular detection unit 21, and uses the calculated results as driving. Is output as the person's face center position FC.

For example, as shown in FIG. 2B, the search line setting unit 23 sets a plurality of search lines SL crossing the driver's face in the face image FP output from the occupant camera 12. Each search line SL is, for example, a line having a vertical width of one pixel for each pixel in the vertical direction of the face image FP and extending in the left-right direction. Note that the vertical width of each search line is not limited to one pixel, and may be other vertical widths.
Further, the search line setting unit 23 is caused by, for example, the presence of a predetermined vertical position (for example, an eye) within a predetermined range CA including the face center position FC detected by the face center position detection unit 22. The search line SL is set at a position other than a position where the luminance variation in the left-right direction is likely to be equal to or greater than the predetermined variation.

  The face edge position detection unit 24 excludes search lines that do not satisfy the predetermined condition (for example, search lines SL1 and SL2 shown in FIG. 2C), and the left and right edges (left and right) of the face for each of the plurality of search lines. Face end position) FL and FR are detected. The predetermined condition is that the left end of the face is on the left side of the face center position FC detected by the face center position detection unit 22 and the right end of the face is on the right side of the face center position FC detected by the face center position detection unit 22. And the distance between the left end of the face and the right end of the face is within a predetermined range.

  The face direction detection unit 25 detects the face center position FC detected by the face center position detection unit 22 and the left and right face end positions FL and FR detected by the face end position detection unit 24 for each of a plurality of search lines. Based on the above, the direction of the driver's face is detected by, for example, a cylinder method for calculating the face direction by approximating a human face to a cylinder shape.

  For example, as shown in FIG. 3, the face direction detection unit 25 sets the distance in the left-right direction between the center position CO between the left and right face edge positions FL and FR and the face center position FC as a face center shift r, The distance between the left and right face edge positions FL and FR is the face width 2R, and the angle α formed by the direction from the center position CO toward the passenger camera 12 with respect to the front direction of the driver seated in the driver's seat is Based on the shift r and the face width 2R, an angle θ formed by the face direction with respect to the front direction of the driver seated in the driver's seat is calculated. For example, assuming that the angle β = 90−α, the following formula (1) by the sine theorem is modified to obtain the following formula (2), and the angle θ is described from the following formula (2) as shown in the following formula (3). Is done.

  Then, the face direction detection unit 25 finally detects the median value of these face direction directions as the driver's face direction direction based on a plurality of face direction directions.

  The face orientation detection device 10 according to this embodiment has the above-described configuration. Next, the operation of the face orientation detection device 10 will be described.

First, for example, in step S01 shown in FIG. 4, the face center position is acquired based on the face image output from the occupant camera 12.
Next, in step S02, a median luminance is detected within a predetermined range around the face center position.
Next, in step S03, in the face image output from the occupant camera 12, vertical positions for setting a plurality of search lines that cross the driver's face are set.

Next, in step S04, processing for detecting the face edge position of each search line, which will be described later, is executed.
Next, in step S05, the face direction is detected for each face end position of each search line.
Next, in step S06, based on a plurality of face direction directions, the median value of these face direction directions is detected as the driver's face direction direction, and the process proceeds to the end.

Hereinafter, the process of detecting the face edge position of each search line in step S04 described above will be described.
For example, in step S11 shown in FIG. 5, it is determined whether or not the following steps S12 to S15 have been completed for all search lines.
If this determination is “YES”, the flow proceeds to return.
On the other hand, if this determination is “NO”, the flow proceeds to step S 12.
In step S12, a process for creating a luminance map on the search line, which will be described later, is executed.
Next, in step S13, processing for detecting the left and right edges of the face is executed.

Next, in step S14, it is determined whether the left and right edges of the face (left and right face edge positions) have been detected.
If this determination is “YES”, the flow proceeds to step S 18 described later.
On the other hand, if this determination is “NO”, the flow proceeds to step S15.
In step S15, the brightness threshold at this point is changed.
In step S16, a brightness map creation process on the search line, which will be described later, is executed.
Next, in step S17, processing for detecting the left and right edges of the face is executed.
And in step S18, it moves to the next search line and returns to step S11 mentioned above.

Hereinafter, the process of creating the luminance map on the search line in step S12 described above will be described.
First, for example, in step S21 shown in FIG. 6, the image processing range is set as a search line.
Next, in step S22, noise removal processing is performed using, for example, a median filter.

Next, in step S23, for example, based on the median luminance calculated in step S02 described above, a luminance threshold indicating whether or not various processes (for example, face left and right edge detection processing) can be performed on the luminance, that is, A luminance threshold value indicating a luminance range determined to be the driver's face is set.

Next, in step S24, the luminance within the luminance range corresponding to the luminance threshold is set to a predetermined value (for example, 255) and the luminance outside the luminance range is set to zero with respect to the luminance on the search line. The luminance is converted by the above process.
Next, in step S25, the luminance after conversion on the search line is smoothed by, for example, a morphological filter, and the process proceeds to return.

Hereinafter, the process of detecting the left and right ends of the face in step S13 described above will be described.
First, for example, in step S31 shown in FIG. 7, initialization is performed by setting the value of the horizontal coordinate X on the search line to zero.
Next, in step S32, whether or not the value of the coordinate X is smaller than a predetermined image width value (that is, the width value of the face image), that is, the value of the coordinate X is the right end of the face image (the right end of the image). It is determined whether or not the value is smaller than the coordinate value of.
If this determination is “NO”, the flow proceeds to step S 41 described later.
On the other hand, if the determination is “YES”, the flow proceeds to step S33.

Next, in step S33, it is determined whether or not the left edge of the face has been detected.
If this determination is “YES”, the flow proceeds to step S 37 described later.
On the other hand, if this determination is “NO”, the flow proceeds to step S 34.
Next, in step S34, the luminance of the coordinate X is a predetermined value (that is, the predetermined value set in step S25 described above, for example, 255), and the coordinate X is the coordinate of the face center position (that is, It is determined whether or not it is smaller than the horizontal coordinate.
If this determination is “NO”, the flow proceeds to step S 36 described later.
On the other hand, if the determination is “YES”, the flow proceeds to step S35.

In step S35, the coordinate X is set as the coordinate of the left end of the face (that is, the horizontal coordinate).
In step S36, a value obtained by adding "1" to the coordinate X is newly set as the value of the coordinate X, and the process returns to the above-described step S32.
Note that the horizontal coordinate values of the face image are set to increase from the left side to the right side.

In step S37, it is determined whether or not the luminance of the coordinate X is zero (that is, zero which is the value set in step S25 described above).
If this determination is “NO”, the flow returns to step S 36 described above.
On the other hand, if this determination is “YES”, the flow proceeds to step S38.
Next, in step S38, it is determined whether (coordinate X−coordinate of the left end of the face) is equal to or smaller than a predetermined face width.
If the determination result is “YES”, the process proceeds to step S39. In this step S39, the coordinate X is set as the coordinate of the right end of the face (that is, the horizontal coordinate), and the process proceeds to step S41.
On the other hand, if this determination is “NO”, the flow proceeds to step S 40, and in this step S 40, initialization is performed to set the coordinate of the left end of the face to zero, and the flow proceeds to step S 41.

In step S41, it is determined whether only the left edge of the face is detected.
If this determination is “NO”, the flow proceeds to step S 44 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S42.
In step S42, it is determined whether or not (coordinate of the right end of the image−coordinate of the left end of the face) is equal to or smaller than a predetermined face width.
If this determination result is “NO”, that is, if the driver's face is completely disregarded at the right end of the face image, the process proceeds to step S44 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S43.
In step S43, the coordinates of the right edge of the image are set as the coordinates of the right edge of the face, and the process proceeds to step S44.

Then, for example, in step S44 shown in FIG. 7, it is determined whether or not the right edge of the face has been detected.
If this determination is “NO”, the flow proceeds to return.
On the other hand, if this determination is “YES”, the flow proceeds to step S45.
In step S45, the luminance within a predetermined range around the search line is acquired in the face image before the luminance conversion in step S24 described above.
Next, in step S46, the coordinate of the left end of the face is set as the coordinate X.

In step S47, the luminance difference between the coordinate X and the coordinate (X-1) adjacent to the left of the coordinate X is equal to or greater than a predetermined difference, or the luminance of the coordinate X is equal to It is determined whether it is smaller than the luminance.
If this determination is “NO”, the flow proceeds to step S 49 described later.
On the other hand, if this determination is “YES”, the flow proceeds to step S48.
In step S48, the coordinate of the right end of the face is set as the coordinate X, and the process proceeds to step S51 described later.
In step S49, a value (X-1) obtained by subtracting "1" from the value of coordinate X is newly set as coordinate X.
In step S50, the coordinate X is set as the coordinate of the left end of the face, and the process returns to step S47.

In step S51, the luminance difference between the coordinate X and the coordinate (X + 1) to the right of the coordinate X is greater than or equal to a predetermined difference, or the luminance of the coordinate X is smaller than the luminance of the coordinate (X + 1). It is determined whether or not.
If this determination is “NO”, the flow proceeds to step S 52, where a value (X + 1) obtained by adding “1” to the value of the coordinate X is newly set as the coordinate X. .
In step S53, the coordinate X is set as the coordinate of the right end of the face, and the process returns to step S51 described above.

As described above, according to the face direction detection device 10 according to the present embodiment, a plurality of face direction directions detected based on the face edge positions detected for each of the plurality of search lines crossing the driver's face. Since the median is finally set as the driver's face direction, the detection accuracy and the detection result are higher than when detecting the face direction only from a specific part of the face or the left and right end positions of the face, for example. Reliability can be improved.
Furthermore, a search line other than the search line whose distance between the left end of the face and the right end of the face is within a predetermined range, the left end is on the left side of the face center position, and the right end is on the right side of the face center position among the plurality of search lines. By excluding, the detection accuracy of the left and right face edge positions detected for each of the plurality of search lines can be improved.

DESCRIPTION OF SYMBOLS 10 Face direction detection apparatus 12 Crew camera (imaging means)
22 Face center position detection unit (position detection means)
23 Search line setting section (search line setting means)
24 face edge position detection unit (position detection means)

Claims (2)

Image pickup means for picking up an image of a human face and outputting a face image, and position detection means for detecting a face center position and face edge position of the person based on the face image output from the image pickup means and outputting a detection result And a face orientation detection device for detecting the face orientation direction of the person based on the detection result output from the position detection means,
Search line setting means for setting a plurality of search lines crossing the human face included in the face image in the face image output from the imaging means,
The position detection means detects a plurality of face edge positions based on the plurality of search lines set by the search line setting means, and outputs detection results;
The face orientation detection means detects a plurality of face orientation directions based on a plurality of the detection results output from the position detection means, and determines a median value of the plurality of face orientation directions as the face orientation direction of the person. A face direction detecting device characterized by detecting as follows. The position detection means has a length of the search line within a predetermined range among the plurality of search lines set by the search line setting means, and a left end of the search line is closer to the face center position. 2. The plurality of face edge positions are detected based on the plurality of search lines that exist on the left side and the right end of the search line exists on the right side of the face center position. The face orientation detection device according to 1.

Images