JP2007271554A - Face attitude detection method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To detect a face attitude direction efficiently, without requiring a complex imaging system. <P>SOLUTION: This face posture detection method has a distance deriving step for determining the distance between portions in the first reference portion group, which is three combinations between right and left pupils and right and left nostrils of an object person A; a position detection step for generating a facial image of the object person A with one camera 2, and detecting a two-dimensional position of the first reference portion group on the facial image, based on the face image; and an attitude deriving step for deriving the facial attitude of the object person A, by calculating the normal direction on a plane including the first reference portion group, based on the distance determined in the distance deriving step, and on the two-dimensional position detected in the position detection step. <P>COPYRIGHT: (C)2008,JPO&INPIT

Description

  The present invention relates to a face posture detection method for detecting the face posture of an observation subject.

Conventionally, it has been considered to mount a function for detecting the driver's face direction in an automobile or the like to monitor driving while looking away. Such a function provides important information for safe driving of a car. As this kind of face direction detection technology, the face image of the subject is captured by two cameras, the three-dimensional positions of the facial feature points are measured from the principle of triangulation, and the positions of these feature points A method of capturing the face direction from the relationship is disclosed (see Non-Patent Document 1 below). In addition, a method is also disclosed in which the coordinates of the pupil and nostril of a subject are calculated using two cameras, and the face direction is obtained according to the calculation result (see Patent Document 1).
JP 2005-266868 A Yoshio Matsumoto, Alexander Zelinsky, “An Algorithm for Real-time Stereo Vision Implementation of Head Pose and Gaze Direction Measurement”, Proceedings of IEEE Fourth International Conference on Face and Gesture Recognition (FG '2000), pp.499-505, 2000

  However, in the conventional face direction detection method described above, when the face direction is detected using two cameras, the detection range of the face direction is limited to a predetermined range (for example, ± 40 degrees). If you want to expand the range, you need another set of cameras. However, providing a plurality of sets of cameras may complicate the configuration of the detection system and increase the cost.

  Therefore, the present invention has been made in view of such problems, and provides a face posture detection method capable of efficiently detecting the face posture direction without requiring a complicated imaging system. The purpose is to do.

  In order to solve the above-described problem, the face posture detection method of the present invention includes a distance derivation step for obtaining a distance between parts in a first reference part group that is a combination of three of the left and right pupils and left and right nostrils of the subject. It was obtained in a position detecting step for generating a face image of the subject by one imaging means and detecting a two-dimensional position of the first reference region group on the face image based on the face image, and a distance deriving step. A posture deriving step for deriving the face posture of the subject by calculating the normal direction of the plane including the first reference region group based on the distance and the two-dimensional position detected in the position detecting step; Prepare.

  According to such a face posture detection method, the distance between three parts of the left and right pupils and nostrils of the subject is obtained, and 2 of the three parts are obtained from the face image imaged by one imaging means. Since the face posture is specified by detecting the dimensional position and calculating the normal direction of the plane including the three parts from the distance between the three parts and the two-dimensional position of the three parts, the feature points are obtained by a set of imaging means. The angle that can be captured by a single imaging unit is wider than the angle that can capture the image, and a simple imaging system enables efficient face posture detection.

  In the distance deriving step, the distance between the parts in the first reference part group is obtained by stereo measurement using two imaging means, and in the position detecting step, the first imaging means is used by using one of the two imaging means. It is preferable to detect a two-dimensional position of one reference region group. In this case, the distance between the feature points for each subject can be easily grasped, and at the same time, the range of detectable face postures can be expanded more efficiently.

  In the distance deriving step, the distance between parts in the second reference part group which is a combination of the left and right pupils and the right and left nostrils other than the first reference part group is also obtained. Based on the image, the two-dimensional position of the second reference part group on the face image is also detected, and in the posture deriving step, the distance related to the second reference part group obtained in the distance deriving step is detected in the position detecting step. The normal direction of the plane including the second reference region group is further calculated based on the two-dimensional position related to the second reference region group, and the normal direction and the normal direction of the plane including the first reference region are calculated. It is also preferable to derive the face posture of the subject using In this way, two combinations of the left and right pupils and nostrils are selected, and the face posture is detected from the normal directions of the two planes including the respective combination parts. More improved.

  Furthermore, in the position detection step, the illumination light is irradiated from the first light source attached to the imaging means toward the subject, and simultaneously the first face image is generated, while the distance from the optical axis of the imaging means is the first distance. A second face image is generated simultaneously with illuminating illumination light toward the subject from a second light source provided to be larger than the first light source, and the first face image and the second face image are generated. It is also preferable to detect the two-dimensional position of the pupil in the first reference region group by taking the difference between. With such a configuration, the face image obtained by irradiating the subject with illumination light from the first light source becomes an image in which the pupil shines brightly (bright pupil image), and the second light source for the subject The face image obtained by irradiating with illumination light is an image in which the pupil appears dark (dark pupil image), and by detecting the difference between the two images, a highly robust pupil can be detected.

  Still further, in the position detection step, the illumination light is irradiated from the first light source toward the subject and at the same time the first face image is generated, while the emission wavelength is different from the first light source, At the same time as illuminating the target person with illumination light, a second face image is generated, and the difference between the first face image and the second face image is taken to obtain 2 of the pupils in the first reference region group. It is also preferable to detect the dimensional position. By taking a difference between facial images obtained by irradiating illumination light of different wavelength regions to the subject, it is possible to detect a highly robust pupil.

  Furthermore, it is also preferable that the first and second light sources are provided so that the distances from the optical axis of the imaging means are equal. In this case, the configuration of the light source can be simplified and reduced in size while causing a luminance difference in the pupil portion.

  According to the face posture detection method of the present invention, the face posture direction can be efficiently detected without requiring a complicated imaging system.

  Hereinafter, preferred embodiments of a face posture detection method according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same or corresponding parts are denoted by the same reference numerals, and redundant description is omitted.

(First embodiment)
A first embodiment of the present invention will be described. First, the configuration of an imaging system for carrying out the face posture detection method according to the present invention will be described with reference to FIG.

  FIG. 1 is a plan view showing the positional relationship between the imaging system and the subject. As shown in the figure, the imaging system 1 includes a single camera (imaging means) 2 that captures the face image of the subject A, a light source 3a provided in the vicinity of the imaging lens on the front surface 2a of the camera 2, And a light source 3b provided at a position away from the front surface 2a of the camera 2.

  The camera 2 is not limited to a specific type as long as it is an imaging means capable of generating the face image of the subject A, but an image sensor such as a CCD or CMOS is used in that the image data can be processed with high real-time properties. Use the built-in digital camera. The subject A is positioned on the optical axis L1 of the imaging lens of the camera 2 when detecting the face posture.

  The light source 3a is configured to be able to irradiate illumination light having a near-infrared light component toward a range covering the subject A located on the optical axis L1 along the optical axis L1 of the camera 2. The light source 3b is fixed at a position where the distance from the optical axis L1 is further away from the light source 3a, and irradiates illumination light having a near-infrared light component toward a range covering the subject A along the optical axis L1. It is configured to be possible. Here, the illumination light emitted from the light sources 3a and 3b is light having different wavelength components (for example, center wavelengths of 850 nm and 950 nm) that cause a luminance difference in the pupil portion, and the light source 3b The distance from the optical axis L1 may be fixed at a position equal to the light source 3a. In this case, the configuration of the light source can be simplified and reduced in size while causing a luminance difference in the pupil portion.

  Note that the camera 2 and the light sources 3a and 3b are provided for the purpose of preventing the reflection of reflected light in the face image when the subject A is wearing glasses and making it easier to detect the nostril of the subject A. It is preferably provided at a position lower than the height of the face (for example, an inclination angle of 20 degrees to 30 degrees with respect to the horizontal plane of the optical axis L1).

  Next, a face posture detection method using the above-described imaging system 1 will be described.

  First, the distance between each of the three parts of the left and right pupil centers and the left nostril center (first reference part group) of the subject A is measured. In addition, the distance between the three parts of the left and right pupil centers and the right nostril center (second reference part group) of the subject A and the distance between the right and left nostril centers are also measured (the distance deriving step). ).

  Next, the subject A is positioned on the optical axis L1 of the camera 2, and a face image of the subject A facing in an arbitrary direction is captured. Based on the face image generated by the camera 2 in this way, the two-dimensional coordinates of the left and right pupil centers and the two-dimensional coordinates of the left and right nostrils on the face image are detected (position detection step). Hereinafter, a method for detecting the pupil center and the nostril center will be described in detail.

(Detection of pupil center)
At the time of capturing a face image, the light sources 3a and 3b are alternately turned on to generate a face image synchronized with each lighting, thereby obtaining a bright pupil image and a dark pupil image. The bright pupil image is an image obtained with the irradiation of the light source 3a, and the luminance of the pupil portion is relatively bright. On the other hand, the dark pupil image is an image obtained with the irradiation of the light source 3b, and the luminance of the pupil portion is relatively dark. These two types of images are obtained due to differences in the intensity of reflected light from the pupil accompanying irradiation of illumination light from the two light sources 3a and 3b. For example, in the case of a camera that employs field scanning, the light source images 3a and 3b are turned on in synchronization with the field signal of the camera 2 to separate the bright and dark pupil images between the odd and even fields. Can do. Then, after taking the difference between the bright pupil image and the dark pupil image, the range of the pupil portion is determined. By performing such difference processing, it is possible to detect a highly robust pupil.

  After that, the contour of the detected pupil is specified, an ellipse that can be approximated to the contour is calculated, and the center of the ellipse is obtained as the center position of the pupil. Alternatively, the position of the center of the pupil may be calculated using the centroid method after binarizing the image using the difference-processed image. At this time, if there is a moving target such as an eyelid in the image, the area other than the pupil may appear brightly, so the selection of the size of the image area when obtaining the center of gravity becomes a problem. Therefore, as described in JP-A-2005-348832, the position of the pupil center may be calculated using a separability filter. That is, the center coordinate that maximizes the degree of separation is obtained using a pattern close to a circle.

(Detection of nostril center)
The two-dimensional coordinates of the left and right nostril centers are detected with reference to the bright pupil image or the dark pupil image. That is, the center point of the left and right pupil centers is obtained, and a large window whose center substantially coincides with the nostril position when the subject A is assumed to face the front is set at a position below that. Detect nostrils with. Then, 0.8% of pixels having the lower luminance are detected by the P-tile method in the large window of the image, and converted into a binary image composed of HIGH pixels and LOW pixels. Thereafter, the detected binarized image is repeatedly expanded and contracted (morphological processing) to clarify the area in the image, and then the labeling process is performed to select the two areas from the larger one. The center, aspect ratio, and area of the rectangle formed from the top, bottom, left, and right end points are calculated. Here, the expansion process is a process of converting a target pixel into a HIGH pixel when at least one of the eight pixels in the vicinity of the target pixel is present in the binary image, and the contraction process is a binary process. This is processing for converting a target pixel into a LOW pixel when at least one of the eight pixels near the target pixel is present in the image. When the aspect ratio is smaller than 0.5 or larger than 0.7 and the entire image size is 640 × 240 pixels, the area is smaller than 100 pixels or larger than 300 pixels, the region indicating the nostril image Judge that is not. Otherwise, a small window of 30 × 30 pixels is set around the center of the rectangle, and 5% of pixels from the lower luminance are selected by the P-tile method for the inside of the small window of the original image. Extract. Thereafter, the above morphological process and labeling process are repeated to obtain a region having the maximum area. If the area of the region is 130 pixels or more and 70 pixels or less, it is determined not to be a nostril image, otherwise it is determined to be a nostril image, and the center of the rectangle formed by the upper, lower, left and right end points of the region is determined. Seek as the center. As a result, when two nostril centers are detected, the correspondence between the left and right nostrils is determined from the size of each coordinate value.

  As described above, when the nostril detection is performed using the large window and the small window, an optimum threshold value can be given for detecting each of the two nostrils having different imaging conditions, and the nostril can be detected reliably.

If only one of the right and left nostrils can be detected during such nostril detection, the distance measured in the distance deriving step and the detected positions of the left and right pupil centers and the center of one nostril are used. Then, the position of the other nostril center is estimated. Now, the coordinates of the right pupil center are (x _RP , y _RP ), the coordinates of the left pupil center are (x _LP , y _LP ), the coordinates of the right nostril center are (x _RN , y _RN ), and the coordinates of the left nostril center are _Assume that (x _LN , y _LN ), the measured distance between the left and right pupil centers is D _P0 , the measured distance between the left and right nostril centers is D _N0 , and the left nostril cannot be detected. Ramp rate I _P, and the pupil center distance D _P between left and right pupils in the face image at this time, the following equation (1) and (2);
I _P = (y _RP −y _LP ) / (x _RP −x _LP ) (1)
^{_{D P = {(x RP -x}} LP) 2 + (y RP -y LP) 2} 1/2 ... (2)
Can be considered.

Here, since the line connecting the left and right pupil centers and the line connecting the left and right nostril centers are considered to be always parallel, the slope ratio I _N = I _P between the left and right nostrils. Further, the nostrils center distance D _N on the face image, the measured distance D _P0 between the left and right pupil _center, found distance D _N0 between the left and right nostrils _center, and from the pupil center distance D _P, the following equation (3) ;
D _N = (D _N0 / D _P0 ) × D _P (3)
It is obtained by. The nostril center distance _DN is expressed by the following formula (4);
^{_{D N = {(x RN -x}} LN) 2 + (y RN -y LN) 2} 1/2 ... (4)
Therefore, the coordinates of the center of the left nostril (x _LN , y _LN ) are the following formulas (5) and (6);
^{_{x LN = x RN - {D}} N 2 / (1 + I N 2)} 1/2 ... (5)
^{_{y LN = y RN - {(}} D N 2 .I N 2) / (1 + I N 2)} 1/2 ... (6)
Can be obtained. Conversely, when the right nostril center is not detected, the following formulas (7) and (8);
^{_{x RN = x LN + {D}} N 2 / (1 + I N 2)} 1/2 ... (7)
^{_{y RN = y LN + {(}} D N 2 .I N 2) / (1 + I N 2)} 1/2 ... (8)
Can be obtained.

(Face posture detection)
Based on the positions of the center of the left and right pupils and the center of the left and right nostrils of the face image detected in the position detection step, and the distance actually measured in the distance derivation step, The three-dimensional coordinates of the left and right nostril centers are calculated to derive the posture of the subject A (posture derivation step). Hereinafter, a method for deriving the posture of the target person will be described in detail.

FIG. 2 is a diagram showing a positional relationship between the image plane PL and the subject A in a two-dimensional coordinate system with the principal point of the imaging lens of the camera 2 as the origin. As shown in the figure, when the three-dimensional coordinate system (X, Y, Z) is set in a direction in which the Z axis coincides with the optical axis of the camera 2, the image plane PL of the two-dimensional coordinate system (x, y) is The distance from the origin O becomes the focal length f of the camera 2 and can be regarded as a plane perpendicular to the Z axis. Further, four feature points of the left and right pupil centers and the right and left nostril centers on the image plane PL of the subject A are represented by P _n = (X _n , Y _n , Z _n ) (n = 1, 2, 3, 4). In other words, the distance L _ij (i, j = 1, 2, 3, 4) between the feature points in the three-dimensional space is expressed by the following formula (9);
L _ij = {(X _i −X _j ) ² + (Y _i −Y _j ) ² + (Z _i −Z _j ) ² } ^1/2 (9)
Given in. When the two-dimensional coordinates of the projected image of such a feature point P _{n on} the image plane PL are Q _n (x _n , y _n ) (n = 1, 2, 3, 4), P _n is projected from the origin O. it can be considered to be on an extension line passing through the image Q _n. Therefore, the position vector _{p n} of _{P n,} with a unit vector _{u n} scalar _{a n} facing the direction of the straight line OQ _n, the following equation (10);
p _n = a _n · u _n (10)
Can be represented by Unit vector _{u n} coordinate values of the projected image _{Q n (x n, y n} ) u using ^{_{n = (x n, y n}} , f) / (x n 2 + y n 2 + f 2) 1/2 and calculation is, since the coordinates of the projected image Q _n are known, it is possible to determine the three-dimensional coordinates of P _n if Motomare is a _n.

Here, the distance L _ij (i, j = 1, 2, 3) between the left and right pupil centers and the left nostril center (first reference region group) of the four feature points P _n is measured in advance. , the following equation using the position vector _{p n} of the feature point _{P n} (11);
| P _i −p _j | = L _ij (i, j = 1, 2, 3) (11)
Therefore, based on Expression (10) and Expression (11), the following Expressions (12) to (14);
a ₁ ² + a ₂ ² −2a ₁ a ₂ (u ₁ , u ₂ ) = L ₁₂ (12)
a ₂ ² + a ₃ ² −2a ₂ a ₃ (u ₂ , u ₃ ) = L ₂₃ (13)
a ₃ ² + a ₁ ² -2a ₃ a ₁ (u ₃ , u ₁ ) = L ₃₁ (14)
Is guided. By solving the scalars a ₁ , a ₂ , a ₃ from these equations (12) to (14), the three-dimensional coordinates P ₁ , P ₂ , P ₃ of the first reference region group can be calculated.

Similarly, based on the measured distance L _ij (i, j = 1, 2, 4) of the left and right pupil centers and the right nostril center (second reference region group) of the four feature points P _n. calculates the three-dimensional coordinates _{P 4.} The three-dimensional coordinates P ₁ (X _PR , Y _PR , Z _PR ), P ₂ (X _PL , Y _PL , Z _PL ), P ₃ (X _NL , Y _NL , Z _NL ), P thus obtained are determined. ₄ (X _NR , Y _NR , Z _NR ), the plane normal vector n _L = (n _LX , n _LY , n _LZ ) including the first reference region group is expressed by the following equations (15) to (17). ;
_{n LX = (Y NL -Y PL} ) (Z PR -Z PL) - (Y PR -Y PL) (Z NL -Z PL) ... (15)
_{n LY = (Z NL -Z PL} ) (X PR -X PL) - (Z PR -Z PL) (X NL -X PL) ... (16)
_{n LZ = (X NL -X PL} ) (Y PR -Y PL) - (X PR -X PL) (Y NL -Y PL) ... (17)
Derived by Further, a normal vector n _R = (n _RX , n _RY , n _RZ ) of the plane including the second reference region group is derived by the same calculation method. Finally, a resultant vector n _F = n _R + n _L = (n _X , n _Y , n _Z ) of the obtained normal vectors n _L and n _R is derived as a face direction vector indicating the final face posture direction.

Further, the centroids (X _GR , Y _GR , Z _GR ) and (X _GL , Y _GL , Z _GL ) of the two triangles including the first and second reference site groups obtained as described above are as follows: Formula (18), (19);
( _XGR , _YGR , _ZGR ) = {( _XPR + _XPL + _XNR ) / 3, ( _YPR + _YPL + _YNR ) / 3, ( _ZPR + _ZPL + _ZNR ) / 3} (18)
( _XGL , _YGL , _ZGL ) = {( _XPR + _XPL + _XNL ) / 3, ( _YPR + _YPL + _YNL ) / 3, ( _ZPR + _ZPL + _ZNL ) / 3} (19)
And the centroid (X _GC , Y _GC , Z _GC ) between the two centroids is calculated by the following equation (20):
( _XGC , _YGC , _ZGC ) = {( _XGR + _XGL ) / 2, ( _YGR + _YGL ) / 2, ( _ZGR + _ZGL ) / 2} (20)
Ask for.

Finally, the face posture of the subject A is specified as passing through the center of gravity (X _GC , Y _GC , Z _GC ) and the direction being represented by the vector n _F. Thereby, the position of the subject's face and the direction of the face can be determined. The centroid (X _GC , Y _GC , Z _GC ) calculated as the face position may be calculated as the centroid of the four feature points.

  According to the face posture detection method described above, the distance between the three parts of the left and right pupils and nostrils of the subject A is obtained, and the three parts are detected from the face image captured by one camera 2. The face posture is determined by detecting the two-dimensional position and calculating the normal direction of the plane including the three parts from the distance between the three parts and the two-dimensional position of the three parts. An angle that can be captured by one camera is wider than an angle at which a feature point can be captured by stereo measurement or the like, and a simple imaging system enables efficient face posture detection. For example, assuming that the horizontal range in which the feature points of the subject can be captured with one camera is ± 45 degrees, the face posture is determined using two cameras whose optical axis tilt angle is about 10 degrees. In contrast to the detection range of ± 40 degrees when the camera is detected (see FIG. 4), according to the detection method of the present invention, a face posture detection range of about ± 45 degrees is realized with one camera. (See FIG. 1).

  Also, two combinations of the left and right pupils and nostrils are selected as the first and second reference region groups, and the face posture is detected from the normal directions of the two planes including the respective region groups. The face posture detection accuracy is further improved.

(Second Embodiment)
Next, a second embodiment of the present invention will be described. FIG. 3 is a plan view showing the positional relationship between the imaging system 101 and the subject A according to the second embodiment of the present invention. As shown in the figure, the imaging system 101 includes two cameras (imaging means) 102 and 112 that capture the face image of the subject A, and the vicinity of the imaging lenses of the front surfaces 102a and 112a of the cameras 102 and 112, respectively. Light sources 103a and 113a, and light sources 103b and 113b provided at positions away from the front surfaces 102a and 112a of the camera 2. That is, the face posture of the subject A is detected using two cameras.

  The camera 102 and the camera 112 are arranged so that their optical axes L2 and L3 are orthogonal to each other. The subject A is located at the intersection of the optical axis L2 and the optical axis L2 when the face posture is detected. The functions of the light sources 103a and 103b and the positional relationship with the camera 102, and the functions of the light sources 113a and 113b and the positional relationship with the camera 112 are the same as those of the light sources 3a and 3b described in the first embodiment.

  Hereinafter, a face posture detection method using the imaging system 101 will be described focusing on differences from the first embodiment.

First, a face image is captured in a state where the face is directed toward the subject A in the direction along the optical axis L2 of the camera 102 or the optical axis L3 of the camera 112, and the positions of the left and right pupil centers and the positions of the right and left nostrils Is detected, a distance D _P1 between the left and right pupil centers on the face image and a distance D _N1 between the left and right nostril centers are calculated. Then, the subject A is caused to face in the direction of the straight line L4 passing through the intersection of the optical axis L2 and the optical axis L3 and the midpoint between the camera 102 and the camera 112. Therefore, each of the two cameras 102 and 112 simultaneously captures a face image and detects the two-dimensional positions of the four feature points on the face image. Here, since the optical axes L2 and L3 of the cameras 102 and 112 are inclined by 45 degrees with respect to the face direction of the subject A, the cameras 102 and 112 may not be able to detect the nostrils. Thus if it can not be detected either nostril, the distance D _N1 between the distance D _P1 and the left and right nostrils center between the left and right pupil center which has been previously calculated, already a distance between the detected pupil center and a D _P, in the manner described above and the same method to estimate the two-dimensional position of the nostrils center. Thereafter, three-dimensional coordinates of four feature points Pn (n = 1, 2, 3, 4) are obtained by stereo measurement from the two-dimensional positions of the feature points in the two face images, and each feature point is obtained from these three-dimensional coordinates. The distance L _ij (i, j = 1, 2, 3, 4) is calculated (the distance deriving step).

Thereafter, the subject A is faced in the direction to be detected, and based on the face image generated by either one of the two cameras 102 and 112, 2 at the center of the left and right pupils on the face image. The dimension coordinates Q ₁ and Q ₂ and the two-dimensional coordinates Q ₃ and Q ₄ of the right and left nostril centers are detected (position detection step). At this time, which camera's face image to select is determined by whether or not the feature point of the subject A has been successfully detected.

Finally, the two-dimensional positions Q _n (n = 1, 2, 3, 4) of the four feature points on the face image detected in the position detection step, and the distance L _ij (i, i, i) calculated in the distance derivation step. j = 1, 2, 3, 4), after calculating the three-dimensional coordinates P ₁ , P _{2 of} the actual left and right nostril centers and the three-dimensional coordinates P ₃ , P ₄ of the right and left nostril centers, The posture of the subject A is derived by obtaining the normal vectors n _L and n _R of the plane including the reference part group or the second reference part group (posture derivation step).

According to the face posture detection method described above, the distance between the feature points P _n for each subject A can be easily grasped, and at the same time, the range of the detectable face posture can be efficiently expanded with a small number of cameras. it can. For example, when the angle formed by the optical axes L2 and L3 of the two cameras 102 and 112 is 90 degrees, the face direction can be detected within a range of ± 90 degrees (see FIG. 3).

  In addition, this invention is not limited to embodiment mentioned above. For example, in the face posture detection method of the present invention, three feature points excluding one of the nostril centers are selected as the reference region group of the subject A for which the normal vector is calculated. Various combinations can be selected as combinations of feature points. For example, feature points excluding the pupil center may be used as the reference region group.

It is a top view which shows the positional relationship of the imaging system concerning 1st Embodiment of this invention, and a subject. It is a figure which shows the positional relationship of the image plane in the two-dimensional coordinate system which made the origin the main point of the imaging lens of the camera of FIG. 1, and the subject. It is a top view which shows the positional relationship of the imaging system concerning 2nd Embodiment of this invention, and a subject. It is a top view which shows the positional relationship of the imaging system concerning a comparative example of this invention, and a subject.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1,101 ... Imaging system, 2,102,112 ... Camera, 3a, 3b, 103a, 103b, 113a, 113b ... Light source, A ... Subject.

Claims (6)

A distance deriving step for obtaining a distance between parts in the first reference part group which is a combination of three of the left and right pupils and right and left nostrils of the subject;
A position detecting step of generating a face image of the subject by one imaging means, and detecting a two-dimensional position of the first reference region group on the face image based on the face image;
Based on the distance obtained in the distance deriving step and the two-dimensional position detected in the position detecting step, calculating a normal direction of a plane including the first reference region group, A posture deriving step for deriving the face posture of the subject;
A face posture detection method comprising: In the distance deriving step, the distance between the parts in the first reference part group is obtained by stereo measurement using two imaging means,
In the position detection step, a two-dimensional position of the first reference region group is detected using one of the two imaging means.
The face posture detection method according to claim 1. In the distance derivation step, the distance between the parts in the second reference part group that is a combination of the left and right pupils and the left and right nostrils other than the first reference part group is also obtained,
In the position detection step, based on the face image, a two-dimensional position of the second reference part group on the face image is also detected,
In the posture derivation step, based on the distance related to the second reference part group determined in the distance derivation step and the two-dimensional position related to the second reference part group detected in the position detection step, Further calculating the normal direction of the plane including the second reference part group, and deriving the face posture of the subject using the normal direction and the normal direction of the plane including the first reference part,
The face posture detection method according to claim 1 or 2. In the position detecting step, illumination light is emitted from the first light source attached to the imaging means toward the subject and a first face image is generated simultaneously, while the distance from the optical axis of the imaging means is A second face image is generated simultaneously with illuminating illumination light toward the subject from a second light source provided to be larger than the first light source, and the first face image and the Detecting a two-dimensional position of the pupil in the first reference region group by taking a difference from the second face image;
The face posture detection method according to any one of claims 1 to 3. In the position detection step, the first face image is generated at the same time as the illumination light is irradiated from the first light source toward the subject, while the second light source having a light emission wavelength different from that of the first light source is used. A pupil in the first reference region group is generated by irradiating illumination light toward the subject and simultaneously generating a second face image and taking a difference between the first face image and the second face image. Detecting the two-dimensional position of
The face posture detection method according to any one of claims 1 to 3. The first and second light sources are provided so that the distances from the optical axis of the imaging means are equal.
The face posture detection method according to claim 5.

Images