JP2010205076A - Eye detecting device, and method used in the device - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an eye detecting device which accurately detects the corner and inner corner of a person's eye even if the shapes of persons' eyes have individual difference. <P>SOLUTION: Contour enhanced images are generated, which enhance the contours of face images which are obtained by imaging the images of persons' faces. On the basis of the contour enhanced images thus generated, the candidates of pixels, which individually indicate the upper eyelids and the lower eyelids, are detected as the upper eyelid candidates and the lower eyelid candidates, respectively. The detected upper eyelid candidates and lower eyelid candidates are combined as the eyelid candidates, and shape values which become indicators for the shapes of the eyelids, are computed for each of the combined eyelid candidates. According to the computed shape values, the adequate tail and inner corner patterns of the persons' eyes are selected to make pattern-matching processing. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a detection apparatus and method for detecting a human eye based on a face image.

  2. Description of the Related Art Conventionally, various detection devices that capture an image including a human face and detect a human eye based on the captured image have been developed. As an example of such a detection apparatus, for example, an eye detection apparatus described in Patent Document 1 (hereinafter referred to as a conventional technique) can be given.

  In the prior art, a face including the background of the driver driving the vehicle is captured as a face image. When a face image is captured, in the prior art, for example, by performing filter processing such as a horizontal edge detection Sobel filter and a vertical edge detection Sobel filter on the face image, the driver's image is detected from the face image. A candidate for a combination of upper and lower eyelids is detected as a candidate for eyelids. When the eyelid candidates are detected, in the prior art, in order to identify the eyelids from the eyelid candidates, eye corner areas are respectively set on the left and right ends of the lower eyelids of the eyelid candidates.

  In the prior art, a predetermined brightness value of a face image pixel is overlapped with a skin area pixel in the overlapped eye corner eye pattern by superimposing a predetermined eye corner eye pattern in the face image before the filtering process. Average value (hereinafter referred to as skin pixel average value) and an average value of luminance values of face image pixels overlapping with pixels of the eye part in the overlapped eye corner pattern (hereinafter referred to as eye pixel average value). Is calculated. When calculating the skin pixel average value and the eye pixel average value, the conventional technique calculates the skin pixel average value and the eye pixel average value of the left and right eye corner areas for each eyelid candidate.

  When the average value of the left and right skin pixels and the average value of the eye pixels are calculated for each eyelid candidate, in the conventional technique, an eye corner eye evaluation value obtained by subtracting the eye pixel average value from the skin pixel average value is set for each set eye corner eye area. calculate. In the prior art, the eye corner eye evaluation value is calculated for each of the left and right eye corner areas of each eyelid candidate. When calculating the eye corner eye evaluation value, in the conventional technology, the addition value of the left eye corner eye evaluation value and the right eye corner eye evaluation value is calculated for each right and left eye corner area of each eyelid candidate, and the calculated addition value Identifies the candidate with the largest eyelid and the candidate with the second largest eyelid as the left and right eyelids having the eye corners indicated by the eye corner-eye pattern, the eye corner closest to the eye corner, the eye corner second and the eye corner closest to the eye, and the eye head, respectively. .

JP 2008-226125 A

  However, the above prior art has the following problems. That is, there are individual differences in the shape of the human eye, such as round eyes and fine eyes. However, in the above prior art, since the eye-eye-eye evaluation value is calculated using only one kind of eye-eye-eye-head pattern, the eye-eye of the eye having a shape greatly different from the eye-eye-eye-head pattern, the eye-ear close to the eye, and the eyelid Candidates cannot be accurately detected as traps.

  Therefore, an object of the present invention is to provide an eye detection device that can accurately detect the corners of the eyes and the eyes even if there are individual differences in the shape of human eyes.

In order to achieve the above object, the present invention has the following features.
According to a first aspect of the present invention, there are provided an imaging unit that captures an image including a face of the subject as a face image, and pixel candidates that respectively indicate the upper eyelid and the lower eyelid of the subject based on the face image captured by the imaging unit. A detection unit that detects each combination as a candidate for eyelids, an eye corner for each eyelid candidate detected by the detection unit in the face image, and an eye corner of a predetermined image for calculating the similarity between the images indicating the eyes Based on the shape value indicating the shape of each eyelid candidate detected by the detecting means and the storage means for storing only two or more eyeball patterns, one eye corner eye pattern is selected from the eye corner eye pattern stored in the memory means. Selection means for selecting and associating with each candidate, an image showing the eye corners and eyes of each eyelid candidate detected by the detecting means in the face image, and the eye selected for each eyelid candidate by the selection means And a specifying unit configured to calculate a similarity between the image of inner corner pattern.

  A second invention is an invention subordinate to the first invention, and is at least one of a corner of the eye and a head of a pixel indicating a candidate for a lower eyelid combined as a candidate for eyelid detected by the detecting means And an inclination detecting means for detecting the inclination of a line connecting the end point of the image and the pixel at the highest vertex in the vertical direction in the face image among the pixels indicating the upper eyelid candidate, and the selecting means detects the inclination Based on the inclination detected by the means, an eye corner pattern is selected.

  A third invention is an invention subordinate to the second invention, wherein the inclination detection means includes an end point on the eye corner side of a pixel indicating a lower eyelid candidate combined as an eyelid candidate detected by the detection means, The inclination of the line connecting the vertex pixel is detected as a shape value.

  A fourth invention is an invention subordinate to the second invention, wherein the inclination detection means includes an end point on the top side of a pixel indicating a lower eyelid candidate combined as a eyelid candidate detected by the detection means, and The inclination of the line connecting the vertex pixel is detected as a shape value.

  The fifth invention is an invention subordinate to the second invention, wherein the inclination detecting means includes a pixel at an end of the eyelid side and a front side of a candidate for a lower eyelid combined as a candidate for eyelid detected by the detecting means. The slope of the line connecting each of the end point pixels and the vertex pixel is detected as a shape value.

  A sixth invention is an invention subordinate to the first invention, further comprising opening degree detecting means for detecting the opening degree of the eyelid as a shape value for each eyelid candidate detected by the detecting means, and the selecting means comprises Then, based on the opening detected by the opening detection means, an eye corner pattern is selected.

  The seventh invention is an invention subordinate to the first invention, and detects, for each wrinkle candidate detected by the detecting means, the curvature of a pixel of an upper eyelid candidate combined as the wrinkle candidate as a shape value. And a curvature detecting means for selecting the eye corner pattern based on the curvature detected by the curvature detecting means.

  The eighth invention is an invention dependent on the first invention, further comprising illuminance detection means for detecting the illuminance of the ambient light irradiated to the subject, and the selection means is detected by the illuminance detection means. When the illuminance of the ambient light is less than a predetermined threshold, an eye corner-eye pattern is selected.

  A ninth invention is an invention subordinate to the first invention, further comprising time detection means for detecting the current time, and the selection means, when the time detected by the time detection means indicates nighttime, Select the corner pattern.

  A tenth invention is an invention subordinate to the first invention, further comprising operation detection means mounted on the vehicle for determining whether or not a wiper of the vehicle is operating, and the selection means is an operation detection When it is determined by the means that the wiper is operating, an eye corner pattern is selected.

  An eleventh aspect of the present invention is an imaging step of capturing an image including a face of the subject as a face image, and pixel candidates indicating the upper eyelid and the lower eyelid of the subject based on the face image captured in the imaging step. The detection step for detecting each combination as a wrinkle candidate, and 2 of images determined in advance to calculate the similarity between the image indicating the eye corners and the eyes for each wrinkle candidate detected in the detection step in the face image Based on the shape value indicating the shape of each eyelid candidate from the above eye corner eye pattern, a selection step for selecting and correlating one eye corner eye pattern for each eyelid candidate, and detection in the detection step in the face image A specific step for calculating the similarity between the image indicating the corners and the eyes of each eyelid candidate and the image of the eye corners pattern selected for each eyelid candidate in the selection step. And a flop.

  According to the first aspect of the invention, since the corners of the face image and the corners of the eyes for detecting the eyes are selected based on the shape of the eyelid candidate in the face image, there are individual differences in the shape of the human eye. Even so, the corners of the eyes and the head of the eyes can be accurately detected.

  According to the second aspect of the present invention, at least one of the endpoints of the top side and the bottom side of the pixel indicating the lower eyelid candidate combined with the upper eyelid candidate as the eyelid candidate, and the vertex of the pixel indicating the upper eyelid candidate An eye corner pattern can be selected based on the shape value detected as the slope of the line connecting the two.

  According to the third aspect, the inclination of the line connecting the end of the pixel indicating the lower eyelid candidate combined with the upper eyelid candidate as the eyelid candidate and the vertex of the pixel indicating the upper eyelid candidate is detected. Based on the shape value, an eye corner pattern can be selected.

  According to the fourth aspect of the invention, it is detected as the slope of the line connecting the top end of the pixel indicating the lower eyelid candidate combined with the upper eyelid candidate as the eyelid candidate and the vertex of the pixel indicating the upper eyelid candidate. Based on the shape value, an eye corner pattern can be selected.

  According to the fifth aspect, each of the line connecting the end point on the eye side of the pixel indicating the lower eyelid candidate combined with the upper eyelid candidate as the eyelid candidate and the vertex of the pixel indicating the upper eyelid candidate Can be selected based on the shape value detected as the slope of each line connecting the two.

  According to the sixth aspect of the present invention, an eye corner pattern can be selected based on the opening degree of the eyelid.

  According to the seventh aspect, it is possible to select an eye corner pattern based on the radius of curvature of the pixel indicating the upper eyelid candidate.

  According to the eighth aspect of the invention, it is possible to select an eye corner pattern based on a face image captured without being affected by ambient light.

  According to the ninth aspect, it is possible to select an eye corner pattern based on a face image captured without being affected by ambient light.

  According to the tenth aspect of the invention, it is possible to select an eye corner pattern based on a driver's face image captured without being affected by ambient light due to rain.

  Further, according to the eye detection method of the present invention, the same effects as those of the above-described eye detection device according to the present invention can be obtained.

Functional block diagram showing a functional configuration of the eye detection apparatus according to the first embodiment The figure which shows an example of the face image imaged in this invention The figure which shows an example of the installation position of the imaging part in this invention The figure which shows an example of the outline emphasis image in this invention The figure explaining the combination of the upper eyelid candidate and the lower eyelid candidate The figure which shows an example of the shape value calculated in 1st Embodiment The figure which shows an example of the shape value when the shape of a wrinkle candidate is fine The figure which shows an example of the shape value in case the shape of a wrinkle candidate is standard The figure which shows an example of the shape value when the shape of a wrinkle candidate is round The figure which shows an example of an eye corner eye pattern corresponding to a fine eyelid candidate The figure which shows an example of an eye corner eye pattern corresponding to a standard eyelid candidate The figure which shows an example of an eye corner eye pattern corresponding to a round eyelid candidate The flowchart which shows the process of the eye detection apparatus which concerns on 1st Embodiment. The figure which shows an example of the shape value calculated as an opening degree The figure which shows an example of the opening degree when the shape of a heel candidate is fine The figure which shows an example of the opening degree when the shape of a heel candidate is standard The figure which shows an example of the opening degree when the shape of a heel candidate is round The figure which shows an example of the shape value calculated as a curvature The figure which shows an example of a circle when calculating a curvature when the shape of a wrinkle candidate is fine The figure which shows an example of a circle when calculating a curvature when the shape of a heel candidate is standard The figure which shows an example of a circle when calculating a curvature when the shape of a wrinkle candidate is round

(First embodiment)
FIG. 1 is a functional block diagram showing a functional configuration of the eye detection apparatus 1 according to the first embodiment of the present invention. The eye detection apparatus 1 according to this embodiment includes an imaging unit 101, a detection unit 102, a combination unit 103, a shape value calculation unit 104, a selection unit 105, a specification unit 106, and a storage unit 107. In the description of the present embodiment, a case where the eye detection apparatus 1 processes a face image obtained by capturing an image including the face of the driver driving the vehicle will be described as an example.

  The imaging unit 101 is typically a CCD camera, and sequentially captures an image including a driver's face as a face image as shown in FIG. 2 as a face image. Face image data indicating an image is generated. The installation position of the imaging unit 101 in the vehicle may be any installation position as long as the eye detection apparatus 1 according to the present embodiment can accurately recognize the corners of the driver's face, the eyes, and the like. As an example, as shown in FIG. 3, the driver's face seated in the driver's seat may be installed on the upper part of the steering column of the vehicle so that the face of the driver can be imaged from the front. In addition, when installing the imaging unit 101, a light source that irradiates infrared light in the imaging range of the imaging unit 101 is also installed, and the imaging unit 101 that has relatively enhanced sensitivity to light of wavelengths in the infrared region is used. A face image may be taken.

  The functional configurations of the detection unit 102, the combination unit 103, the shape value calculation unit 104, the selection unit 105, and the specification unit 106 are typically controlled by integrated circuits such as a CPU, LSI, and microcomputer. The control unit reads out a predetermined program from the storage unit 107 and interprets and executes the program, so that the control unit appropriately processes each function configuration.

  The detection unit 102 acquires face image data generated each time face image data is generated by the imaging unit 101. When the face image data is acquired, the detection unit 102 causes the storage unit 107 to store the acquired face image data. When the face image data is stored in the storage unit 107, the detection unit 102 detects facial parts such as upper eyelid, lower eyelid, nostril, and mouth from the face image indicated by the face image data stored in the storage unit 107. A process of detecting candidate pixels each indicating at least upper eyelid and lower eyelid in (hereinafter referred to as a face part) is performed.

  More specifically, the detection unit 102 first performs a filter process for emphasizing a contour on the face image indicated by the face image data stored in the storage unit 107, and the face image indicated by the face image data. A process for generating an image in which the contour of the facial part is enhanced as a contour-enhanced image is performed. When the contour enhanced image is generated, the detection unit 102 further stores the generated contour enhanced image in the storage unit 107. When the edge enhancement image is stored in the storage unit 107, the detection unit 102 uses the technique such as pattern matching to indicate pixels indicating the upper eyelid and the lower eyelid from the edge enhancement image stored in the storage unit 107, respectively. Are detected as an upper eyelid candidate and a lower eyelid candidate, respectively.

  When detecting the upper eyelid candidate and the lower eyelid candidate, respectively, the detection unit 102 may further perform processing for improving the detection accuracy when detecting the upper eyelid candidate and the lower eyelid candidate based on the contour-enhanced image. As an example of processing for improving the detection accuracy when the detection unit 102 detects the upper eyelid candidate and the lower eyelid candidate, the upper edge or the lower eyelid is indicated by processing using a method such as pattern matching in the contour-enhanced image. Among pixels determined to be candidates, pixels closer than a predetermined distance are grouped together as candidates for pixels having the same upper eyelid or the same lower eyelid, or one upper eyelid candidate, or one A grouping process is listed as a candidate for Shimojo. Further, as another example of the processing for improving the detection accuracy when the detection unit 102 detects the upper eyelid candidate and the lower eyelid candidate, after detecting all the upper eyelid candidates and the lower eyelid candidates in the contour-enhanced image Processing for grouping the upper eyelid candidates or the lower eyelid candidates that include overlapping pixels among the detected upper eyelid candidates and lower eyelid candidates as one upper eyelet candidate or lower eyelid candidate. Is mentioned.

  FIG. 4 is a diagram illustrating an example of the upper eyelid candidate and the lower eyelid candidate detected by the detection unit 102. The example illustrated in FIG. 4 illustrates a case where the detection unit 102 detects five upper eyelid candidates of upper eyelid candidates um1 to um5 and three lower eyelid candidates of lower eyelid candidates sm1 to sm3. The upper eyelid candidates um1, um3, and um5 shown in FIG. 4 are pixels in which the outlines of the eyebrows and the mouth are actually emphasized, but in the method for detecting the upper eyelid candidates and the lower eyelid candidates described above, In some cases, one or more pixels having a shape similar to the upper eyelid or the lower eyelid, such as a pixel indicating the outline of the eyebrows, are detected as the upper eyelid candidate and the lower eyelid candidate.

  In FIG. 4, for convenience of explanation, the upper eyelid candidate and the lower eyelid candidate are indicated by broken lines, but the pixels indicating the upper eyelid candidate and the lower eyelid candidate are not indicated by broken lines in the outline-enhanced image. May be. Further, the upper eyelid candidate and the lower eyelid candidate detected by the detection unit 102 may be detected as a line shape as shown in FIG. 4 respectively, or pixels having a shape similar to the shape of the upper eyelid or the lower eyelid. May be detected as a collection of In addition, when the detection unit 102 detects the upper eyelid candidate and the lower eyelid candidate, respectively, the detection unit 102 generates by performing horizontal Sobel filter processing on the face image indicated by the face image data acquired from the imaging unit 101. The contour-enhanced image may be used.

  When detecting the upper eyelid candidate and the lower eyelid candidate, the detecting unit 102 determines whether the detected upper eyelid candidate and the lower eyelid candidate are left and right candidates for each detected upper eyelid candidate and lower eyelid. Judgment is made for each candidate. When determining whether the detected upper eyelid candidate and lower eyelid candidate are left or right candidates, the detection unit 102 performs face image processing on the face image indicated by the face image data acquired from the imaging unit 101. The determination may be made based on a vertical contour emphasis image generated by performing filter processing for emphasizing the vertical contour.

  More specifically, when the detection unit 102 determines whether the detected upper eyelid candidate and lower eyelid candidate are the left and right candidates, first, the detection unit 102 indicates the face image data stored in the storage unit 107. A vertical contour emphasis image is generated by performing filter processing (for example, vertical Sobel filter processing, etc.) for emphasizing the vertical contour of the face image on the face image to be stored, and further storing it in the storage unit 107 . When the vertical contour enhanced image is stored in the storage unit 107, the detection unit 102 accumulates the luminance values of the pixels of the vertical contour enhanced image for each pixel column in the vertical direction, and the accumulated luminance values of the pixels along the horizontal direction. Create a histogram of the vertical contour-enhanced images arranged side by side. The horizontal axis of the histogram of the vertical contour emphasized image indicates the pixel position in the horizontal direction of the vertical contour emphasized image, and the vertical axis indicates the integrated luminance value. In the histogram of the vertical contour emphasis image, peak values are generated at horizontal positions corresponding to both ends of the face. The detection unit 102 detects the horizontal position of the left end of the face and the horizontal position of the right end of the face based on the horizontal position of the peak value generated in the histogram of the vertical contour enhanced image, and further detects The center of each horizontal position is detected as the horizontal position of the center of the face.

  When the horizontal position of the center of the face is detected, the detection unit 102 compares the detected horizontal position of the center of the face with the positions of pixels indicating the already detected upper eyelid candidate and lower eyelid candidate. To do. In the present embodiment, pixel positions in images such as a face image, a contour-enhanced image, and a vertical contour-enhanced image are XY based on the origin of the image as shown in FIG. 3 and FIG. 4 as an example. It shall be indicated in the coordinate system. Then, the detection unit 102 is located at an X coordinate smaller than the X coordinate of the upper eyelid candidate and the lower eyelid candidate positioned to the left of the horizontal center position of the face in the contour-enhanced image, that is, the face center X coordinate. A pixel upper eyelid candidate and a lower eyelid candidate are determined as an upper eyelid candidate and a lower eyelid candidate on the right side of the face, respectively. On the other hand, the detection unit 102 is positioned at an X coordinate larger than the X coordinate of the upper eyelid candidate and the lower eyelid candidate positioned to the right of the horizontal position of the face center in the contour-enhanced image, that is, the face center. The upper eyelid candidate and the lower eyelid candidate of the pixel are determined as the upper eyelid candidate and the lower eyelid candidate on the left side of the face, respectively. Thereby, the detection unit 102 can determine whether the detected upper eyelid candidate and lower eyelid candidate exist on the left or right side of the face in the face image.

  Note that the upper eyelid candidate and the lower eyelid candidate detected by the detection unit 102 are each a collection of pixels having a linear shape or a shape similar to the shape of the eyelid (upper eyelid or lower eyelid), as described above. , Detected as a collection of a plurality of pixels. When the detection unit 102 compares the horizontal position of the detected center of the face with the positions of the pixels indicating the detected upper eyelid candidates and lower eyelid candidates, the upper eyelid candidate or the lower Any position of the pixels indicating the eyelid candidates may be compared with the horizontal position of the center of the face, but as an example, the horizontal positions of the pixels indicating the upper eyelid candidates or the lower eyelid candidates. And a method of comparing the average value of the face and the horizontal position of the center of the face.

  When the detection unit 102 detects the upper eyelid candidate and the lower eyelid candidate, and determines whether the detected upper eyelid candidate and the lower eyelid candidate exist on the left or right, the detected upper eyelid candidate and the lower eyelid candidate The upper and lower eyelid candidate data indicating the above are generated. More specifically, for each detected upper eyelid candidate, the detection unit 102 has each position in the contour-enhanced image of all pixels indicating one upper eyelid candidate, and the upper eyelid candidate exists on either the left or right side of the face Upper eyelid candidate data is generated by associating information indicating whether or not to be identified with identifiers for identifying upper eyelid candidates. Further, the detection unit 102 indicates, for each detected lower eyelid candidate, each position in the contour-enhanced image of all pixels indicating one lower eyelid candidate and whether the lower eyelid candidate exists on the left or right of the face. The lower eyelid candidate data indicating the information and the identifiers for identifying the lower eyelid candidates in association with each other is generated. Then, when the detection unit 102 generates the upper eyelid candidate data and the lower eyelid candidate data, respectively, the detection unit 102 generates data indicating the generated upper eyelid candidate data and the lower eyelid candidate data as upper and lower eyelid candidate data, and the storage unit It is memorized in 107.

  The combination unit 103 combines the upper eyelid candidate and the lower eyelid candidate indicated by the upper and lower eyelid candidate data stored in the storage unit 107 as the eyelid candidate. More specifically, when the upper and lower eyelid candidate data is stored in the storage unit 107, the combination unit 103 combines the upper eyelid candidate data and the lower eyelid candidate stored as the upper and lower eyelid candidate data stored in the storage unit 107. The upper eyelid candidate and the lower eyelid candidate respectively indicated by the data are combined to make the eyelid candidate.

  When combining the upper eyelid candidate and the lower eyelid candidate, the combining unit 103 divides and combines the upper eyelid candidate and the lower eyelid candidate existing on the left and right respectively. More specifically, when combining the upper eyelid candidate and the lower eyelid candidate, the combining unit 103 combines the left upper eyelid candidate and the lower eyelid candidate, and combines the right upper eyelid candidate and the lower eyelid candidate. . Further, when combining the upper eyelid candidate and the lower eyelid candidate, the combination unit 103 moves downward by a predetermined number of pixels st from the vertical position of the upper eyelid candidate on the contour-enhanced image, as shown in FIG. The lower eyelid candidate and the upper eyelid candidate existing in the region may be combined respectively. At this time, the position of the upper eyelid candidate on the contour-enhanced image in the vertical direction may be the position of the pixel present at the highest vertical position among the pixels indicating one upper eyelid candidate.

  When combining the upper eyelid candidate and the lower eyelid candidate as the eyelid candidate, the combination unit 103 generates eyelid candidate data indicating the eyelid candidate combining the upper eyelid candidate and the lower eyelid candidate. More specifically, the combination unit 103 displays, for each candy candidate, candy candidate data indicating the combined upper candy candidate identifier, the combined lower candy candidate identifier, and the identifier for identifying the candy candidate in association with each other. Is generated. When the wrinkle candidate data is generated, the combination unit 103 causes the storage unit 107 to store the generated wrinkle candidate data. In the example shown in FIG. 5, a pair of the upper eyelid candidate um1 and the lower eyelid candidate sm1, a pair of the upper eyelid candidate um2 and the lower eyelid candidate sm1, a pair of the upper eyelid candidate um3 and the lower eyelid candidate sm2, A pair of the candidate um4 and the lower eyelid candidate sm2 is combined as an eyelid candidate by the combining unit 103, and eyelid candidate data indicating the combined eyelid candidates is generated. On the other hand, since the upper eyelid candidate um5 and the lower eyelid candidate sm3 are not combined with the upper eyelid candidate or the lower eyelid candidate, respectively, the upper eyelid candidate um5 and the lower eyelid candidate sm3 are included in the combination. Does not become a candy candidate indicated by the candy candidate data.

  The process from when the detection unit 102 detects the upper eyelid candidate and the lower eyelid candidate to when the combination unit 103 generates the eyelid candidate data, as described above, selects the upper eyelid candidate and the lower eyelid candidate. Any known method may be used instead of the above-described processing as long as it can be detected and data that can be processed by the shape value calculation unit 104, the selection unit 105, and the specification unit 106 as described later can be generated.

  When the eyelid candidate data is stored in the storage unit 107, the shape value calculating unit 104 calculates a value indicating the shape of the eyelid candidate as a shape value for each eyelid candidate indicated by the stored eyelid candidate data.

  FIG. 6 is a diagram illustrating an example of a shape value calculated by the shape value calculation unit 104. In FIG. 6, for the convenience of explanation, the upper eyelid candidate um2 and the lower eyelid candidate sm1 among the upper eyelid candidate and the lower eyelid candidate shown in FIG. When the shape value calculation unit 104 calculates the shape value of one eyelid candidate, as shown in FIG. 6, the shape value calculating unit 104 has the highest vertical direction among the pixels indicating one upper eyelid candidate associated with the eyelid candidate data. The position of the apex pixel ut present at a high position, the end mj on the eye corner side of the pixel indicating the lower eyelid candidate of the identifier associated with the identifier of the upper eyelid candidate, and the end point mg on the eye side The slope of a straight line connecting at least one of the end points is calculated as a shape value. It should be noted that the position of the vertex pixel ut present at the highest position in the vertical direction among the pixels indicating one upper eyelid candidate and the position in the face image are the same position.

  The inclination calculated by the shape value calculation unit 104 is determined as a heel candidate based on the position of the vertex pixel ut present at the highest position in the vertical direction among pixels indicating one upper heel candidate and the identifier of the upper heel candidate. It may be an inclination of a straight line mjl (hereinafter referred to as an eye corner inclination) that connects an end point mj on the eye corner side indicating a lower eyelid candidate associated with the candidate data. In addition, the inclination calculated by the shape value calculation unit 104 is determined based on the position of the vertex pixel ut present at the highest position in the vertical direction among the pixels indicating one upper eyelid candidate and the identifier of the upper eyelid candidate. The slope of a straight line mgl connecting the end point mg on the eye side indicating the lower eyelid candidate of the identifier associated with the eyelid candidate data (hereinafter referred to as the eye inclination). In addition, the inclination calculated by the shape value calculation unit 104 may be both the above-described eye corner inclination and the above-mentioned eye inclination.

  FIGS. 7A to 7C show that the shape of the eyelid candidate combining the upper eyelid candidate um2 and the lower eyelid candidate sm1 detected by the detection unit 102 is close to a fine shape, a shape close to a standard shape, and a round shape, respectively. It is a figure which respectively shows an example of the corner | angular inclination mjk and the eye inclination mgk when it is a near shape. As shown in FIGS. 7A to 7C, the eye corner inclination mjk and the eye inclination mgk calculated as shape values by the shape value calculation unit 104 increase in the order of fine, standard, and round, respectively. And the shape of the heel candidate combining the lower heel candidates. Note that the eye corner inclination mjk and the eye head inclination mgk calculated as shape values by the shape value calculation unit 104 may be indicated by angles as shown in FIGS. 7A to 7C, respectively, or by the inclinations of the respective straight lines. There may be.

  When the shape value calculation unit 104 calculates a shape value for each of the eyelid candidates indicated by the eyelid candidate data stored in the storage unit 107, the shape value calculation unit 104 associates the calculated shape value with the identifier of the eyelid candidate indicated by the eyelid candidate data. Then, it is stored in addition to the eyelid candidate data stored in the storage unit 107.

  When the shape value is added to the eyelid candidate data stored in the storage unit 107 by the shape value calculating unit 104, the selection unit 105 identifies the eye corner and the eye head for each eyelid candidate indicated by the eyelid candidate data. An eye corner pattern for specifying at 106 is selected.

  Here, an eye corner pattern stored in advance in the storage unit 107 will be described with reference to FIGS. 8A to 8C. FIG. 8A to FIG. 8C are diagrams each showing an eye corner pattern stored in advance in the storage unit 107 according to the present embodiment. FIG. 8A to FIG. 8C show, as an example, an eye corner pattern of a right eyelid candidate and an eye corner eye pattern for specifying the eye head by the specifying unit 106.

  The eye corner pattern in the present embodiment is a pattern in which the eye corner pattern and the eye pattern shown in FIGS. 8A to 8C are combined. The eye corner pattern and the eye head pattern each have a predetermined area having two areas corresponding to the eye corner and the skin part and the eye part of the eye head, respectively. It is an image. In addition, the eye corner pattern in the present embodiment indicates the skin portion (the white portion in FIGS. 8A to 8C) with the upper limit value of the luminance value of the pixel of each image, and the lower limit value of the luminance value of the pixel of each image. Although the eye part (black part in FIGS. 8A to 8C) is shown, the way of showing the skin part and the eye part in the eye corner pattern is not limited to this. In addition, the eye corner pattern and the eye pattern stored in the storage unit 107 are pixels that indicate candidates for the lower eyelid of the pattern so that the specifying unit 106 arranges the eye corner pattern and the eye pattern on the face image, respectively. The position of the pixel corresponding to the end point (end point on the contact side with the upper eyelid candidate) is predetermined as a reference point.

  In the present embodiment, as an example, the storage unit 107 together with identifiers that predetermine three eye corner patterns corresponding respectively to the shapes of the narrow, standard, and round eyelid candidates shown in FIGS. 8A to 8C and identify each of them. Shall be memorized. As is clear from FIGS. 8A to 8C, the ratio of the area between the skin part and the eye part in the eye corner pattern and the eye pattern is different from each other in the eye corner pattern corresponding to each of the fine, standard, and round eyes. More specifically, the ratio of the skin part is the largest and the ratio of the eye part is the smallest in the eye corner pattern and the eye head pattern of the eye corner eye pattern corresponding to the fine details. On the other hand, the ratio of the skin part is the smallest and the ratio of the eye part is the largest in the eye corner pattern and the eye pattern corresponding to the round eyes.

  As described above, in this embodiment, as shown in FIGS. 8A to 8C, in each of the corners of the eyes and the eyes that change according to the shape of the eyes, such as fine, standard, and round eyes, in the above-described predetermined area. The eye corner-eye pattern is stored in the storage unit 107 in advance so that the ratio of the area between the skin part and the eye part becomes close to each shape.

  The selection unit 105 includes a threshold value for selecting an eye corner pattern stored in the storage unit 107 in accordance with the shape value associated with the eyelid candidate by the eyelid candidate data stored in the storage unit 107. Is predetermined. As an example, the selection unit 105 according to the present embodiment selects a threshold value a and a threshold value b that increase in order in order to select three eye corner patterns stored in the storage unit 107. These two threshold values are predetermined. Here, it is assumed that there is a relationship of a <b between the threshold value a and the threshold value b.

  When the shape value is added to the eyelid candidate data stored in the storage unit 107, the selection unit 105 adds the shape value and the threshold value associated with each eyelid candidate indicated by the stored eyelid candidate data. Compare When comparing the shape value with the threshold value, the selection unit 105 first compares the shape value associated with one wrinkle candidate with the smallest threshold value a. Then, when the selection unit 105 determines that the compared shape value is smaller than the threshold value a, the selection unit 105 determines that the shape of the eyelid candidate associated with the shape value is fine, and the corner of the eye corresponding to the detail The eye pattern identifier is associated with the eyelid candidate identifier and added to the eyelid candidate data stored in the storage unit 107 and stored. When the selection unit 105 determines that the compared shape value is greater than or equal to the threshold value a, the selection unit 105 compares the shape value with the threshold value b that is the next larger than the threshold value a. When the selection unit 105 determines that the compared shape value is smaller than the threshold value b, the selection unit 105 determines that the shape of the wrinkle candidate associated with the shape value is a standard, and an eye corner pattern corresponding to the standard Is associated with the identifier of the candidate for wrinkle and added to the candidate for wrinkle data stored in the storage unit 107 for storage. When the selection unit 105 determines that the compared shape value is equal to or greater than the threshold value b, the selection unit 105 determines that the shape of the eyelid candidate associated with the shape value is a round eye, The eye pattern identifier is associated with the eyelid candidate and added to the eyelid candidate data stored in the storage unit 107 and stored.

  As described above, the selection unit 105 selects the eye corner-eye pattern stored in the storage unit 107 according to the shape value of the eyelid candidate, and thus, according to the shape value serving as an index of the shape of the eyelid candidate as described above. Thus, it is possible to select an appropriate eye-eye-eye pattern for each eyelid candidate indicated by the eyelid candidate data, from among the eye-eye corner patterns determined in advance according to the shape of the eyelid candidate.

  When the identifier of the eye corner eye pattern is added to the eyelid candidate data stored in the storage unit 107, the specifying unit 106 adds the eye corner and the eyeball to each eyelid candidate indicated by the eyelid candidate data stored in the storage unit 107. Specify the position of.

  More specifically, when specifying the eye corners and eye positions of the eyelid candidates, the specifying unit 106 stores each of the eye corner patterns and the eye patterns of the eye corner eye patterns associated with the eyelid candidates in the storage unit 107. The image is placed on the face image. More specifically, the identifying unit 106 sets the eyelid side end pixel position of the lower eyelid candidate of the identifier associated with each eyelid candidate by the eyelid candidate data stored in the storage unit 107 in the position of the eyelid. Processing is performed to match the reference points of the eye corner pattern of the eye corner eye pattern of the identifier associated with the candidate and to superimpose the eye corner pattern on the face image. In addition, the specifying unit 106 associates the eyelid candidate with the eyelid candidate corresponding to the eyelid end point pixel position of the lower eyelid candidate associated with each eyelid candidate by the eyelid candidate data stored in the storage unit 107. A process is performed in which the eye pattern reference point of the eye corner pattern of the identifier is matched and the eye pattern is superimposed on the face image. Note that the image in the face image in which the eye corner pattern and the eye pattern are superimposed by the specifying unit 106 is an image that is considered to indicate the eye corner and the eye head for each eyelid candidate indicated by the eyelid candidate data. Become.

  Note that the position of the lower eyelid candidate pixel stored in the storage unit 107 is the position on the contour enhanced image, but in this embodiment, the number of pixels of the contour enhanced image and the face image enhances the contour. Since it does not change even if the process is performed, the position of the lower eyelid candidate pixel on the contour emphasized image directly corresponds to the position on the face image.

  More specifically, in the present embodiment, when generating an edge-enhanced image, the face image indicated by the face image data is directly filtered, and the number of pixels does not change between the images. Therefore, the pixel position in the contour enhanced image and the pixel position in the face image correspond to each other as they are. However, in order to reduce the processing burden when generating the contour-enhanced image, when the process of generating the contour-enhanced image after thinning out the pixels of the face image indicated by the face image data, The position of the pixel does not correspond directly to the position of the pixel in the face image. Therefore, based on the position of the eyelid side end point of the lower eyelid candidate and the position of the eye side end point detected in the contour-enhanced image generated after the identification unit 106 thins out the pixels of the face image, When the eye corner pattern and the eye pattern are superimposed on each other, the position of the pixel at the end of the eye corner and the position of the pixel at the end of the eye are converted into the position of the pixel on the face image to correspond to each other. There must be.

  When the identifying unit 106 performs processing to superimpose the eye corner pattern and the eye pattern on the face image, the identifying unit 106 on the face image corresponding to the position of the pixel on each of the right eyelid side and the eye corner side of the lower eyelid candidate In the position, the right eye corner pattern and the eye head pattern described with reference to FIGS. 8A to 8C are superimposed as they are, and the position of the pixel at each of the left and lower eyelid candidates on the left and right corners of the eyelid candidate 8A to 8C, the right eye corner pattern described with reference to FIGS. 8A to 8C and the pattern obtained by inverting the right and left of the eye pattern are superimposed on the position on the face image corresponding to. In another embodiment, the left and right eye corner patterns are preliminarily stored in the storage unit 107, and correspond to the positions of the respective end point pixels on the left and right eyelid candidates. The right and left eye corner patterns may be superimposed on positions on the face image.

  When the eye part pattern and the eye pattern are superimposed on the face image, the specifying unit 106 superimposes the image of the area where the eye part pattern and the eye pattern are superimposed on the face image, the superimposed eye part pattern, and the eye pattern. A pattern matching process is performed to calculate the degree of similarity with the image defined as. The identification unit 106 performs pattern matching processing for each of the eyelid candidates indicated by the eyelid candidate data stored in the storage unit 107, and then the sum of the similarity between the corners of the eyes and the eyes is the highest in each of the left and right sides of the face image. Identify large heel candidates as left and right heels, respectively. The pattern matching process performed by the specifying unit 106 may use any known method.

  The above is the description of the schematic configuration of the eye detection device 1 according to the present embodiment. Next, processing of the eye detection apparatus 1 according to the present embodiment will be described with reference to the flowchart shown in FIG. In the process shown in the flowchart of FIG. 9, as described above, the control unit that has read a predetermined program from the storage unit 107 detects the detection unit 102, the combination unit 103, the shape value calculation unit 104, and the selection unit 105. And, it is executed by the control unit by appropriately processing and functioning as the functional configuration of the specifying unit 106.

  In step S <b> 101, the control unit, as the detection unit 102, determines whether face image data has been generated by the imaging unit 101. When the control unit determines in step S101 that face image data has been generated, the control unit proceeds to step S102. On the other hand, when the control unit determines in step S101 that face image data has not been generated, the control unit repeats the process of step S101.

  In step S <b> 102, the control unit acquires, as the detection unit 102, the face image data generated by the imaging unit 101, stores it in the storage unit 107, and converts the face image data indicated by the face image data stored in the storage unit 107. On the other hand, the above-described filter processing for emphasizing the contour is performed to generate a contour-enhanced image, and the generated contour-enhanced image is stored in the storage unit 107. When the control section completes the process step of step S102, the control section advances the process to step S103.

  In step S <b> 103, the control unit detects the above-described upper eyelid candidate by performing pattern matching processing or the like based on the contour-enhanced image stored in the storage unit 107 as the detection unit 102. When the control section completes the process step of step S103, the control section advances the process to step S104.

  In step S <b> 104, the control unit detects the above-described lower eyelid candidate using a method such as pattern matching based on the contour-enhanced image stored in the storage unit 107 as the detection unit 102. When the control section completes the process step of step S104, the control section advances the process to step S105.

  In step S <b> 105, the control unit generates the above-described upper and lower eyelid candidate data as the detection unit 102. More specifically, in step S105, the control unit determines, for each upper eyelid candidate detected in step S103, each position in the contour-enhanced image of all pixels indicating one upper eyelid candidate, and whether the upper eyelid candidate is a face. Upper eyelid candidate data is generated by associating information indicating whether it exists on the left or right with an identifier for identifying each upper eyelid candidate. Further, in step S105, the control unit determines, for each lower eyelid candidate detected in step S104, each position in the contour-enhanced image of all pixels indicating one lower eyelid candidate, and whether the lower eyelid candidate is on the left or right side of the face. The lower eyelid candidate data is generated by associating information indicating whether the lower eyelid candidates exist with identifiers for identifying the lower eyelid candidates. Further, when generating the upper eyelid candidate data and the lower eyelid candidate data in step S105, the control unit generates data indicating the generated upper eyelid candidate data and the lower eyelid candidate data as upper and lower eyelid candidate data. And stored in the storage unit 107. When the control section completes the process step of step S105, the control section advances the process to step S106.

  In step S106, the control unit combines the upper eyelid candidate and the lower eyelid candidate indicated by the upper and lower eyelid candidate data stored in the storage unit 107 as the combination unit 103, as described above, respectively as the eyelid candidates, The cocoon candidate data indicating the combined cocoon candidates is generated, and the generated cocoon candidate data is stored in the storage unit 107. When the control section completes the process step of step S106, the control section advances the process to step S107.

  In step S107, the control unit calculates a shape value for each wrinkle candidate indicated by the wrinkle candidate data stored in the storage unit 107 as the shape value calculating unit 104, and the calculated shape value is indicated by the wrinkle candidate data. In association with the identifiers of the wrinkle candidates, the wrinkle candidate data stored in the storage unit 107 are added and stored. When the control section completes the process step of step S107, the control section advances the process to step S108.

  In step S108, the control unit stores the selection unit 105 in the storage unit 107 according to the shape value of the wrinkle candidate as described above for each wrinkle candidate indicated by the wrinkle candidate data stored in the storage unit 107. The selected corner area pattern is associated with the eyelid candidate identifier and added to the eyelid candidate data stored in the storage unit 107 and stored. When the control section completes the process step of step S108, the control section advances the process to step S109.

  In step S <b> 109, the control unit, as the specifying unit 106, for each eyelid candidate indicated by the eyelid candidate data stored in the storage unit 107, the end point on the eye corner side of the lower eyelid candidate pixel combined as the eyelid candidate. As described above, the eye corner pattern and the eye pattern of the eye corner eye pattern of the identifier respectively associated with the position of the face corresponding to the position of the pixel and the position of the pixel at the end point on the eye side, respectively. The pattern matching process is performed by superimposing them, and the left and right wrinkles in the face image are specified. When the control section completes the process step of step S109, the control section returns the process to step S101.

  The above is the description of the eye detection device 1 according to the present embodiment. According to the eye detection device 1 according to the present embodiment, an appropriate eye corner pattern used for the pattern matching process can be selected according to the shape of the eyelid candidate detected based on the contour-enhanced image. Even if there is a difference, it is possible to accurately identify wrinkles.

  Further, according to the eye detection device 1 according to the present embodiment, it is possible to accurately store even a limited number of eye corner patterns in the storage unit 107, even if there are individual differences in human eye shapes. A bag can be identified, and the amount of information stored in the storage unit 107 can be reduced.

  In the description of the first embodiment, the shape value calculation unit 104 exists as the shape value at the highest position in the vertical direction among pixels indicating one upper eyelid candidate associated with the eyelid candidate data. At least one of the position of the vertex pixel ut and the end mj on the outer corner of the pixel indicating the lower eyelid candidate of the identifier associated as the eyelid candidate with the identifier of the upper eyelid candidate and the end mg on the eye side It was assumed that the slope of a straight line connecting the end points was calculated as a shape value. However, the shape value calculation unit 104 according to another embodiment may calculate the opening degree of the eyelid candidate as the shape value for each eyelid candidate indicated by the eyelid candidate data.

  FIG. 10 is a diagram illustrating the opening degree of the wrinkle candidate calculated by the shape value calculation unit 104 according to another embodiment. For the convenience of explanation, FIG. 10 shows the upper eyelid candidate um2 and the lower eyelid candidate sm1 enlarged among the upper eyelid candidate and the lower eyelid candidate shown in FIG. When the shape value calculating unit 104 calculates the opening degree of the eyelid candidate as the shape value of one eyelid candidate, as illustrated in FIG. 10, the pixel value indicating one upper eyelid candidate associated with the eyelid candidate data is displayed. The pixel ut at the vertex in the highest position in the vertical direction and the lowest pixel in the vertical direction among the pixels indicating the lower eyelid candidate of the identifier associated as the eyelid candidate with the identifier of the upper eyelid candidate You may calculate the length of the straight line which connects the pixel dt of the existing vertex as an opening degree.

  11A to 11C show the shape of the eyelid candidate combining the upper eyelid candidate um2 and the lower eyelid candidate sm1 detected by the detection unit 102, respectively, a shape close to a fine shape, a shape close to a standard shape, and a round shape. It is a figure which shows an example of the opening degree kd when it is a near shape, respectively. As shown in FIG. 11A to FIG. 11C, the opening degree kd calculated as the shape value by the shape value calculation unit 104 increases in the order of fine, standard, and round, respectively, and similarly to the inclination described above, It becomes an index of the eye shape of the eyelid candidate combining the upper eyelid candidate and the lower eyelid candidate.

  Even when the shape value calculation unit 104 calculates the opening degree as the shape value, the selection unit 105 compares the shape value with the threshold value as described above, as in the case where the inclination is calculated as the shape value. According to the shape value that is an index of the shape of the eyelid candidate, an appropriate eye corner eye pattern for each eyelid candidate indicated by the eyelid candidate data is selected from among the eye corner eye patterns previously determined according to the shape of the eyelid candidate. Can be selected.

  In addition, the shape value calculation unit 104 according to another embodiment may calculate, as the shape value, the curvature of the circle closest to the shape of the upper eyelid candidate combined as the eyelid candidate for each eyelid candidate.

  FIG. 12 is a diagram illustrating the curvature of a wrinkle candidate calculated by the shape value calculation unit 104 according to another embodiment. For the convenience of explanation, FIG. 12 shows the upper eyelid candidate um2 and the lower eyelid candidate sm1 among the upper eyelid candidate and the lower eyelid candidate shown in FIG. When the shape value calculation unit 104 calculates the curvature of the eyelid candidate as the shape value of one eyelid candidate, as shown in FIG. 12, the shape of the pixel indicating one upper eyelid candidate associated with the eyelid candidate data It is also possible to calculate a circle having a radius R closest to, and calculate the curvature of the calculated circle as a shape value.

  FIG. 13A to FIG. 13C show that the shape of the eyelid candidate combining the upper eyelid candidate um2 and the lower eyelid candidate sm1 detected by the detection unit 102 is close to the fine shape, the shape close to the standard shape, and the round shape, respectively. It is a figure which respectively shows an example of the circle | round | yen of the radius R which calculates a curvature when it is a near shape. As shown in FIGS. 13A to 13C, the curvature calculated as the shape value by the shape value calculation unit 104 is such that the radius R of the circle decreases in the order of fine, standard, and round, respectively. It becomes larger in the order of standard and rounded eyes, and becomes an index of the eye shape of the eyelid candidate combining the upper eyelid candidate and the lower eyelid candidate in the same manner as the inclination described above.

  Even when the shape value calculation unit 104 calculates the curvature as the shape value, the selection unit 105 that determines the threshold value as described above compares the shape value with the threshold value as described above, and thereby the inclination is obtained as the shape value. As in the case of the calculation, the eyelid candidate indicated by the eyelid candidate data is selected from the eye corner eye patterns determined in advance according to the shape of the eyelid candidate according to the shape value serving as an index of the shape of the eyelid candidate. An appropriate eye corner pattern can be selected for each.

  Further, in the eye detection device 1 according to another embodiment, among the processing shown in the flowchart of FIG. 9, the processing from step S101 to step S108, that is, the processing until the selection of the eye corner pattern is performed, for example, on the vehicle. It may be performed only when the illuminance of the ambient light detected by the attached illuminance sensor (conlight sensor) is less than a predetermined threshold value. Then, once the eye corner pattern is selected, as long as the driver does not change, the process from step S101 to step S109 in which the process of the selection unit 105 is omitted may be specified. The illuminance sensor attached to the vehicle is, for example, an illuminance that is installed on an instrument panel in the back direction of the instrument panel indicating the speed of the vehicle as shown in FIG. 3 and detects the illuminance of ambient light around the vehicle It may be a sensor. Based on facial images that are not affected by ambient light, including sunlight, by selecting the corner-eye pattern only when the ambient light illuminance detected by the illuminance sensor is below a predetermined threshold Thus, after more accurately selecting the outer corner of the eye pattern, it is possible to identify the eyelid with the more accurate outer corner of the eye pattern.

  Further, in the eye detection device 1 according to another embodiment, a timepiece such as a clock or timer that detects the current time is used, for example, only when the time indicated by the timepiece indicates nighttime. Of the processes shown in the flowchart of FIG. 9, the processes from step S101 to step S108, that is, the process up to the selection of the corner area pattern may be performed. Then, once the eye corner pattern is selected, as long as the driver does not change, the process from step S101 to step S109 in which the process of the selection unit 105 is omitted may be specified. By selecting the upper corner of the eye pattern only when it is at night, a more accurate outer corner of the eye after selecting the outer eye pattern more accurately based on face images that are not affected by ambient light including sunlight. You can identify cocoons with patterns.

  Further, in the eye detection device 1 according to another embodiment, only when the wiper mounted on the vehicle is operating, the processing from step S101 to step S108 in the processing shown in the flowchart of FIG. The processing up to the selection of the corner area pattern may be performed. Then, once the eye corner pattern is selected, as long as the driver does not change, the process from step S101 to step S109 in which the process of the selection unit 105 is omitted may be specified. By selecting the outer corner pattern only when the wiper is in operation, it is more accurate after selecting the outer corner pattern on the basis of a face image that is rainy and unaffected by sunlight. The eyelids can be identified by the pattern of the eyes.

  The eye detection device 1 according to the present invention can be applied to a system generally called a driver monitor system that monitors the state of a driver mounted on a moving body such as a vehicle in recent years. it can. More specifically, in recent years, a moving body such as a vehicle has been detected to detect the opening degree of the heel and the angle of the face based on the driver's face image, and perform a doze driving, a side-viewing driving, etc. It is equipped with a system that issues a warning to the driver when judged. In such a system, when detecting the opening degree of the driver's eyelids, the angle of the face, etc., only the opening degree of the eyelid candidate whose similarity is calculated in the present invention is calculated. There is a case where a method is adopted in which the position of the eyelid candidate determined to have blinked based on the calculated opening degree of the eyelid candidate is determined as the position in the face image of the driver's eye. Then, based on the position of the driver's eyes in the face image determined in this way, the face angle may be detected by the above-described system. According to the present invention, it is possible to improve the accuracy when calculating the similarity in such a system.

  Also, as described above, only when the ambient light illuminance is less than the threshold value, only at night, and only when the wiper is in operation, steps S101 through S101 are included in the processing shown in the flowchart of FIG. When the present invention is applied to the above-described system in another embodiment in which the processing up to S108, that is, the processing up to the selection of the eye corner pattern is performed, once the system tries to identify the eyelid, However, there is a possibility that a corner pattern is not selected. In such a case, the eyelid may be specified using a corner-eye pattern that is set in advance as a default.

  Although the present invention has been described in detail above, the above description is merely an example of the present invention in all respects and is not intended to limit the scope thereof. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention.

  ADVANTAGE OF THE INVENTION According to this invention, the eye detection apparatus which can detect especially an eye corner and an eye head especially in a human face can be provided, For example, it can utilize for the eye detection apparatus etc. which are mounted in moving bodies, such as a motor vehicle.

DESCRIPTION OF SYMBOLS 1 Eye detection apparatus 101 Imaging part 102 Detection part 103 Combining part 104 Shape value calculation part 105 Selection part 106 Identification part 107 Storage part

Claims (11)

Imaging means for imaging an image including the face of the subject as a face image;
Detection means for detecting each candidate combination of pixels indicating the upper eyelid and lower eyelid of the subject as the eyelid candidates based on the face image imaged by the imaging means;
A memory for storing only two or more eye corner patterns of a predetermined image for calculating a similarity between the eye corners for each eyelid candidate detected by the detection means in the face image and an image indicating the eye head Means,
Based on the shape value indicating the shape of each eyelid candidate detected by the detecting means, one eye corner eye pattern is selected from the eye corner eye pattern stored in the storage means and associated with each eyelid candidate. A selection means;
The similarity between the image showing the eye corners and the eyes of each eyelid candidate detected by the detection means in the face image, and the image of the eye corners eye pattern selected for each eyelid candidate by the selection means An eye detection apparatus comprising: a specifying unit that calculates each of. Among the pixels indicating the upper eyelid candidate and at least one end point of the eye corner side and the eye side of the pixel indicating the lower eyelid candidate combined as the eyelid candidate detected by the detecting means Further comprising an inclination detection means for detecting an inclination of a line connecting a pixel at the highest vertex in the vertical direction in the face image as the shape value;
The eye detection device according to claim 1, wherein the selection unit selects the eye corner pattern based on the inclination detected by the inclination detection unit.   The inclination detection means uses, as the shape value, an inclination of a line connecting the end point on the outer corner side of the pixel indicating the lower eyelid candidate combined with the eyelid candidate detected as the eyelid candidate detected by the detection means, and the vertex pixel. The eye detection device according to claim 2, which detects the eye detection device.   The inclination detection means uses, as the shape value, an inclination of a line connecting an end point on the top side of a pixel indicating the lower eyelid candidate combined with the eyelid candidate detected as the eyelid candidate detected by the detection means, and the vertex pixel. The eye detection device according to claim 2, which detects the eye detection device.   The inclination detection unit connects each of the pixel at the corner of the eye corner and the pixel at the corner of the eye of the lower eyelid candidate combined with each of the candidates for the eyelid detected by the detection unit and the pixel at the vertex. The eye detection device according to claim 2, wherein an inclination of a line is detected as the shape value. Opening degree detecting means for detecting the opening degree of the eyelid as the shape value for each eyelid candidate detected by the detecting means;
The eye detection apparatus according to claim 1, wherein the selection unit selects the eye corner pattern based on the opening detected by the opening detection unit. Curvature detection means for detecting, as the shape value, the curvature of the upper eyelid candidate pixel combined as the eyelid candidate for each of the eyelid candidates detected by the detection means,
The eye detection apparatus according to claim 1, wherein the selection unit selects the eye corner pattern based on the curvature detected by the curvature detection unit. Further comprising illuminance detection means for detecting the illuminance of the ambient light irradiated to the subject,
The eye detection apparatus according to claim 1, wherein the selection unit selects the eye corner pattern when the illuminance of the ambient light detected by the illuminance detection unit is less than a predetermined threshold value. It further comprises time detection means for detecting the current time,
The eye detection device according to claim 1, wherein the selection unit selects the eye corner pattern when the time detected by the time detection unit indicates night. Mounted on the vehicle,
An operation detecting means for determining whether or not the vehicle wiper is operating;
The eye detection device according to claim 1, wherein the selection unit selects the eye corner pattern when it is determined by the operation detection unit that a wiper is operating. An imaging step of capturing an image including the face of the subject as a face image;
A detection step of detecting each candidate combination of pixels indicating the upper eyelid and the lower eyelid of the target person as an eyelid candidate based on the face image imaged in the imaging step;
Among the two or more eye corner eye patterns of a predetermined image for calculating the degree of similarity between the eye corner for each eyelid candidate detected in the detection step in the face image and an image indicating the eye head, A selection step of selecting and correlating one eye corner pattern for each eyelid candidate based on a shape value indicating the shape of each eyelid candidate;
The similarity between the image showing the eye corners and the eyes for each eyelid candidate detected in the detection step in the face image, and the image of the eye corners eye pattern selected for each eyelid candidate in the selection step And a specific step of calculating each of the eye detection methods.

Images