JP2010092180A - Eyelid likelihood arithmetic device and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To calculate eyelid likelihood with satisfactory accuracy even when a captured image includes a face in profile. <P>SOLUTION: Sums of a value (w<SB>right</SB>×λ<SB>r_eyelid</SB>) acquired by applying weight w<SB>right</SB>corresponding to the right eyelid to the right eyelid likelihood of the right comparison eyelid model in the combination of the right and left comparison eyelid models whose eyelid openings are the same and a value (w<SB>left</SB>×λ<SB>l_eyelid</SB>) acquired by applying weight w<SB>left</SB>corresponding to the left eyelid to the left eyelid likelihood of the left comparison eyelid model for each combination as the right and left eyelid likelihood λ<SB>eyelid</SB>, wherein the weight corresponding to the eyelid likelihood of the eyelid far from a camera 12 is set smaller than the weight corresponding to the eyelid likelihood of the eyelid closer to the camera according to a face direction angle θ (124), and the maximum right and left eyelid likelihood λ<SB>eyelid_max</SB>among the plurality of calculated right and left eyelid likelihood λ<SB>eyelid</SB>is calculated as the whole eyelid likelihood λ<SB>eyelid_all</SB>(128). <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a heel likelihood calculation device and a program.

2. Description of the Related Art Conventionally, a technique for tracking a black eye by condensation using an eye model for detecting a moving direction of a line of sight is known (see, for example, Non-Patent Document 1). In the technique described in Non-Patent Document 1, at least one model parameter of the right heel shape model (瞼 model) is changed a plurality of times and image coordinates on which the shape of the right heel shape model is projected On the (two-dimensional coordinates), the likelihood of the right eyelid is estimated based on the image data of the image captured by the imaging means such as a camera that captures the image and outputs the image data. Similarly, at least one model parameter of the left heel shape model is changed a plurality of times, and the image captured by the imaging unit is projected on the image coordinates on which the shape of the left heel shape model is projected. The likelihood of the left eyelid is estimated based on the image data. Then, the sum of the likelihood of the estimated right eyelid and the likelihood of the left eyelid is estimated as the overall likelihood of the eyelid.
Yosuke Kitagawa, Takekazu Kato, Motoshi Kure, Toshikazu Wada “Kurome Tracking with Condensation Using Eye Models” Image Recognition and Understanding Symposium (MIRU2005) Interactive Session, pp.1343--1350, (2005).

  However, in the technique described in Non-Patent Document 1, as shown in FIG. 7A, the situation where the face to be imaged does not face sideways with respect to the camera 90 that captured the image, that is, when the image was captured The likelihood of the entire eyelid is estimated on the assumption that the distance between the position of the camera 90 and the right eyelid 92 and the distance between the position of the camera 90 and the left eyelid 94 are equal, and FIG. As shown in FIG. 4, the situation where the face to be imaged faces sideways with respect to the camera 90 that captured the image, that is, the distance between the position of the camera 90 and the right eyelid 92 when the image is captured, and the camera 90 In the situation where the distance between the position of and the left eyelid 94 is different, the accuracy of the image of the region including the farther eyelid is farther from the camera 90 (the left eyelid 94 in the example of FIG. 7B). Is not as good as the accuracy of the image of the area containing the wrinkles closer to Ku, this because of the eyelids likelihood of far more of the eyelids can not be estimated with good accuracy, and thus the entire eyelid likelihood can not be estimated with good accuracy, there is a problem in that.

  The present invention has been made to solve the above problem, and even when the image of the face to be photographed included in the image faces sideways, the likelihood of wrinkles can be improved with better accuracy. It is an object of the present invention to provide a wrinkle likelihood calculation device and a program that can be calculated.

  In order to achieve the above object, a first heel likelihood calculating apparatus according to the present invention includes a photographing unit that captures an image including a face image and outputs image data, and the face based on the image data. Detecting means for detecting the face orientation angle, a starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left upper eyelid by a plurality of second model parameters And a storage means for storing a port model defining the shape of the left lower eyelid, and creating a plurality of right comparison eyelid models by changing at least one model parameter of the first model parameter a plurality of times A creating means for creating a plurality of left comparison kites by changing at least one model parameter of the second model parameter a plurality of times, and a right comparison kit model A right heel likelihood indicating the degree of matching between the shape of the right upper eyelid and the shape of the right lower eyelid and the shape of the right upper eyelid and the shape of the right lower eyelid of the image photographed by the photographing means; and The left heel likelihood indicating the degree to which the shapes of the left upper heel and the left lower heel represented by the left comparative heel model match the shapes of the left upper heel and the left lower heel of the image. A first calculating means for calculating, and a starboard and a left upper eye including a right upper eyelid and a right lower eyelid of an image photographed by the photographing means according to a face orientation angle detected by the detecting means And the size corresponding to the likelihood of the eyelid farther from the position where the image was photographed by the photographing means among the left eyelids including the left lower eyelid is the size corresponding to the eyelid likelihood of the near eyelid The weight corresponding to the right eyelid out of the weights corresponding to the left and right eyelids determined to be smaller than A value weighted to the right heel likelihood of the right comparative heel model in the combination of the right comparative heel model and the left comparative heel model having the same opening, and a weight corresponding to each of the left and right heel models A second calculation for performing a plurality of calculations for each combination as a right and left heel likelihood in the combination of a weight corresponding to the left heel and a value obtained by weighting the left heel likelihood of the left comparison heel model Means, and third computing means for computing the largest left and right eyelid likelihood as the overall eyelid likelihood among the left and right eyelid likelihood for each combination calculated by the second computing means. It is configured.

  According to the eyelid likelihood calculation apparatus of the present invention, when a face image included in an image captured by an imaging unit, such as a camera, is facing sideways (that is, an image of a face to be captured included in the image is Even if it is facing sideways), the size corresponding to the eyelid likelihood of the eyelid farther to the position where the image was taken is smaller than the size corresponding to the eyelid likelihood of the near eyelid. The weight corresponding to the right eyelid among the weights corresponding to the left and right eyelids determined in the above is the right comparison eyelid model with the same eyelid opening degree and the right comparison eyelid model in the combination of the left eyelid eyelid model. The sum of the value weighted to the right eyelid likelihood and the weight corresponding to the left eyelid among the weights corresponding to the left and right eyelids, and the value weighted to the left eyelid likelihood of the left comparative eyelid model Multiple calculations for each combination as left and right heel likelihood in this combination Of the left and right wing likelihoods for each of the combinations calculated, the largest left and right heel likelihoods are calculated as the overall 瞼 likelihood, so the 瞼 likelihood of the cocoon with the lower accuracy relative to the overall 瞼 likelihood , And the overall likelihood of wrinkles can be calculated with better accuracy.

  In order to achieve the above object, the second heel likelihood computing device of the present invention is based on an image capturing unit that captures an image including a face image and outputs image data, and the image data. The distance to the right eye and the left eye of the face, the distance to the right ear of the face and the distance to the left ear, or the distance from the right eye to the right eye and the left eye to the left eye A detecting means for detecting a distance to the eyes, a starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left upper eyelid by a plurality of second model parameters And a storage means for storing a port model defining the shape of the left lower eyelid, and creating a plurality of right comparison eyelid models by changing at least one model parameter of the first model parameter a plurality of times Creating means for creating a plurality of left comparison kites by changing at least one model parameter of the second model parameter a plurality of times, and a right upper kite and a right model represented by the right comparison kit Expressed by the right eyelid likelihood indicating the degree of matching between the shape of the lower eyelid and the shape of the right upper eyelid and the shape of the right lower eyelid of the image photographed by the photographing means, and the left comparative eyelid model First computing means for computing a left heel likelihood indicating a degree of matching between the shapes of the left upper heel and the left lower heel and the shapes of the left upper heel and the left lower heel of the image; The distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear detected by the detecting means, or the distance from the right eye to the right eye and the left eye The eyes of the left eye Of the image taken by the photographing means according to the distance at the right starboard including the right upper eyelid and the right lower eyelid and the left upper eyelet and the left starboard including the left lower eyelid by the photographing means. Corresponding to each of the left and right eyelids, the size corresponding to the eyelid likelihood of the eyelid farther from the position where the image was taken is smaller than the size corresponding to the eyelid likelihood of the near eyelid Of the weights, the weight corresponding to the right eyelid is weighted to the right eyelid likelihood of the right comparison eyelid model and the right comparison eyelid model in the combination of the right eyelid opening model and the left eyelid eyelid model. The sum of the value and the weight corresponding to the left eyelid among the weights corresponding to each of the left and right eyelids, and the value obtained by weighting the left eyelid likelihood of the left comparative eyelid model, A second calculating means for calculating a plurality of likelihoods for each combination; Among the left and right eyelid likelihoods for each combination calculated by the second computing means, the third arithmetic means for computing the largest left and right eyelid likelihood as the overall eyelid likelihood is configured. .

  According to the eyelid likelihood calculation apparatus of the present invention, when a face image included in an image captured by an imaging unit, such as a stereo camera, is facing sideways (that is, an image of a face to be captured included in the image). Even when the image is taken sideways), the size corresponding to the eyelid likelihood of the eyelid farther from the position where the image was taken is smaller than the size corresponding to the eyelid likelihood of the near eyelid. Of the weights corresponding to each of the left and right eyelids determined as described above, the weight corresponding to the right eyelid is set to the right comparison eyelid model in the combination of the right eyelid opening model and the left eyelid eyelid model having the same eyelid opening. The weight of the right heel likelihood and the weight corresponding to the left heel among the weights corresponding to the left and right heels, and the value weighted to the left heel likelihood of the left comparative heel model For each combination as the left and right heel likelihood in this combination Of the left and right heel likelihoods for each combination that has been arithmetically calculated, the largest left and right heel likelihood is calculated as the overall heel likelihood, so the accuracy of the heel with the lower accuracy relative to the overall heel likelihood The ratio of the heel likelihood becomes smaller, so that the overall heel likelihood can be calculated with better accuracy.

  In order to achieve the above object, the third heel likelihood computing device of the present invention is based on an image capturing unit that captures an image including a face image and outputs image data, and the image data. First computing means for computing the contrast of the right eye region of the face image and the contrast of the left eye region of the face image, or the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image And a starboard model in which the shape of the right upper eyelid and the right lower eyelid are determined by a plurality of first model parameters, and a shape of the left upper eyelid and the left armpit are determined by a plurality of second model parameters. A storage means for storing the left port model, and at least one model parameter of the first model parameter is changed a plurality of times to create a plurality of right comparison model models, and the second model Creating means for creating a plurality of left comparison eyelid models by changing at least one model parameter of the parameter a plurality of times, and each shape of the right upper eyelid and the right lower eyelet represented by the right comparison eyelid model And the left upper eye represented by the left comparative eye model indicating the degree to which the shape of the right upper eyelid and the right lower eyelid of the image photographed by the photographing means coincide with each other. And a second computing means for computing a left heel likelihood indicating a degree of matching between each shape of the left lower eyelid and each shape of the left upper eyelid and the left lower eyelid of the image, According to the contrast of the right eye region and the contrast of the left eye region calculated by the calculating unit, or the edge strength of the right eye region and the edge strength of the left eye region, the upper right corner of the image photographed by the photographing unit and Right Of the starboard including the heel and the left upper heel and the left heel including the left lower heel, the size corresponding to the 瞼 likelihood of the heel with the smaller contrast or edge strength is the one with the larger contrast or edge strength. Of the weights corresponding to the left and right eyelids, the weight corresponding to the right eyelid is set to be smaller than the size corresponding to the eyelid likelihood of the eyelid, and the right comparative eyelid model having the same eyelid opening degree. And a value weighted to the right heel likelihood of the right comparative heel model in the combination of the left comparative heel model and a weight corresponding to the left heel among the weights corresponding to each of the left and right heels, A third computing means for computing a plurality of combinations for each combination as a right and left likelihood for the combination, and a plurality of computations by the third computing means. Left for each combination Among the right wing likelihoods, a fourth calculation means for calculating the largest left and right heel likelihoods as the overall heel likelihood is configured.

  According to the eyelid likelihood calculation apparatus of the present invention, when a face image included in an image captured by an imaging unit, such as a stereo camera, is facing sideways (that is, an image of a face to be captured included in the image). The size corresponding to the likelihood of the eyelid with the smaller contrast or edge strength is the size corresponding to the likelihood of the eyelid with the higher contrast or edge strength. The weight corresponding to each of the left and right eyelids determined to be smaller than that, that is, the magnitude corresponding to the eyelid likelihood of the eyelid farther from the position where the image was taken, is the eyelid likelihood of the near eyelid. Of the weights corresponding to each of the left and right eyelids determined to be smaller than the corresponding size, the weight corresponding to the right eyelid is a combination of the right comparison eyelid model and the left comparison eyelid model having the same eyelid opening. Right in A value weighted to the right eye likelihood of the comparative eyelid model and a weight corresponding to the left eyelid among the weights corresponding to each of the left and right eyelids, weighted to the left eyelid likelihood of the left eyelid model Is calculated for each combination as the left and right heel likelihoods in this combination, and the largest left and right heel likelihood is calculated as the overall heel likelihood for each combination. Therefore, the ratio of the heel likelihood of the cocoon with the lower accuracy relative to the overall heel likelihood becomes smaller, whereby the overall heel likelihood can be calculated with better accuracy.

  In order to achieve the above object, the fourth heel likelihood calculation device of the present invention is configured to capture an image including a face image and output image data, and based on the image data, First detection means for detecting a face orientation angle of the face, second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data, and the face of the face based on the image data Third detection means for detecting a distance to the right ear and a distance to the left ear, a distance from the right eye to the right eye based on the image data, and a distance from the left eye to the left eye Fourth detection means for detecting a distance, first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data, and based on the image data Processing means comprising at least one means of second computing means for computing the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image; and a right by a plurality of first model parameters Storage means for storing a starboard model that defines the shape of the upper arm and the lower arm of the right, and a port model that defines the shape of the left upper arm and the left armpit according to a plurality of second model parameters; Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison eyelid models, each shape of the right upper eyelid and right lower eyelid represented by the right comparison eyelid model, and the photographing means Therefore, the right upper eyelid likelihood indicating the degree of coincidence between the shape of the right upper eyelid and the right lower eyelid of the captured image, and the left upper eyelid and the lower left left represented by the left comparative eyelid model Third computing means for computing a left eyelid likelihood indicating a degree of matching between each shape of the eyelid and each shape of the left upper eyelid and the left lower eyelid of the image, the detected face orientation angle, The distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, the distance from the right eye to the right eye and the distance from the left eye to the left eye The right contrast of the right eye area and the contrast of the left eye area, and the edge intensity of the right eye area and the edge intensity of the left eye area calculated according to at least one of the calculated right eye area contrast and the left eye area. Right including heel and right lower heel The size corresponding to the heel likelihood of the heel farther from the position where the image was photographed by the photographing means out of the left heel including the heel, the left upper heel, and the left lower heel The weight corresponding to the right eyelid among the weights corresponding to the left and right eyelids determined to be smaller than the size corresponding to the likelihood, the right comparative eyelid model having the same eyelid opening and the left The right comparison 瞼 model of the right comparison 瞼 model in the combination of the right heel likelihood and the weight corresponding to the left heel among the weights corresponding to the left and right heels, the left comparison 瞼 model A fourth calculation means for performing a plurality of calculations for each combination as a left and right wrinkle likelihood in the combination, and a combination calculated by the fourth calculation means; The largest left of each left and right heel likelihood The eyelids likelihood is configured to include a fifth calculating means for calculating the overall eyelid likelihood.

  According to the eyelid likelihood calculation apparatus of the present invention, when a face image included in an image captured by an imaging unit, such as a stereo camera, is facing sideways (that is, an image of a face to be captured included in the image). Even when the image is taken sideways), the size corresponding to the eyelid likelihood of the eyelid farther from the position where the image was taken is smaller than the size corresponding to the eyelid likelihood of the near eyelid. Of the weights corresponding to each of the left and right eyelids determined as described above, the weight corresponding to the right eyelid is set to the right comparison eyelid model in the combination of the right eyelid opening model and the left eyelid eyelid model having the same eyelid opening. The weight of the right heel likelihood and the weight corresponding to the left heel among the weights corresponding to the left and right heels, and the value weighted to the left heel likelihood of the left comparative heel model For each combination as the left and right heel likelihood in this combination Of the left and right heel likelihoods for each combination that has been arithmetically calculated, the largest left and right heel likelihood is calculated as the overall heel likelihood, so the accuracy of the heel with the lower accuracy relative to the overall heel likelihood The ratio of the heel likelihood becomes smaller, so that the overall heel likelihood can be calculated with better accuracy.

  In order to achieve the above object, a fifth heel likelihood calculation device according to the present invention includes a photographing means for photographing an image including a face image and outputting image data, and the image data based on the image data. First detection means for detecting a face orientation angle of the face, second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data, and the face of the face based on the image data Third detection means for detecting a distance to the right ear and a distance to the left ear, a distance from the right eye to the right eye based on the image data, and a distance from the left eye to the left eye Fourth detection means for detecting a distance, first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data, and based on the image data Processing means comprising at least one means of second computing means for computing the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image; and a right by a plurality of first model parameters Storage means for storing a starboard model that defines the shape of the upper arm and the lower arm of the right, and a port model that defines the shape of the left upper arm and the left armpit according to a plurality of second model parameters; Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison eyelid models, each shape of the right upper eyelid and right lower eyelid represented by the right comparison eyelid model, and the photographing means Therefore, the right upper eyelid likelihood indicating the degree of coincidence between the shape of the right upper eyelid and the right lower eyelid of the captured image, and the left upper eyelid and the lower left left represented by the left comparative eyelid model Third computing means for computing a left eyelid likelihood indicating a degree of matching between each shape of the eyelid and each shape of the left upper eyelid and the left lower eyelid of the image, the detected face orientation angle, The distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, the distance from the right eye to the right eye and the distance from the left eye to the left eye The right contrast of the right eye area and the contrast of the left eye area, and the edge intensity of the right eye area and the edge intensity of the left eye area calculated according to at least one of the calculated right eye area contrast and the left eye area. Right including heel and right lower heel The size corresponding to the heel likelihood of the heel farther from the position where the image was photographed by the photographing means out of the left heel including the heel, the left upper heel, and the left lower heel The maximum right heel likelihood calculated by the third calculating means is set to a weight corresponding to the right wing among weights corresponding to each of the left and right wings, which is determined to be smaller than a size corresponding to the likelihood. Of the weights corresponding to the left and right eyelids, and the weights corresponding to the left eyelids are weighted to the maximum left eyelid likelihood computed by the third computing means. Of these, a selection means for selecting the larger value and a fourth calculation means for calculating the value selected by the selection means as the likelihood of the selected eyelid are included.

  According to the eyelid likelihood calculation apparatus of the present invention, when a face image included in an image captured by an imaging unit, such as a stereo camera, is facing sideways (that is, an image of a face to be captured included in the image). Even when the image is taken sideways), the size corresponding to the eyelid likelihood of the eyelid farther from the position where the image was taken is smaller than the size corresponding to the eyelid likelihood of the near eyelid. The weight corresponding to the right eyelid among the weights corresponding to each of the left and right eyelids determined as above, the value weighted to the maximum right eyelid likelihood, and the left of the weights corresponding to the left and right eyelids A larger value is selected from values obtained by weighting the weight corresponding to the heel to the maximum left heel likelihood, and the selected value is calculated as the heel likelihood of the selected heel. That is, the heel likelihood calculating device of the present invention selects a larger value among the weighted values, and calculates the selected value as the heel likelihood of the selected heel, which is better. The heel likelihood can be calculated with high accuracy.

  In addition, the heel likelihood calculating apparatus according to the present invention provides a value obtained by weighting a weight corresponding to the right heel to the maximum right heel likelihood, and a weight corresponding to the left heel is the maximum left When the weighted values of the heel likelihood are both equal to or greater than a predetermined reference value, the right heel opening represented by the right comparison heel model of the maximum right heel likelihood and the maximum left heel likelihood A fifth computing means for detecting a port side opening degree of the port side represented by the left comparison side model of the degree and calculating a ratio of the detected starboard side opening degree and the left side eyelid opening degree; The heel opening degree of the heel represented by the comparison heel model of the heel likelihood calculated by the computing means is detected, and the heel opening degree is not detected based on the detected heel opening degree and the ratio. A fifth detection means for detecting the eyelid opening degree of the eyelid having the lower eyelid likelihood by calculating the eyelid opening degree of the eyelid may be further included.Accordingly, when only one of the right comparative eyelid model and the left comparative eyelid model is highly reliable, the eyelid opening degree of the lower reliability eyelid can be accurately detected.

  In order to achieve the above object, a sixth heel likelihood computing device of the present invention includes a photographing means for photographing an image including a face image and outputting image data, and the image data based on the image data. First detection means for detecting a face orientation angle of the face, second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data, and the face of the face based on the image data Third detection means for detecting a distance to the right ear and a distance to the left ear, a distance from the right eye to the right eye based on the image data, and a distance from the left eye to the left eye Fourth detection means for detecting a distance, first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data, and based on the image data Processing means comprising at least one means of second computing means for computing the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image; and a right by a plurality of first model parameters Storage means for storing a starboard model that defines the shape of the upper arm and the lower arm of the right, and a port model that defines the shape of the left upper arm and the left armpit according to a plurality of second model parameters; Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparative eyelid models, and a right eye area that is an image of an eye area of the right eye included in the face image based on the image data; A first ratio that is a ratio of the number of pixels equal to or higher than a predetermined luminance to the number of pixels of the image, and a pixel that is equal to or higher than a predetermined luminance relative to the number of pixels of the left eye region image that is an image of the left eye region included in the face image A third calculation means for calculating a second ratio, which is a ratio of numbers, and the right comparison kit model only when the first ratio calculated by the third calculation means is less than or equal to a predetermined value. The right upper eyelid and the right lower eyelid are calculated and the right upper eyelid likelihood indicating the degree to which the shapes of the right upper eyelid and the right lower eyelid of the image photographed by the photographing means coincide with each other is calculated. And the shape of the left upper eyelid and the left lower eyelid represented by the left comparative eyelid model only when the second ratio calculated by the third calculating means is less than or equal to a predetermined value, Degree of matching of the shape of the upper left heel and left lower heel of the image A fourth calculation means for calculating the left eyelid likelihood indicating the detected face orientation angle, the distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, The distance from the right eye eye to the right eye eye and the distance from the left eye eye to the left eye eye, the calculated contrast of the right eye region and the left eye region, and the edge strength of the right eye region and the According to at least one of the edge intensities of the left eye region, a starboard including a right upper eyelid and a right lower eyelid and a left upper eyelid and a left lower eyelid of the image photographed by the photographing means. Of the left and right sides, the size corresponding to the heel likelihood of the farther heel is smaller than the size corresponding to the heel likelihood of the closer heel Each pair When the first ratio is less than or equal to a predetermined value and the second ratio is less than or equal to the predetermined value, the weight corresponding to the right eyelid is calculated by the fourth calculation means Of the values weighted to the maximum right heel likelihood and the weights corresponding to each of the left and right heels, the weight corresponding to the left heel is the maximum of the left If the larger value of the values weighted to the likelihood is selected and only one of the first ratio and the second ratio is less than or equal to a predetermined value, the fourth computing means Selection means for selecting a value obtained by weighting the maximum heel likelihood of the cocoon having the first ratio or the second ratio calculated by the formula (1) or less being a predetermined value or less with a weight corresponding to the cocoon; The value selected by the selection means is set to the likelihood of the selected one. It is configured to include a fifth computing means for computing a.

  The eyelid likelihood calculation apparatus according to the present invention calculates a first ratio (a high luminance pixel area area ratio of the right eye region) and a second ratio (a high luminance pixel area area ratio of the left eye region), and the first ratio is The right heel likelihood is calculated only when the value is equal to or less than the predetermined value, and the left heel likelihood is calculated only when the second ratio is equal to or less than the predetermined value. Then, the heel likelihood calculating apparatus according to the present invention, when the first ratio is equal to or less than a predetermined value and the second ratio is equal to or less than the predetermined value among the weights corresponding to the left and right heels, The value corresponding to the right heel likelihood is weighted to the right heel likelihood, and the weight corresponding to the left heel is weighted to the left heel likelihood and the larger value is selected, and the first ratio When only one of the second ratios is less than or equal to a predetermined value, the maximum likelihood of the heel whose ratio is less than or equal to the predetermined value (the maximum right likelihood or the maximum left likelihood) Degree), a value obtained by weighting the weight corresponding to this wrinkle is selected. As a result, it is possible to calculate the heel likelihood more easily without calculating the heel likelihood of the heel that cannot calculate the heel likelihood with high accuracy by a complicated operation.

  Further, according to the heel likelihood calculating device of the present invention, the larger value among the weighted values is selected, and the selected value is calculated as the heel likelihood of the selected heel. The heel likelihood can be calculated with better accuracy.

  In order to achieve the above object, a seventh heel likelihood calculating apparatus according to the present invention includes a photographing unit that captures an image including a face image and outputs image data, and the image data based on the image data. First detection means for detecting a face orientation angle of the face, second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data, and the face of the face based on the image data Third detection means for detecting a distance to the right ear and a distance to the left ear, a distance from the right eye to the right eye based on the image data, and a distance from the left eye to the left eye Fourth detection means for detecting a distance, first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data, and based on the image data Processing means comprising at least one means of second computing means for computing the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image; and a right by a plurality of first model parameters Storage means for storing a starboard model that defines the shape of the upper arm and the lower arm of the right, and a port model that defines the shape of the left upper arm and the left armpit according to a plurality of second model parameters; Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparative eyelid models, and a right eye area that is an image of an eye area of the right eye included in the face image based on the image data; A first ratio that is a ratio of the number of pixels equal to or higher than a predetermined luminance to the number of pixels of the image, and a pixel that is equal to or higher than a predetermined luminance relative to the number of pixels of the left eye region image that is an image of the left eye region included in the face image A third calculating means for calculating a second ratio, which is a ratio of numbers, and the right comparison model when the first ratio calculated by the third calculating means is a predetermined value or less. The right upper eyelid and the right lower eyelid, and the right upper eyelid likelihood indicating the degree of coincidence between the right upper eyelid and the right lower eyelid shape of the image photographed by the photographing means is calculated. When the first ratio is larger than a predetermined value, the right heel likelihood is calculated by setting the predetermined first value as the right heel likelihood, and is calculated by the third calculation means. When the second ratio is less than or equal to a predetermined value, Therefore, the left upper eyelid and left lower eyelid shapes represented by the left upper eyelid and left lower eyelid shapes of the image are calculated, and the left eyelid likelihood is calculated. A fourth computing means for computing the left eyelid likelihood by setting the predetermined first value as the left eyelid likelihood when the ratio of 2 is larger than a predetermined value; and a detected face orientation angle; The distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, the distance from the right eye to the right eye, and the left eye to the left eye According to at least one of the distance, the calculated contrast of the right eye region and the contrast of the left eye region, and the edge strength of the right eye region and the edge strength of the left eye region, the right side of the image photographed by the photographing unit Starboard including upper arm and lower arm The size corresponding to the likelihood of the eyelid farther from the position where the image was taken by the photographing means among the left eyelid including the left upper eyelid and the left lower eyelid is the likelihood of the closer eyelid Among the weights corresponding to each of the left and right eyelids so as to be smaller than the size corresponding to the degree, the weight corresponding to the right eyelid is the maximum right eyelid likelihood calculated by the fourth arithmetic means. Of the weights corresponding to the left and right eyelids, the weight corresponding to the left eyelid is weighted to the maximum left eyelid likelihood computed by the fourth computing means. Selection means for selecting the larger one of the values, and fifth calculation means for calculating the value selected by the selection means as the 瞼 likelihood of the selected eyelid .

  According to the heel likelihood calculating apparatus of the present invention, when the first ratio is larger than the predetermined value, the predetermined first value (for example, 0) is calculated as the right heel likelihood, and the second When the ratio is larger than a predetermined value, a predetermined first value (for example, 0) is calculated as the left eyelid likelihood. As a result, the heel likelihood of the moth that cannot accurately calculate the heel likelihood is set to the predetermined first value without being calculated by a complicated operation. It can be calculated.

  Further, according to the heel likelihood calculating device of the present invention, the larger value among the weighted values is selected, and the selected value is calculated as the heel likelihood of the selected heel. The heel likelihood can be calculated with better accuracy.

  In order to achieve the above object, the first program of the present invention includes a computer, the detection means in the first 瞼 likelihood calculation device or the second 瞼 likelihood calculation device of the present invention, the creation Means, first computing means, second computing means, and third computing means. Therefore, according to the program of the present invention, the ratio of the heel likelihood of the heel having a lower accuracy with respect to the overall heel likelihood becomes smaller, and thereby the overall heel likelihood can be calculated with better accuracy. it can.

  In order to achieve the above object, the second program of the present invention causes a computer to execute the first computing means, the creating means, the second computing device in the third 瞼 likelihood computing device of the present invention. It functions as a calculation means, the third calculation means, and the fourth calculation means. Therefore, according to the program of the present invention, the ratio of the heel likelihood of the heel having a lower accuracy with respect to the overall heel likelihood becomes smaller, and thereby the overall heel likelihood can be calculated with better accuracy. it can.

  In order to achieve the above object, a third program of the present invention comprises a computer, the processing means, the creating means, the third computing means in the fourth 瞼 likelihood calculating device of the present invention, It functions as the fourth calculation means and the fifth calculation means. Therefore, according to the program of the present invention, the ratio of the heel likelihood of the heel having a lower accuracy with respect to the overall heel likelihood becomes smaller, and thereby the overall heel likelihood can be calculated with better accuracy. it can.

  In order to achieve the above object, a fourth program of the present invention comprises a computer, the processing means, the creating means, the third calculating means in the fifth に お け る likelihood calculating device of the present invention, It functions as the selection means and the fourth calculation means. Therefore, according to the program of the present invention, the larger one of the weighted values is selected, and the selected value is calculated as the likelihood of the selected moth. Can be used to calculate the likelihood.

  Further, the program of the present invention further includes a value obtained by weighting a weight corresponding to the right eyelid to the maximum right eyelid likelihood, and a weight corresponding to the left eyelid being the maximum left eyelid. When both weighted values of likelihood are equal to or greater than a predetermined reference value, the right eyelid opening degree represented by the right comparison eyelid model of the maximum right eyelid likelihood and the maximum left eyelid likelihood A fifth computing means for detecting a port opening of the port represented by the left comparison rod model and calculating a ratio between the detected port opening of the starboard and the port opening of the port; and the fourth operation The heel opening degree of the heel represented by the comparison heel model of the heel likelihood calculated by the means is detected, and based on the detected heel opening degree and the ratio, The computer functions as a fifth detecting means for detecting the eyelid opening degree of the eyelid having the lower eyelid likelihood by calculating the eyelid opening degree. It may be allowed. Accordingly, when only one of the right comparative eyelid model and the left comparative eyelid model is highly reliable, the eyelid opening degree of the lower reliability eyelid can be accurately detected.

  In order to achieve the above object, a fifth program of the present invention causes a computer to execute the processing means, the creating means, the third means in the sixth or seventh likelihood likelihood arithmetic apparatus of the present invention. The calculation means, the fourth calculation means, the selection means, and the fifth calculation means are caused to function. Therefore, according to the program of the present invention, the larger one of the weighted values is selected, and the selected value is calculated as the likelihood of the selected moth. Can be used to calculate the likelihood. Also, the left and right eyelid likelihood can be calculated more easily.

  As described above, according to the eyelid likelihood calculation apparatus and program of the present invention, even when the face image included in the image is facing sideways, the eyelid likelihood is improved with better accuracy. It is possible to obtain the effect of being able to calculate with

  Hereinafter, with reference to the drawings, embodiments of the heel likelihood computing device of the present invention will be described in detail.

[First Embodiment]
First, the first embodiment will be described. As shown in FIG. 1, the heel likelihood calculating apparatus 10 according to the present embodiment includes a camera 12 and a computer 14 as photographing means for photographing an object to be photographed. Note that, as an object to be photographed, for example, the face of a person who is to calculate the overall likelihood of wrinkles can be considered.

  The camera 12 captures an image including an image of a face, and from a right heel 64, a left upper heel 66, and a left lower heel 68 comprising a right upper eyelid 60 and a right lower eyelid 62 as shown in FIG. The image data including the face image 72 including the left port 70 can be output.

  The computer 14 includes an I / O (input / output) port 14a, a ROM (Read Only Memory) 14b, an HDD (Hard Disk Drive) 14c, a CPU (Central Processing Unit) 14d, a RAM (Random Access Memory) 14e, and these I / Os. The bus 14f connects the O port 14a, the ROM 14b, the HDD 14c, the CPU 14d, and the RAM 14e to each other.

  A basic program such as an OS is stored in the ROM 14b as a storage medium.

  The HDD 14c serving as a storage medium stores a program for executing a processing routine of a heel likelihood calculation process, which will be described in detail below.

In addition, the HDD 14c uses, for example, a known technique such as “Kurome tracking by Condensation using an eye model” (written by Yosuke Kitagawa, Takekazu Kato, Gen Wu, and Toshikazu Wada). The point P ₁ (= (x ₁ , y ₁ , z) corresponding to the right eye's eye contact as shown in FIG. 3 is determined based on r _b , the radius r _{w of} the eyeball, and the distance d between the centers of both eyes. ₁ )), a point P ₃ (= (x ₃ , y ₃ , z ₃ )) corresponding to the right eye, and a point P ₂ (= (x ₂ , y ₂ , z ₂ )) and the point P ₄ (= (x ₄ , y ₄ , z ₄ )) corresponding to the lower eyelid of the right eye, Each of the plurality of starboard control points is appropriately determined, and the right upper three-dimensional shape and the right lower three-dimensional The shape (that is, the starboard three-dimensional shape) is determined by a Bezier curve, and a predetermined two-dimensional starboard model obtained by projecting the determined starboard three-dimensional shape onto the image coordinates is stored. Has been. The coordinates that define each starboard control point used to form a predetermined starboard kite model correspond to the first model parameter.

Similarly, the HDD 14c uses the above-described known technique to determine the point P corresponding to the eye of the left eye, which is determined based on the radius r _{b of} the black eye, the radius r _{w of} the eyeball, the distance d between the centers of both eyes, and the like. ₅ (= (x ₅ , y ₅ , z ₅ )), a point P ₇ (= (x ₇ , y ₇ , z ₇ )) corresponding to the left eye's eyelid, between the head and the eyelid in the upper eyelid of the left eye Of the corresponding point P ₆ (= (x ₆ , y ₆ , z ₆ )) and the corresponding point P ₈ (= (x ₈ , y ₈ , z ₈ )) between the eyes and the eyelid in the lower eyelid of the left eye Each of these plurality of port control points is appropriately determined with each as a port control point, and the left upper 3D shape and the left lower 3D shape (ie, (3D shape) is defined by a Bezier curve, and a quadratic obtained by projecting the determined port 3D shape onto image coordinates The eyelid model given left eyelid is stored. Note that the coordinates that define each of the above-mentioned port control points used to form a predetermined port model are corresponding to the second model parameter.

  The CPU 14d reads the program from the ROM 14b and the HDD 14c and executes it. Various data are temporarily stored in the RAM 14e.

  The camera 12 is connected to the I / O port 14a.

  Next, a processing routine for the wrinkle likelihood calculation process executed by the CPU 14d of the computer 14 will be described with reference to FIG. In the present embodiment, the heel likelihood calculation process is executed every predetermined time interval (for example, several tens of milliseconds) from the time when a switch (not shown) of the heel likelihood calculation device 10 is turned on.

  First, in step 100, image data of an image taken by the camera 12 is captured.

  In the next step 102, face area image data is obtained by extracting a face area, which is a face area, from the image represented by the image data captured in step 100. As a method of extracting the face area in step 102, for example, a method of searching for a face area from an image represented by image data by image processing by a template matching method may be used, or the operator may use a keyboard (FIG. Selecting a face area on the image by face area selection means for selecting a face area such as a mouse (not shown), a mouse (not shown), an electronic pen (not shown), or a light pen (not shown). A method of extracting a face region by using may be used.

  In the next step 104, the eye region image data is obtained by extracting the eye region, which is the eye region including the eyelids and irises, from the face region image data extracted in step 102. As a method of extracting the eye region in step 104, for example, a method of searching the eye region from an image represented by the image data by image processing by a template matching method may be used, or the operator may use a keyboard (see FIG. Selecting an eye area on the image by an eye area selecting means for selecting an eye area such as a mouse (not shown), a mouse (not shown), an electronic pen (not shown) or a light pen (not shown). A method of extracting an eye region may be used.

  In the next step 106, the face orientation angle θ of the face represented by the image data is detected based on the image data of the face area acquired in step 102. As a method of detecting the face orientation angle θ in step 106, for example, a method of detecting the face orientation angle θ of the face represented by the image data by image processing by the template matching method may be used, or “adaptive diffusion” may be used. Face orientation using known techniques such as those described in “Head Posture Estimation Using Controlled Particle Filter” (written by Kenji Oka, Yoichi Sato, Yasuto Nakanishi, Hideki Koike) A method of detecting the angle θ may be used.

  In the next step 108, at least one model parameter of the first model parameter is changed a plurality of times to create a plurality of right comparison models and a plurality of at least one model parameter of the second model parameter are set. The comparison model creation process for creating a plurality of the left comparison eyelid models is executed by changing the number of times. Here, the details of the processing routine of the comparison model creation process in step 108 will be described with reference to FIG.

  First, in step 200, a starboard heel model and a port heel model whose model parameters are set to initial values are read from the HDD 14c, and the read starboard heel model is set as an initial right comparison heel model. The read left eyelid model is used as an initial left comparison eyelid model, and the left and right comparison eyelid models are stored in the RAM 14e, thereby creating a right comparison eyelid model and a left comparison eyelid model.

  Here, each of the right upper eyelid and the lower right eyelid included in the image of the eye area extracted in step 104 in the calculation process when calculating the right eyelid likelihood described later is performed on the right comparative eyelid model. It is a cocoon model compared with a shape. Further, the left comparative eyelid model is a shape of each of the left upper eyelid and the left lower eyelid included in the image of the eye region extracted in step 104 in the calculation process when calculating the left eyelid likelihood described later. It is a cocoon model compared with.

In the next step 202, a point P ₂ (= (x ₂ , y ₂ , z ₂ )) corresponding to the distance between the head and the eyelid in the upper eyelid of the right comparative eyelid model created in step 200 above, And a combination of points P ₄ (= (x ₄ , y ₄ , z ₄ )) corresponding to the distance between the head and the eyelid in the lower eyelid of the right eye, a plurality of points determined in advance for each of the points P ₂ and P ₄ The coordinates of the point P ₂ and the point P ₄ that have not yet been selected in the case of the combination of the coordinates of the current point P ₁ and the point P ₃ from the plurality of coordinate patterns for creating the right comparison model Select one of the combinations. Similarly, a point P ₆ (= (x ₆ , y ₆ , z ₆ )) corresponding to the distance between the eyelid and the eyelid in the upper eyelid of the left eye of the left comparative eyelid model created in step 200 above, and For a combination of points P ₈ (= (x ₈ , y ₈ , z ₈ )) corresponding to the distance between the head and the eyelid in the lower eyelid of the left eye, a plurality of predetermined points are set for each of the points P ₆ and P ₈ . from a pattern of a plurality of coordinates for creating left comparison eyelid model, the coordinates of P ₆ and the point P ₈ that has not been selected in the case of a combination of the coordinates of the point P ₅ and the point P ₇ of moment Select one combination.

In the next step 204, the coordinates of _{P 2} and the point _{P 4} terms of time, to change the coordinate of _{P 2} and the point _{P 4} points selected in step 202, a point after the change _{P 2} and the point _{P 4} Using the first model parameters including the coordinates (the coordinates of each of the points P ₁ to P ₄ ), the three-dimensional shape of the right upper eyelid and the three-dimensional shape of the lower right eyelid (that is, the third order of the starboard) The original shape) is defined by a Bezier curve, and a two-dimensional eyelid model obtained by projecting the determined starboard three-dimensional shape onto image coordinates is defined as a new right comparison eyelid model. Similarly, the coordinates of _{P 6} and the point _{P 8} point time, change the coordinates of _{P 6} and the point _{P 8} points selected in step 202, the coordinates of _{P 6} and the point _{P 8} points after change Using the second model parameters (the coordinates of each of the points P ₅ to P ₈ ), the left upper three-dimensional shape and the left lower three-dimensional shape (ie, the left three-dimensional ) Is defined by a Bezier curve, and a two-dimensional wrinkle model obtained by projecting the determined port three-dimensional shape onto image coordinates is defined as a new left comparison wrinkle model. Then, the new right comparison kite model and the left comparison kite model are stored in the RAM 14e, thereby creating a new right comparison kite model and a left comparison kite model.

In the next step 206, at step 202, in the case of a combination of the coordinates of the point P ₁ and the point P ₃ of the current, whether selected for all pattern combinations coordinates of the point P ₂ and the point P ₄ In addition, in the case of the combination of the coordinates of the current point P ₅ and the point P ₇ , it is determined whether or not the combination of the coordinates of the point P ₆ and the point P ₈ has been selected for all patterns.

If it is determined in step 206 that all patterns are not selected in at least one of the combination of the coordinates of the points P ₂ and P ₄ and the combination of the coordinates of the points P ₆ and P ₈ , Returning to step 202 above, the processing corresponding to the coordinates of the combination determined not to be selected for all patterns is performed in the same manner as described above.

On the other hand, in step 206, it is determined that the combination of the coordinates of the point P ₂ and the point P ₄ is determined to be selected for all the patterns, and to select for all patterns combinations coordinates of the point P ₆ and the point P ₈ If YES, go to the next step 208.

In the next step 208, the determining whether to select for all patterns combinations coordinates of the point P ₁ and the point P ₃ in step 210 whose details are described below, the coordinates of the point P ₅ and the point P ₇ It is determined whether or not the combination is selected for all patterns.

If it is determined in step 208 that not all patterns have been selected in at least one of the combination of the coordinates of the points P ₁ and P ₃ and the combination of the coordinates of the points P ₅ and P ₇ , Proceed to the next step 210.

In step 210, from a pattern of a plurality of coordinates for creating each plurality of right comparison eyelid model predetermined for each of the points P ₁ and the point P _3, yet selected a point not P ₁ and point the combination of the coordinates of P ₃ 1 single select. Similarly, from a pattern of a plurality of coordinates for creating each plurality of left comparison eyelid model predetermined for each of the point P ₅ and the point P _7, yet selected a point not P ₅ and It selects one combination of the coordinates of the point P _7. Then, the process returns to step 204. In the case where back from step 210 to step 204, in step 204, in the pattern of a plurality of coordinates for creating each multiple comparison eyelid model predetermined for each of the points P ₂ and the point P ₄ from selects one combination of yet selected a point not coordinate of P ₂ and the point P ₄ in the case of a combination of selected points the coordinates of P ₁ and the point P ₃ in the most recent step 210. Then, the coordinates of P ₁ and the point P ₃ point current, the change in the immediate vicinity of the coordinates of P ₁ and the point P ₃ points selected in step 210, the coordinates of P ₂ and the point P ₄ terms of current The first model parameters including the coordinates of the point P ₁ , the point P ₂ , the point P _3, and the point P ₄ are changed to the coordinates of the point P ₂ and the point P ₄ selected in the most recent step 202. The three-dimensional shape of the right upper eyelid and the three-dimensional shape of the lower right eyelid (that is, the three-dimensional shape of the right starboard) are expressed by a Bezier curve using (the coordinates of each of the points P ₁ to P ₄ ). A two-dimensional eyelid model obtained by projecting the determined three-dimensional starboard shape onto the image coordinates is set as a new right comparison eyelid model. In step 204, similarly, from the pattern of a plurality of coordinates for creating each plurality of left comparison eyelid model predetermined for each of the points P ₆ and the point P _8, the most recent step 210 selects one combination of yet selected a point not coordinates of P ₆ and the point P ₈ in the in the case of a combination of selected points the coordinates of P ₅ and the point P _7. Then, the coordinates of P ₅ and the point P ₇ point current, the change in the immediate vicinity of the coordinates of P ₅ and the point P ₇ selected points in step 210, the coordinates of P ₆ and the point P ₈ points the current The first parameter (including the coordinates of the point P ₅ , the point P ₆ , the point P _7, and the point P ₈ after the change is changed to the coordinates of the point P ₆ and the point P ₈ selected in the latest step 204. three-dimensional shape of the three-dimensional shape and left under eye eyelid upper eyelid of the left eye using each of the coordinates) of the point P _{5 ~} point P ₈ (i.e., defined by the three-dimensional shape) of the Bezier curve of the left eyelid The two-dimensional eyelid model obtained by projecting the determined three-dimensional shape of the left eyelid onto the image coordinates is used as a new eye eye comparison eyelid model. Then, a new comparison eyelid model and a left eye comparison eyelid model are stored in the RAM 16e as new comparison eyelid models, thereby creating a new comparison eyelid model.

On the other hand, in step 208, it is determined that the combination of the coordinates of the point P ₁ and point P ₃ is determined to be selected for all the patterns, and to select for all patterns combinations coordinates of the point P ₅ and the point P ₇ If it is found, the comparison model creation process is terminated.

  As described above, a plurality of right comparison eyelid models and a plurality of left comparison eyelid models are created by the comparison model creation processing.

  In the next step 110, one right comparison kite model that has not yet been selected in this step 110 is selected from the plurality of right comparison kit models created in step 108.

In the next step 112, the upper and lower eyelid shapes represented by the right comparative eyelid model selected in step 110 and the right upper eyelid and the right eyelid included in the image of the eye area extracted in step 104 are described. The right eyelid likelihood _{λr_eyelid} indicating the degree of matching with each shape of the right lower eyelid is calculated, and the calculated right eyelid likelihood λr_eyelid is stored in the _RAM 14e.

Note that the right eyelid likelihood λ _{r_eyelid} has a shadow on the eyelid part, and a luminance gradient from the shadow to the skin direction is generated. Therefore, the above-mentioned “black eye tracking by condensation using an eye model” (Yosuke Kitagawa, The calculation can be performed in consideration of the direction of the normal vector and the luminance gradient vector by using a known technique as described in Takekazu Kato, Wu Kaimoto and Toshikazu Wada.

  In the next step 114, it is determined whether or not all the right comparison eyelid models have been selected in step 110.

  If it is determined in step 114 that all the right comparison eyelid models have not been selected, the process returns to step 110.

  On the other hand, if it is determined in step 114 that all right comparison eyelid models have been selected, the process proceeds to the next step 116.

Note that, by repeating the processing from step 110 to step 114, the right _heel likelihood λ _{r_eyelid} of each of the plurality of right comparative _heel models created in step 108 is calculated.

  In the next step 116, one of the left comparative eyelid models that has not yet been selected in this step 116 is selected from the plurality of left comparative eyelid models created in step 108.

In the next step 118, the upper and lower eyelid shapes represented by the left comparative eyelid model selected in the above step 116, the left upper eyelid and the left eyelid included in the image of the eye area extracted in the above step 104, and the like. The left _heel likelihood λ _{l_eyelid} indicating the degree of matching with each shape of the left lower eyelid is calculated, and the calculated left _heel likelihood λ _{l_eyelid} is stored in the _RAM 14e.

Note that the left eyelid likelihood λ _{l_eyelid} has a shadow on the eyelid portion, and a luminance gradient from the shadow to the skin direction is generated. Therefore, like the above right eyelid likelihood λ _{r_eyelid} , Using known techniques, such as those described in “Kurome Tracking by Condensation Using Models” (written by Yosuke Kitagawa, Takekazu Kato, Toshikazu Wu, and Toshikazu Wada), the orientation of normal vectors and luminance gradient vectors is considered. Can be calculated.

  In the next step 120, it is determined whether or not all the left comparative eyelid models have been selected in step 116.

  If it is determined in step 120 that all the left comparison eyelid models have not been selected, the process returns to step 116.

  On the other hand, if it is determined in step 120 that all the left comparison eyelid models have been selected, the process proceeds to the next step 122.

Note that, by repeating the processing from step 116 to step 120, the left _heel likelihood λ _{l_eyelid} of each of the plurality of left comparative _heel models created in step 108 is calculated.

  In step 122, one of the combination of the right comparative kite model and the left comparative kite model having the same kite opening degree is selected from the plurality of right comparative kite models and the plurality of left comparative kite models created in step 108. To do. In step 108, one combination of the right comparative eyelid model and the left comparative eyelid model with substantially the same eyelid opening degree may be selected. Here, “having almost the same opening degree” means, for example, that the maximum value of the difference between the opening degrees when the right comparison opening model and the left comparison opening model can be regarded as the same. In other words, it refers to a case where the difference between the likelihoods of the right comparative eyelid model and the left comparative eyelid model is within the above-described maximum value.

In the next step 124, in accordance with the face orientation angle θ detected in step 106, the starboard and port side included in the face image represented by the image data including the face area extracted in step 102 is displayed. Among them, the size corresponding to the eyelid likelihood of the eyelid farther to the position where the image including the face image was photographed by the camera 12 is determined to be smaller than the size corresponding to the eyelid likelihood of the near eyelid. Weights w _right and w _left , right _瞼 likelihood λ _{r —} eyelid of the right comparison _heel model of the combination selected in the most recent step 122, and _left comparison _heel model of the combination selected in the most recent step 122 Based on the left _heel likelihood λ _{l_eyelid} , the right _heel likelihood λ _{r_eyelid} with the weight w _right and the weight as shown in the following equation (1): Mabutayudo of the sum of the eyelids likelihood _λ l_eyelid of left w _left is attached _{((w right · λ r_eyelid +} w left · λ l_eyelid) / (w right + w left)) the Ryomabuta (left and right of the eyelids) λ Calculate as _eyelid .

Here, with reference to FIG. 6, an example of a method for determining the weight w _right and the weight w _left in step 124 in accordance with the face orientation angle θ detected in step 106 will be described.

6, the size of the weights _{w left} corresponding to the face direction angle θ and the relationship between the magnitude of the weights _{w. Right} corresponding to the right of the eyelid likelihood lambda _{R_eyelid,} and face angle θ and left eyelids likelihood lambda _{L_eyelid} It is a figure showing these relationships.

When the face angle θ is a positive value (in this embodiment, θ takes a positive value when facing the right direction, and a negative value when facing the left direction. Since the distance from the _starboard to the camera 12 can be considered to be greater than the distance from the _starboard to the camera 12, the accuracy of the right eyelid likelihood _{λr_eyelid} is Can be considered to be worse than the accuracy of the 瞼 likelihood λ _{l_eyelid} of. Similarly, when the face orientation angle θ is a negative value, it can be considered that the distance from the _starboard to the camera 12 is larger than the distance from the _starboard to the camera 12, so the left eyelid likelihood λ _{l_eyelid} Can be considered to be worse than the accuracy of the right _heel likelihood λ _{r —} eyelid. Then, it is considered that the accuracy of the right _eye likelihood λ _{r_eyelid} decreases as the face orientation angle θ increases, and the accuracy of the left _eye likelihood λ _{l_eyelid} decreases as the face orientation angle θ decreases. It is done.

In the present embodiment, in consideration of these _points , the accuracy for the overall 瞼 likelihood λ _{eyelid_all} is low by reducing the weight of the 瞼 likelihood that has worse accuracy in accordance with the face orientation angle θ. The ratio of 瞼 likelihood of the (bad) moth is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step 124, for example, as shown in FIG. 6, the face orientation angle θ detected in step 106 is 15 degrees (in the example of FIG. 6, the face orientation angle θ when the face faces the front with respect to the camera 12 is 0 degree, the right face direction angle θ is a positive value, and the left face direction angle θ is a negative value.), The weight w _right is 0.25, and the weight 0.75 is defined as w _left . Then, as shown in the following formula (2), the sum of the right eyelid likelihood lambda _{R_eyelid} the weight 0.25 is given, the eyelid likelihood lambda _{L_eyelid} the left weight 0.75 is assigned lambda _eyelid Is calculated. In the example of FIG. 6, the sum of the weight w _right and the weight w _left is 1.0.

In the next step 126, it is determined whether or not all combinations have been selected in step 122. If it is determined in step 126 that not all combinations have been selected, the process returns to step 122 and one unselected combination is selected. On the other hand, if it is determined in step 126 that all combinations have been selected, the process proceeds to the next step 128. Note that, by repeating the processing from step 122 to step 126, a plurality of _heel likelihoods λ _eyelid of both eyelids in a combination of left and right comparative eyelid models having the same eyelid opening degree are calculated.

In step 128, a plurality of the eyelid likelihood lambda _eyelid of both eyelids calculated in step 124, the highest value is high (the largest) of the entire eyelid likelihood lambda _{Eyelid_max} both eyelids Mabutayudo lambda _{Eyelid_all} As a result, the overall heel likelihood λ _{eyelid_all} is calculated. Then, the wrinkle likelihood calculation process ends.

As described above, the eyelid likelihood calculation device 10 according to the present embodiment includes the camera 12 as an imaging unit that captures an image including a face image and outputs image data, and a plurality of first model parameters. Means for storing a starboard model in which the shapes of the right upper arm and the right lower arm are defined by the left and a port model in which the shapes of the left upper arm and the left lower arm are defined by a plurality of second model parameters The HDD 14c is configured to detect the face orientation angle θ based on the image data, and a plurality of right comparisons by changing at least one model parameter of the first model parameter a plurality of times. A plurality of left comparison models are created by creating a model and changing at least one model parameter of the second model parameter a plurality of times. The right upper eyelid and the right lower eyelid shape represented by the right upper eyelid and the right lower eyelid shape indicating the degree to which the shapes of the right upper eyelid and the right lower eyelid of the image taken by the camera 12 match. The _{shapes of} the left upper eyelid and the left lower eyelid represented by λ _{r_eyelid} and the left comparative eyelid model coincide with the shapes of the left upper eyelid and the left lower eyelid of the image taken by the camera 12. The left _heel likelihood λ _{l_eyelid} indicating the degree is calculated. Then, the wrinkle likelihood calculation device 10 according to the present embodiment is a position where the image is shot by the camera 12 among the starboard and the port side of the image shot by the camera 12 according to the detected face orientation angle θ. The weights w _right and w _left corresponding to each of the left and right wings are determined such that the size corresponding to the heel likelihood of the farther heel is smaller than the size corresponding to the heel likelihood of the closer _heel. The weight w _right corresponding to the right eyelid is weighted to the right eyelid likelihood λ _{r_eyelid} of the right comparison eyelid model in the combination of the right eyelid opening model and the left eyelid eyelid model having the same eyelid opening degree. value as the _{(w right} · λ _{r_eyelid),} the weights _{w left} corresponding to the left of the eyelids, weighted values to the left of the eyelids likelihood _λ l_eyelid of the left comparison eyelids model of _{(w left} · λ _{l_ey} multiple operations for each combination _lid) and the sum of the _{((w right · λ r_eyelid +} w left · λ l_eyelid) / (w right + w left)) as eyelid likelihood lambda _eyelid of the left and right in this combination. The eyelids likelihood calculation unit 10 according to the present embodiment, among the left and right eyelid likelihood lambda _eyelid of each combination in which a plurality calculation, the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo lambda _{Eyelid_all} Calculate.

According to the eyelid likelihood calculation apparatus 10 according to the present embodiment, when the face image included in the image captured by the imaging unit (the camera 12 in the present embodiment) is facing sideways (that is, included in the image). Even if the image of the face to be photographed faces sideways), the size corresponding to the eyelid likelihood of the eyelid farther from the position where the image was taken is The weight w _right corresponding to the _{right eye} among the weights w _right and w _left corresponding to each of the left and right eyelids, which is determined to be smaller than the size corresponding to the _{right eye} weight likelihood λ _{r_eylid} ( and _{w right} · λ r_eyelid), the sum of the weights _{w left} the was weighted to the left of the eyelids likelihood _λ l_eyelid value corresponding to the left of the eyelids _{(w left · λ l_eyelid) (} (w right · λ r_ _{yelid + w left · λ l_eyelid)} / (w right + w left)) a plurality of operation for each combination as left and right eyelid likelihood lambda _eyelid in this combination, of the right and left eyelids likelihood lambda _eyelid of each combination in which a plurality arithmetic since calculates the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo lambda _{Eyelid_all,} the proportion of eyelid likelihood of eyelid rather low accuracy for the entire Mabutayudo lambda _{Eyelid_all} becomes smaller, thereby The overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

Note that the overall heel likelihood λ _{eyelid_all} calculated by the heel likelihood calculating device 10 of the present embodiment can be used as a likelihood when estimating the shape of the _heel , for example. In addition, for example, the eyelid opening degree can be detected from the shape of the eyelid estimated in this way.

In addition, the heel likelihood calculating apparatus 10 of the present embodiment estimates a _heel model based on the calculated total heel likelihood λ _{eyelid_all,} and the 瞼 of the heel _represented by the heel model is estimated from the estimated _heel model. It is good also as a structure provided with the eaves opening degree detection function which detects an opening degree. In this case, the heel likelihood calculating device 10 of the present embodiment can be used as a heel opening degree detecting device.

[Second Embodiment]
Next, a second embodiment will be described. In addition, about the structure similar to 1st Embodiment, and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In this embodiment, the difference from the first embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 8 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and the CPU 14d executes the heel likelihood calculation process shown in FIG. As shown in the figure, step 300 is executed after step 104, and step 302 is executed after step 122. In the present embodiment, as the camera 12, an imaging means capable of detecting the distance to the captured object, for example, a stereo camera is used.

First, the process in step 300 will be described. In step 300, the image data of the eye region extracted in step 104 is subjected to image processing, and a distance d _right to a predetermined portion (for example, the center of the upper eyelid) of the right eye region extracted in step 104 is detected. By detecting the distance d _right from the camera 12 to the right eye, and detecting the distance d _left to a predetermined part (for example, the center of the upper eyelid) of the left eye region extracted in step 104 above, A distance d _left from the camera 12 to the left eye is detected.

Next, the process in step 302 will be described. In step 302, the image is included in the face image represented by the image data of the image including the face area extracted in step 102 according to the distance d _{right to the right} eye and the distance d _left to the _left eye detected in step 106. The size corresponding to the eyelid likelihood of the eyelid farther from the position where the image including the face image is captured by the camera 12 among the starboard and the left eyelid corresponds to the eyelid likelihood of the near eyelid. The weights w _{right — 2} and w _{left — 2} determined to be smaller than the size, the right _wrinkle likelihood λ _{r —} eyelid of the right comparison _wrinkle model of the combination selected in the most recent step 122, and the most recent step 122 Based on the left _heel likelihood λ _{l_eyelid} of the left comparison _heel model of the combination, as shown in the following equation (3), the weight w _{right _2} and right eyelid likelihood _λ r_eyelid that has been marked with the sum of the eyelids likelihood _λ l_eyelid left the weight _{w left_2} is attached _{((w right_2 · λ r_eyelid +} w left_2 · λ l_eyelid) / (w right_2 + w left_2 )) Is calculated as the 瞼 likelihood λ _eyelid of both heels (left and right heels).

Referring now to FIG. 9, in step 302, an example of a method in accordance with the distance _{d left} Distance to _{d. Right} and left to right, which is detected in step 300, determine the weights _{w Right_2} and weights _{w Left_2} Will be described.

Figure 9 _corresponds to _{(d left -d right)} and the relationship between the magnitude of the weights _{w Right_2} corresponding to the right of the eyelid likelihood lambda _{R_eyelid,} and _{(d left -d right)} and left eyelid likelihood lambda _{L_eyelid} It is a figure showing the relationship with the magnitude _| size of weight _{wleft_2} .

Here, when the distance d _right from the _starboard to the camera 12 and the distance d _left from the _starboard to the camera 12 are the same, the accuracy (reliability) of the right _eye likelihood λ _{r_eyelid} and the left It can be considered that the accuracy of the _瞼 likelihood λ _{l_eyelid} is the same.

On the other hand, when the distance d _right from the _starboard to the camera 12 is greater than the distance d _left from the _starboard to the camera 12, the accuracy of the right _eye likelihood λ _{r_eyelid is} the accuracy of the left _eye likelihood λ _{l_eyelid} . Can be considered worse. Similarly, when the distance d _left from the _starboard to the camera 12 is larger than the distance d _right from the _starboard to the camera 12, the accuracy of the left _eye likelihood λ _{l_eyelid} is _equal to that of the right _eye likelihood λ _{r_eyelid} . It can be considered worse than accuracy. Then, it is considered that the difference in accuracy increases as the difference between the distance d _right and the distance d _left increases.

In the present embodiment, in consideration of these points, the larger the difference between the distance d _right and the distance d _left , the smaller the weight of the 瞼 likelihood that the accuracy becomes worse, thereby reducing the overall 瞼The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step 302, for example, as shown in FIG. 9, the difference between the distance d _right to the right eye detected in step 300 and the distance d _{left to} the _left eye (d _left −d _right ) is 2.5 cm (in the example of FIG. 9, , D _left > d _right is a positive value, and d _left <d _right is a negative value.) If weight w _{right — 2} is 0.75, and weight w _{left — 2} is Set 0.25. Then, as shown in the following formula (4), the sum of the right eyelid likelihood lambda _{R_eyelid} the weight 0.75 is given, the eyelid likelihood lambda _{L_eyelid} the left weight 0.25 is assigned lambda _eyelid Is calculated. In the example of FIG. 9, the sum of the weight w _{right — 2} and the weight w _{left — 2} is 1.0.

As described above, the eyelid likelihood calculating apparatus according to the present embodiment performs the starboard and image staring of the image captured by the camera 12 according to the detected distance d _{right to the right} eye and the distance d _left to the _left eye. The size corresponding to the likelihood of the eyelid farther to the position of the camera 12 when the image is taken by the camera 12 (photographing position) of the left eyelid is the size corresponding to the likelihood of the eyelid closer to the eyelid determined to be less than is the weight _{w Right_2} corresponding to the right of the eyelid of the weights _{w Right_2} and _{w Left_2} corresponding to each of the right and left eyelid, the right eyelid opening is the same comparative eyelid model and left A value (w _{right — 2} · λ _{r —} eyelid) weighted to the right _eye likelihood λ _{r_eyelid} of the right comparison _eye model in the combination of the comparison _{eye models} and a weight corresponding to the left eyelid the _w left_2, the sum of the left eyelid likelihood λ value was weighted to _{l_eyelid} of the left comparison eyelids model of _{(w left_2 · λ l_eyelid) (} (w right_2 · λ r_eyelid + w left_2 · λ l_eyelid) / (w right_2 + w left_2 )) _Is calculated for each combination as left and right _wing likelihood λ _eyelid in this combination. The eyelids likelihood computation device according to the present embodiment, among the left and right eyelid likelihood lambda _eyelid of each combination in which a plurality computation, calculates the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo lambda _{Eyelid_all} To do.

According to the eyelid likelihood calculation apparatus according to the present embodiment, when the face image included in the image captured by the imaging unit (stereo camera in the present embodiment) is facing sideways (that is, included in the image). Even if the image of the face to be photographed is facing sideways), the size corresponding to the eyelid likelihood of the eyelid farther to the position where the image was taken is the eyelid likelihood of the near eyelid corresponding determined to be smaller than the size, the weight _{w Right_2} and weighted values of the weights _{w Right_2} corresponding to the right of the eyelid to the right of the eyelid likelihood lambda _{R_eyelid} of _{w Left_2} corresponding to each of the right and left eyelids (w the sum of the _{right_2} · λ _{r_eyelid),} and the value of the weights _{w left_2} was weighted to the left of the eyelids likelihood _λ l_eyelid corresponding to the left of the eyelids _{(w left_2} · _λ l_eyelid) _{(W right_2 · λ r_eyelid + w} left_2 · λ l_eyelid) / (w right_2 + w left_2)) a plurality of operation for each combination as left and right eyelid likelihood lambda _eyelid in this combination, Mabutayu of the left and right for each combination of a plurality of operation among degrees lambda _eyelid, so calculates the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo _λ eyelid_all, more the proportion of eyelids likelihood of eyelid rather low accuracy for the entire Mabutayudo lambda _{Eyelid_all} As a result, the overall 瞼 likelihood λ _{eyelid_all} can be calculated with better accuracy.

Note that the overall heel likelihood λ _{eyelid_all} calculated by the heel likelihood calculating device of the present embodiment can be used as a likelihood when estimating the shape of the _heel , for example. In addition, for example, the eyelid opening degree can be detected from the shape of the eyelid estimated in this way.

In addition, the heel likelihood calculating apparatus according to the present embodiment estimates a _heel model based on the calculated total heel likelihood λ _{eyelid_all} , and from the estimated _heel model, _cleaves the heel _represented by the heel model. It is good also as a structure provided with the eaves opening degree detection function which detects a degree. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Third Embodiment]
Next, a third embodiment will be described. In addition, about the structure similar to 2nd Embodiment, and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In this embodiment, the difference from the second embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 10 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and the CPU 14d executes the heel likelihood calculation process shown in FIG. As shown in the figure, step 304 is executed after step 102, step 306 is executed after step 304, and step 308 is executed after step 122. In the present embodiment, as in the second embodiment, as the camera 12, an imaging means capable of detecting the distance to the captured object, for example, a stereo camera is used.

  First, the process in step 304 will be described. In step 304, an eye region, which is an eye region including eyelids and irises, is extracted from the image represented by the image data acquired in step 102, as in the first and second embodiments. Thus, the image data of the eye area is acquired. Furthermore, in the present embodiment, in step 304, the ear region image data including the right ear and the left ear is extracted, thereby acquiring the ear region image data. As a method of extracting the ear region in step 304, for example, a method of searching a region including the right ear and a region including the left ear from the image represented by the image data by image processing by the template matching method may be used. Alternatively, the image is obtained by an ear region selection means for the operator to select an ear region such as a keyboard (not shown), a mouse (not shown), an electronic pen (not shown), or a light pen (not shown). A method of extracting the ear region by selecting the ear region on the screen may be used.

Next, the process at step 306 will be described. In step 306, the image data of the ear region extracted in step 304 is subjected to image processing, and a distance d _{right_2} to a predetermined portion (for example, the center of the right ear) extracted in step 304 is detected. by detects a distance _{d Right_2} from the camera 12 to the right ear, a predetermined portion of the ear regions of the left extracted in step 304 (e.g., the center of the left ear) by detecting the distance _{d Left_2} to , to detect the distance _{d left_2} from the camera 12 to the left ear.

Next, the process at step 308 will be described. At step 308, depending on the distance _{d Left_2} Distance to _{d Right_2} and left ear to the detected right ear in step 306, the image of the image data represents a face image including a face region extracted in step 102 The size corresponding to the heel likelihood of the heel farther from the position where the image including the face image is captured by the camera 12 among the starboard and portside included in the The weights w _{right — 3} and w _{left — 3} determined to be smaller than the corresponding size, the right _瞼 likelihood λ _{r —} eyelid of the right comparison _瞼 model of the combination selected in the most recent step 122, and the selection in the most recent step 122 Based on the left _heel likelihood λ _{l_eyelid} of the left comparison _heel model of the combination, the weight w _r as shown in the following equation (5) _{ight_3} and right eyelid likelihood _λ r_eyelid that has been marked with the sum of the eyelids likelihood _λ l_eyelid left the weight _{w left_3} is attached _{((w right_3 · λ r_eyelid +} w left_3 · λ l_eyelid) / (w right_3 + w left_3 )) Is calculated as the 瞼 likelihood λ _eyelid of both heels (left and right heels).

Here, the method with reference to FIG. 11, in step 308, in accordance with the distance _{d Left_2} Distance to _{d Right_2} and left ear to the detected right ear at step 306, determine the weights _{w Right_3} and weights _{w Left_3} An example will be described.

Figure 11 _corresponds to _(d left_2 _-d right_2) and the relationship between the magnitude of the weights _{w Right_3} corresponding to the right of the eyelid likelihood lambda _{R_eyelid,} and _{(d left_2 -d right_2)} and left eyelid likelihood lambda _{L_eyelid} It is a figure showing the relationship with the magnitude _| size of weight _{wleft_3} .

Here, the distance _{d Right_2} from the right ear to the camera 12, when the distance _{d Left_2} from the left ear to the camera 12 are the same, the right eyelid likelihood lambda _{R_eyelid} accuracy (reliability) and left It can be considered that the accuracy of the _瞼 likelihood λ _{l_eyelid} is the same.

On the other hand, the distance _{d Right_2} from the right ear to the camera 12, is greater than the distance _{d Left_2} from the left ear to the camera 12, the accuracy of the right eyelid likelihood lambda _{R_eyelid} the left eyelid likelihood lambda _{L_eyelid} accuracy Can be considered worse. Similarly, the distance _{d Left_2} from the left ear to the camera 12, is greater than the distance _{d Right_2} from the right ear to the camera 12, the left eyelid likelihood lambda _{L_eyelid} accuracy, the right eyelid likelihood lambda _{R_eyelid} It can be considered worse than accuracy. Then, it is considered that the difference in accuracy increases as the difference between the distance d _{right — 2} and the distance d _{left — 2} increases.

In this embodiment, in consideration of these points, the larger the difference between the distance d _{right — 2} and the distance d _{left — 2} , the smaller the weight of the likelihood that the accuracy is worsened. The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step 308, for example, as shown in FIG. 11, the difference between the distance d _{right_2} to the right ear detected in step 306 and the distance d _{left_2 to the} left ear (d _{left_2} −d _{right_2} ) is 2.5 cm (in FIG. 11). In the example, a case where d _{left — 2} > d _{right — 2} is a positive value, and a case where d _{left — 2} <d _{right — 2} is expressed as a negative value.) When the weight w _{right — 3} is 0.75, and a weight w 0.25 is determined as _{left_3} . Then, as shown in the following formula (6), the sum of the right eyelid likelihood lambda _{R_eyelid} the weight 0.75 is given, the eyelid likelihood lambda _{L_eyelid} the left weight 0.25 is assigned lambda _eyelid Is calculated. In the example of FIG. 11, the sum of the weight w _{right — 2} and the weight w _{left — 2} is 1.0.

As described above, the eyelid likelihood computation device according to this embodiment, depending on the distance d _{Left_2} Distance to d _{Right_2} and left ear to the detected right ear, the right image captured by the camera 12 The size corresponding to the eyelid likelihood of the eyelid farther from the position of the camera 12 (photographing position) when the image is taken by the camera 12 among the eyelids and the left eyelid corresponds to the eyelid likelihood of the near eyelid. determined to be smaller than the size of the weight _{w Right_3} corresponding to the right of the eyelid of the weights _{w Right_3} and _{w Left_3} corresponding to each of the right and left eyelids, right comparison eyelid models and eyelid opening is the same _Corresponds to a value weighted to the right eyelid likelihood λ _{r_eyelid} of the right comparative eyelid model in the combination of the left eyelid model (w _right — ₃ · λ _{r_eyelid} ) and the left eyelid The sum of the weights _{w left_3,} and left eyelid likelihood λ value was weighted to _{l_eyelid} of the left comparison eyelids model of _{(w left_3} · λ _{l_eyelid)} to _{((w right_3 · λ r_eyelid +} w left_3 · λ l_eyelid) / (w right_3 + W _{left — 3} )) is calculated for each combination as left and right _heel likelihood λ _eyelid in this combination. The eyelids likelihood computation device according to the present embodiment, among the left and right eyelid likelihood lambda _eyelid of each combination in which a plurality computation, calculates the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo lambda _{Eyelid_all} To do.

According to the heel likelihood calculating apparatus according to the present embodiment, as in the first embodiment and the second embodiment, the heel likelihood of the _heel having a lower accuracy with respect to the overall heel likelihood λ _{eyelid_all} . The ratio becomes smaller, which makes it possible to calculate the overall 瞼 likelihood λ _{eyelid_all} with better accuracy.

Note that the overall heel likelihood λ _{eyelid_all} calculated by the heel likelihood calculating device of the present embodiment can be used as a likelihood when estimating the shape of the _heel , for example. In addition, for example, the eyelid opening degree can be detected from the shape of the eyelid estimated in this way.

In addition, the heel likelihood calculating apparatus according to the present embodiment estimates a _heel model based on the calculated total heel likelihood λ _{eyelid_all} , and from the estimated _heel model, _cleaves the heel _represented by the heel model. It is good also as a structure provided with the eaves opening degree detection function which detects a degree. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Fourth Embodiment]
Next, a fourth embodiment will be described. In addition, about the structure similar to 1st Embodiment, and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In this embodiment, the difference from the first embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 12 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and the CPU 14d executes the likelihood likelihood calculation process shown in FIG. As shown in the figure, step 310 is executed after step 104, and step 312 is executed after step 122.

  First, the process in step 310 will be described. In step 310, an eye region contrast calculation process for calculating the contrast of the right eye region and the contrast of the left eye region is executed.

  The eye area contrast calculation process will be described with reference to FIG. 13. First, in step 400, among the eye areas extracted in step 104, the density values (pixels) of all the pixels in the right eye area (right eye area). Value: In this embodiment, for example, any value of 0 to 255) is detected.

In the next step 402, first, as shown in FIG. 14A, a histogram with the horizontal axis representing the density value and the vertical axis representing the number of pixels is created based on the density value of each pixel detected in step 400 above. To do. Then, as shown in the figure, by calculating the difference Iw _right between the maximum and minimum density values when the number of pixels is larger than a predetermined threshold value Ith _right , the contrast Iw _right of the _right eye region is calculated. Calculate.

  In the next step 404, the density values of all the pixels in the left eye region (left eye region) among the eye regions extracted in step 104 are detected.

In the next step 406, first, as shown in FIG. 14B, based on the density value of each pixel detected in the above step 404, a histogram is created with the horizontal axis representing the density value and the vertical axis representing the number of pixels. To do. Then, as shown in the figure, by calculating the difference Iw _left between the maximum and minimum density values when the number of pixels is larger than a predetermined threshold value Ith _left , the contrast Iw _left of the _left eye area is calculated. Calculate. Then, the eye area contrast calculation process ends. As described above, the contrasts of the left and right eye regions are calculated by the eye region contrast calculation process in step 310.

  Here, the lower the contrast (smaller), the more blurred the image. Therefore, the lower the contrast, the farther the distance between the camera 12 and its subject (shooting target location) can be considered. Similarly, the higher the contrast (higher), the clearer the image. Therefore, the higher the contrast, the closer the distance between the camera 12 and the subject to be photographed (photographing target location).

Next, the process at step 312 will be described. In step 312, the face represented by the image data of the image including the face region extracted in step 102 according to the contrast Iw _{right of the right} eye region and the contrast Iw _left of the _left eye region calculated in step 310. The size corresponding to the eyelid likelihood of the eyelid having the smaller contrast (the one farther from the position where the image including the face image was taken by the camera 12) of the starboard and the left eyed included in the image of The weights w _{right — 4} and w _{left — 4} determined to be smaller than the size corresponding to the _heel likelihood of the larger (closer) _wing , the right of the right comparison _wing model of the combination selected in the previous step 122 Mabutayudo _λ r_eyelid, and combination selected in the last step 122 the left comparison eyelid model left of Based on the likelihood lambda _{L_eyelid,} as shown in the following formula (7), and right eyelid likelihood lambda _{R_eyelid} a weight _{w Right_4} is attached, and eyelids likelihood lambda _{L_eyelid} the left weights _{w Left_4} is attached The sum of (( _{wright_4} · _{λr_eyelid} + _{wleft_4} · _{λl_eyelid} ) / ( _{wright_4} + _{wleft_4} )) is calculated as the 瞼 likelihood λ _eyelid of both wings (left and right _wings ).

Referring now to FIG. 15, in step 312, in response to the contrast _{Iw left} contrast _{Iw. Right} and left eye of the eye region of the right eye of the eye region which is calculated in step 310, the weights _{w Right_4} and weights _{w Left_4} An example of the determining method will be described. In the present embodiment, the method for determining the weight w _{right — 4} and the weight w _{left — 4 differs} depending on the magnitude relationship between Iw _right and Iw _left as shown below.

FIG. 15A shows a ratio (Iw _left / Iw _right ) that is a value obtained by dividing (dividing) Iw _left by Iw _right in the case of Iw _left <Iw _right and a weight corresponding to the right _heel likelihood λ _{r_eylid.} relationship between the size of _w right_4, and a graph showing the relation between the size of the weights _{w Left_4} the ratio and _{(Iw left /} Iw _right) corresponding to the left eyelid likelihood _λ l_eyelid.

Here, when the ratio (Iw _left / Iw _right ) is 1, it is _{considered that} the accuracy (reliability) of the right _heel likelihood λ _{r_eyelid and} the accuracy of the left _heel likelihood λ _{l_eyelid} are the same. Can do.

On the other hand, when the ratio _{(Iw left /} Iw _right) is smaller than 1, the accuracy of the left eyelid likelihood lambda _{L_eyelid} can be considered worse than the precision of the right eyelid likelihood _λ r_eyelid. Then, it is considered that the accuracy of the left _heel likelihood λ _{l_eyelid} is worse than the accuracy of the right _heel likelihood λ _{r_eyelid} as the ratio (Iw _left / Iw _right ) becomes smaller than 1.

In the present embodiment, in consideration of these, the weight of the overall likelihood is reduced by reducing the weight of the likelihood that the accuracy becomes worse as the ratio (Iw _left / Iw _right ) becomes smaller than 1. The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step 312, if Iw _left <Iw _right , as shown in FIG. 15A, the ratio that is a value obtained by dividing the contrast Iw _left of the _left eye region by the contrast Iw _right of the _right eye region. (Iw _left / Iw _right ) is calculated to determine the corresponding weight w _{right — 4} and weight w _{left — 4} . More specifically, for example, when the ratio (Iw _left / Iw _right ) in the case of Iw _left <Iw _right is 0.75, the weight w _{right — 4} is 0.75, and the weight w _{left — 4} is 0.25. Determine. Then, as shown in the following equation (8), the sum of the right eyelid likelihood lambda _{R_eyelid} the weight 0.75 is given, the eyelid likelihood lambda _{L_eyelid} the left weight 0.25 is assigned lambda _eyelid Is calculated. In the example of FIG. _15A , the sum of the weight w _{right — 4} and the weight w _{left — 4} is 1.0.

FIG. _15B corresponds to the ratio (Iw _right / Iw _left ), which is a value obtained by dividing (dividing) Iw _right by Iw _left in the case of Iw _left ≧ Iw _right and the right likelihood λ _{r_eylid} relationship between the magnitude of the weights _{w Right_4} to, and a graph showing the relation between the size of the weights _{w Left_4} the ratio and _(Iw right _/ Iw _left) correspond to the left eyelid likelihood _λ l_eyelid.

Here, when the ratio (Iw _right / Iw _left ) is 1, it is _{considered that} the accuracy (reliability) of the right _heel likelihood λ _{r_eyelid and} the accuracy of the left _heel likelihood λ _{l_eyelid} are the same. Can do.

On the other hand, when the ratio _(Iw right _/ Iw _left) is smaller than 1, the accuracy of the right eyelid likelihood lambda _{R_eyelid} can be considered worse than the precision of the left eyelid likelihood _λ l_eyelid. Then, it is considered that the accuracy of the right _heel likelihood λ _{r_eyelid} becomes worse than the accuracy of the left _heel likelihood λ _{l_eyelid} as the ratio (Iw _right / Iw _left ) becomes smaller than 1.

In this embodiment, in consideration of the above, the smaller than the ratio _{(Iw right /} Iw _left) is 1, by reducing the weight of the eyelid likelihood of better precision is deteriorated, the entire eyelid The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step 312, if Iw _left ≧ Iw _right , as shown in FIG. 15B, the ratio that is the value obtained by dividing the contrast Iw _right of the _right eye region by the contrast Iw _left of the _left eye region determine the weights _{w Right_4} and weights _{w Left_4} corresponding by calculating _(Iw right _{/ Iw left).} More specifically, for example, when the ratio (Iw _right / Iw _left ) in the case of Iw _left ≧ Iw _right is 0.75, the weight w _{right — 4} is 0.25, and the weight w _{left — 4} is 0.75. Determine. Then, the sum λ _eyelid is calculated as described above. In the example of FIG. _15B , the sum of the weight w _{right — 4} and the weight w _{left — 4} is 1.0.

As described above, the eyelid likelihood calculating apparatus according to the present embodiment performs the starfish and image staring of the image photographed by the camera 12 according to the calculated contrast Iw _{right of the right} eye region and the contrast Iw _left of the _left eye region. The size corresponding to the likelihood of the eyelid of the left eyelid having the smaller contrast (the one farther from the position of the camera 12 (photographing position) when the image is taken by the camera 12) is the one having the larger contrast ( Nearby more) it was determined to be smaller than a size corresponding to the eyelid likelihood of eyelid of the weight _{w Right_4} corresponding to the right of the eyelid of the weights _{w Right_4} and _{w Left_4} corresponding to each of the right and left eyelid, eyelid open heavy degrees to the right of the eyelid likelihood lambda _{R_eyelid} right comparison eyelid model in combination right comparison eyelid model and left comparison eyelid models are identical The sum of the assigned value _{(w right_4} · λ _{r_eyelid),} the weights _{w left_4} corresponding to the left of the eyelids, and the left of the eyelids likelihood λ value was weighted to _{l_eyelid} of the left comparison eyelids model of _{(w left_4} · λ _{l_eyelid)} ((W _right — ₄ · λ _r — _eyelid + w _left — ₄ • λ _{l — eyelid} ) / (w _{right — 4} + w _{left — 4} )) is calculated for each combination as a plurality of right and left likelihoods λ _eyelid . The eyelids likelihood computation device according to the present embodiment, among the left and right eyelid likelihood lambda _eyelid of each combination in which a plurality computation, calculates the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo lambda _{Eyelid_all} To do.

According to the heel likelihood calculating apparatus according to the present embodiment, as in the first to third embodiments, the ratio of the heel likelihood of the _heel having a lower accuracy with respect to the overall heel likelihood λ _{eyelid_all} is smaller. Thus, the overall heel likelihood λ _{eyelid_all} can be calculated with better accuracy.

Note that the overall heel likelihood λ _{eyelid_all} calculated by the heel likelihood calculating device of the present embodiment can be used as a likelihood when estimating the shape of the _heel , for example. In addition, for example, the eyelid opening degree can be detected from the shape of the eyelid estimated in this way.

In addition, the heel likelihood calculating apparatus according to the present embodiment estimates a _heel model based on the calculated total heel likelihood λ _{eyelid_all} , and from the estimated _heel model, _cleaves the heel _represented by the heel model. It is good also as a structure provided with the eaves opening degree detection function which detects a degree. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Fifth Embodiment]
Next, a fifth embodiment will be described. In addition, about the structure similar to each 1st-4th embodiment and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In the present embodiment, the difference from the fourth embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 16 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and the CPU 14d executes the likelihood likelihood calculation process shown in FIG. As shown in the figure, step 314 is executed after step 104, and step 316 is executed after step 122.

  First, the process in step 314 will be described. In step 314, an eye region edge strength calculation process for calculating the edge strength of the right eye region and the edge strength of the left eye region is executed.

  This eye region edge strength calculation processing will be described with reference to FIG. 17. First, in step 500, among the eye regions extracted in step 104, the edge strengths of all the pixels in the right eye region (right eye region) are calculated. To detect. Here, the edge strength will be described. The edge strength indicates the degree of change in the pixel value of adjacent pixels, and serves as an index of image sharpness. For example, when an image in which an object is clearly displayed (clear image) is compared with an image in which the object is blurred (unclear image), the sharpness of the boundary line that separates the object from its surroundings (strong edge) The sharp image is larger than the unclear image. Therefore, the edge strength is an index of image sharpness.

In the next step 502, first, as shown in FIG. 18A, based on the edge intensity of each pixel detected in step 500, a histogram is created with the horizontal axis representing the edge intensity and the vertical axis representing the number of pixels. To do. Then, as illustrated in the figure, by the number of pixels for calculating the difference between _{Ew. Right} between the maximum and minimum of the edge intensities in the case greater than the predetermined threshold _{Eth. Right,} edge intensity of the right eye of the eye region _{Ew. Right} Is calculated.

  In the next step 504, edge strengths of all pixels in the left eye region (left eye region) among the eye regions extracted in step 104 are detected.

In the next step 506, first, as shown in FIG. 18B, based on the edge intensity of each pixel detected in step 504, a histogram is created with the horizontal axis representing the edge intensity and the vertical axis representing the number of pixels. To do. Then, as illustrated in the figure, by the number of pixels for calculating the difference between _{Ew left} between the maximum and minimum of the edge intensities in the case greater than the predetermined threshold value _{Eth left,} edge intensity of the left eye of the eye region _{Ew left} Is calculated. Then, the eye region edge strength calculation process is terminated. As described above, the edge strengths of the left and right eye regions are calculated by the eye region edge strength calculation processing in step 314.

  Here, since the image becomes blurred as the edge strength is low (small), it can be seen that the distance between the camera 12 and the subject to be photographed (photographing target location) becomes farther as the edge strength is low. Similarly, since the image becomes clearer as the edge strength is higher (larger), it can be seen that the higher the edge strength is, the closer the distance between the camera 12 and the subject to be photographed (photographing target location) is.

Next, the process at step 316 will be described. In step 316, the image data of the image including the face area extracted in step 102 is obtained in accordance with the edge intensity E _{right of the right} eye area and the edge intensity E _left of the _left eye area calculated in step 314. The size corresponding to the likelihood of the eyelid having the smaller edge strength (the one farther from the position where the image containing the face image was photographed by the camera 12) of the starboard and the left eyed included in the face image to be represented , The weights w _{right — 5} and w _{left — 5} determined to be smaller than the size corresponding to the _{wrinkle likelihood} of the _wrinkle having the larger edge strength (the closer one), the right comparison _{wrinkle of} the combination selected in step 122 above. model of the right eyelid likelihood _{λ r_eyelid,} and eyelids likelihood of left immediate left comparison eyelid model combination selected in step 122 lambda Based on _{_Eyelid,} as shown in equation (9) below, the sum of the right eyelid likelihood lambda _{R_eyelid} a weight _{w Right_5} is attached, the eyelid likelihood lambda _{L_eyelid} the left weights _{w Left_5} is attached ( (W _right — ₅ · λ _r — _eyelid + w _left — ₅ • λ _{l — eyelid} ) / (w _{right — 5} + w left — ₅ )) is calculated as the likelihood λ _eyelid of both wings (left and right _wings ).

Here, referring to FIG. 19, in step 316, the weight w _{right — 5} and the weight w according to the edge strength _{E right of the right} eye region and the edge strength _{E left} of the _left eye region calculated in step 314 above. An example of a method for determining _{left_5} will be described. In the present embodiment, the method for determining the weight w _{right — 5} and the weight w _{left — 5 differs} depending on the magnitude relationship between _E w _right and _E w _left as shown below.

19 (A) _is the weight corresponding to Ew _{left <ratios} are _{Ew left} was divided by _{Ew. Right} (divided) value in the case of _{Ew right (Ew left / Ew right} ) and right eyelid likelihood lambda _{R_eyelid} relationship between the size of _w right_5, and a graph showing the relation between the size of the ratio _{(Ew left /} Ew _right) and the weight _{w Left_5} corresponding to the left eyelid likelihood _λ l_eyelid.

Here, when the ratio (Ew _left / Ew _right ) is 1, it is _{considered that} the accuracy (reliability) of the right _heel likelihood λ _{r_eyelid and} the accuracy of the left _heel likelihood λ _{l_eyelid} are the same. Can do.

On the other hand, when the ratio _{(Ew left /} Ew _right) is smaller than 1, the accuracy of the left eyelid likelihood lambda _{L_eyelid} can be considered worse than the precision of the right eyelid likelihood _λ r_eyelid. Then, it is considered that the accuracy of the left _heel likelihood λ _{l_eyelid} is worse than the accuracy of the right _heel likelihood λ _{r_eyelid} as the ratio (Ew _left / Ew _right ) is smaller than 1.

In the present embodiment, in consideration of these, the weight of the overall likelihood is reduced by reducing the weight of the likelihood that the accuracy becomes worse as the ratio (Ew _left / Ew _right ) becomes smaller than 1. The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step 316, when Ew _left <Ew _right , as shown in FIG. 19A, the edge intensity E _left of the _left eye area is divided by the edge intensity E wright of the _right eye area. A certain ratio (Ew _left / Ew _right ) is calculated to determine the corresponding weight w _{right — 5} and weight w _{left — 5} . More specifically, for example, when the ratio (Ew _left / Ew _right ) in the case of Ew _left <Ew _right is 0.75, the weight w _{right — 5} is 0.75, and the weight w _{left — 5} is 0.25. Determine. Then, as shown in the following formula (10), the sum of the right eyelid likelihood lambda _{R_eyelid} the weight 0.75 is given, the eyelid likelihood lambda _{L_eyelid} the left weight 0.25 is assigned lambda _eyelid Is calculated. In the example of FIG. _19A , the sum of the weight w _{right — 5} and the weight w _{left — 5} is 1.0.

Further, _FIG. 19 (B) corresponds to _{Ew left} ≧ _Ew ratio is _{Ew. Right} was divided by _{Ew left} (divided) value in the case of _{right (Ew right /} Ew _left) and right eyelid likelihood lambda _{R_eyelid} relationship between the magnitude of the weights _{w Right_5} to, and a graph showing the relation between the size of the ratio _(Ew right _/ Ew _left) and the weight _{w Left_5} corresponding to the left eyelid likelihood _λ l_eyelid.

The ratio in the case of _(Ew right _/ Ew _left) is 1, be considered to be identical to the right of the eyelid likelihood lambda _{R_eyelid} accuracy (reliability) and left eyelid likelihood lambda _{L_eyelid} accuracy Can do.

On the other hand, when the ratio _(Ew right _/ Ew _left) is smaller than 1, the accuracy of the right eyelid likelihood lambda _{R_eyelid} can be considered worse than the precision of the left eyelid likelihood _λ l_eyelid. Then, it is considered that the accuracy of the right _eye likelihood λ _{r_eyelid} is worse than the accuracy of the left _eye likelihood λ _{l_eyelid} as the ratio (Ew _right / Ew _left ) becomes smaller than 1.

In this embodiment, in consideration of the above, the ratio _{(Ew right /} Ew _left) is the smaller than 1, by reducing the weight of the eyelid likelihood of better precision is deteriorated, the entire eyelid The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step 316, when Ew _left ≧ Ew _right , as shown in FIG. 19B, the edge intensity E _right of the _right eye area is divided by the edge intensity E _left of the _left eye area. determine the weights _{w Right_5} and weights _{w Left_5} corresponding calculates a certain ratio _(Ew right _/ Ew _left). More specifically, for _example, the ratio in the case of the _{Ew left ≧ Ew right (Ew right} / Ew left) is, when it is 0.75, as the weight _{w right_5} 0.25, and as the weight _{w Left_5} 0.75 Determine. Then, the sum λ _eyelid is calculated as described above. Also in the example of FIG. _19B , the sum of the weight w _{right — 5} and the weight w _{left — 5} is 1.0.

As described above, the eyelid likelihood calculation apparatus according to the present embodiment performs the right operation on the image captured by the camera 12 in accordance with the calculated edge intensity Ewright of the _right eye region and the edge intensity Ewleft of the _left eye region. The size corresponding to the likelihood of the eyelid of the eyelid or the left eyelid having the smaller edge strength (the one farther from the position of the camera 12 when the image is taken by the camera 12) is the edge strength. weights _{w Right_5} which is larger determined to be smaller than the size corresponding to the eyelid eyelids likelihood of (near better), corresponding to the right of the eyelid of the weights _{w Right_5} and _{w Left_5} corresponding to each of the right and left eyelid the eyelid opening was weighted to the right of the eyelid likelihood lambda _{R_eyelid} right comparison eyelid model in combination right comparison eyelid model and left comparison eyelid models are identical And _(w right_5 · λ r_eyelid), the sum of the weights _{w left_5} corresponding to the left of the eyelids, and the left of the eyelids likelihood λ value was weighted to _{l_eyelid} of the left comparison eyelids model of _{(w left_5 · λ l_eyelid) (} (w _{right_5} · λ _{r_eyelid} + w _left — ₅ · λ _{l_eyelid} ) / (w _{right — 5} + w _{left — 5} )) is calculated for each combination as a plurality of right and left likelihoods λ _eyelid . The eyelids likelihood computation device according to the present embodiment, among the left and right eyelid likelihood lambda _eyelid of each combination in which a plurality computation, calculates the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo lambda _{Eyelid_all} To do.

According to the heel likelihood calculating apparatus according to the present embodiment, as in the first to fourth embodiments, the ratio of the heel likelihood of the _heel having a lower accuracy with respect to the overall heel likelihood λ _{eyelid_all} is smaller. Thus, the overall heel likelihood λ _{eyelid_all} can be calculated with better accuracy.

Note that the overall heel likelihood λ _{eyelid_all} calculated by the heel likelihood calculating device of the present embodiment can be used as a likelihood when estimating the shape of the _heel , for example. In addition, for example, the eyelid opening degree can be detected from the shape of the eyelid estimated in this way.

In addition, the heel likelihood calculating apparatus according to the present embodiment estimates a _heel model based on the calculated total heel likelihood λ _{eyelid_all} , and from the estimated _heel model, _cleaves the heel _represented by the heel model. It is good also as a structure provided with the eaves opening degree detection function which detects a degree. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Sixth Embodiment]
Next, a sixth embodiment will be described. In addition, about the structure similar to each 1st-5th embodiment and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In this embodiment, the difference from the fourth embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 20 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and the CPU 14d executes the wrinkle likelihood calculation process shown in FIG. As shown in the figure, step 318 is executed after step 104, and step 320 is executed after step 122.

First, the process at step 318 will be described. In step 318, the right eye eye-to-eye distance d _{right — 3} is detected from the image data of the right eye region (right eye region) among the eye regions extracted in step 104, and the left eye region (left eye region) is detected. ) To detect the distance d left — 3 between the head of the left eye and the _eyelid . For example, in step 318, a known technique described in a non-patent document “C. Harris and M. Stephens,“ A combined corner and edge detector ”, Proc. Alvey Vision Conf., Pp. 147-151, 1988.” is used. Then, the position of the right eye's eyes and the position of the right eye's eyes, and the position of the left eye's eyes and the position of the left eye's eyes are detected, and from each of the detected positions, the distance d right — ₃ and the left eye's eyes the distance _{d Left_3} the inner corner and the left eye of crow's feet can be detected.

Here, by comparing the distances d _{right — 3} and d _{left — 3} , the eyelid with the shorter distance between the eyes and the eyes has a shorter distance to the camera 12 than the eyelash with the longer distance between the eyes and the eyes. It can be determined that the position is large (the position from the camera 12 is far).

Next, the process at step 320 will be described. At step 320, depending on the distance _{d Left_3} between the distance _{d Right_3} and left eye inner canthus and crow's feet and the detected right eye inner canthus and eye corner in step 318, images of the image containing the face region extracted in step 102 Of the starboard and the starboard included in the face image represented by the data, the size corresponding to the eyelid likelihood of the eyelid farther from the position where the image including the face image was captured by the camera 12 is the closer Weights w _{right — 6} and w _{left — 6} determined to be smaller than the size corresponding to the _瞼 likelihood of _瞼 , the right _瞼 likelihood λ _{r_eyelid} of the right comparison _瞼 model of the combination selected in the most recent step 122, and the latest based on the left of the left comparison eyelid model combination selected in step 122 of the eyelid likelihood _λ l_eyelid, the following equation (1 As shown in), and the right of the eyelids likelihood _λ r_eyelid the weight _{w right_6} is attached, the sum of the eyelids likelihood _λ l_eyelid left the weight _{w left_6} is attached _{((w right_6 · λ r_eyelid +} w left_6 · λ _{l_eyelid} ) / ( _{wright_6} + _{wleft_6} )) is calculated as the 瞼 likelihood λ _eyelid of both wings (left and right _wings ).

Referring now to FIG. 21, in step 320, depending on the distance _{d Left_3} between the distance _{d Right_3} and left eye inner canthus and crow's feet and the detected right eye inner canthus and eye corner in step 318, the weights _{w Right_6} and An example of a method for determining the weight w _{left — 6} will be described. In the present embodiment, the method of determining the weight w _{right — 6} and the weight w _{left — 6} is different depending on the magnitude relationship between d _{right — 3} and d _{left — 3} as shown below.

FIG. 21A shows a ratio (d _{left — 3} / d right — ₃ ) that is a value obtained by dividing (dividing) d _{left — 3} by d _{right — 3 in} the case of d _{left — 3} <d right — 3, and a weight corresponding to the right likelihood λ _{r —} eyelid It is a _{figure showing} the relationship with the magnitude _| size of _{wright_6} , and the relationship between ratio ( _{dleft_3} / _{dright_3} ) and the magnitude _| size of the weight _{wleft_6} corresponding to the left eyelid likelihood (lambda) _{l_eyelid} .

Here, when the ratio (d _{left — 3} / d _{right — 3} ) is 1, it is _{considered that} the accuracy (reliability) of the right _heel likelihood λ _{r_eyelid} is the same as the accuracy of the left _heel likelihood λ _{l_eyelid.} Can do.

On the other hand, when the ratio _{(d left_3 /} d _{right_3)} is smaller than 1, the accuracy of the left eyelid likelihood lambda _{L_eyelid} can be considered worse than the precision of the right eyelid likelihood _λ r_eyelid. It is considered that the accuracy of the left _heel likelihood λ _{l_eyelid} is worse than the accuracy of the right _heel likelihood λ _{r_eyelid} as the ratio (d _{left — 3} / d _{right — 3} ) is smaller than 1.

In the present embodiment, in consideration of these _points , the weight of the overall likelihood is reduced by reducing the weight of the likelihood that the accuracy becomes worse as the ratio (d _{left — 3} / d _{right — 3} ) becomes smaller than 1. The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step _320, if it is _d left_3 _<d right_3, as shown in FIG. 21 (A), the distance _{d Left_3} the inner corner and the corner of the eye of the eye, divided by the distance _{d Right_3} the right eye inner corner and corner of the eye A ratio (d _{left — 3} / d _{right — 3} ), which is a value, is calculated to determine the corresponding weight w _{right — 6} and weight w _{left — 6} . More specifically, for example, when the ratio (d _{left — 3} / d _{right — 3} ) in the case of d _{left — 3} <d _{right — 3} is 0.75, the weight w _{right — 6} is 0.75, and the weight w _{left — 6} is 0.25. Determine. Then, as shown in the following formula (12), the sum of the right eyelid likelihood lambda _{R_eyelid} the weight 0.75 is given, the eyelid likelihood lambda _{L_eyelid} the left weight 0.25 is assigned lambda _eyelid Is calculated. In the example of FIG. _21A , the sum of the weight w _{right — 6} and the weight w _{left — 6} is 1.0.

FIG. 21B _corresponds to a ratio ( _{dright_3} / _{dleft_3} ) that is a value obtained by dividing (dividing) _{dright_3} by _{dleft_3 in} the case of _{dleft_3} ≧ _{dright_3} and the right likelihood λ _{r_eylid} It is a _{figure showing} the relationship with the magnitude _| size of weight _{wright_6 to perform} , and the relationship between ratio ( _{dright_3} / _{dleft_3} ) and the magnitude _| size of the weight _{wleft_6} corresponding to the left eyelid likelihood (lambda) _{l_eyelid} .

Here, when the ratio (d _{right — 3} / d _{left — 3} ) is 1, it is _{considered that} the accuracy (reliability) of the right _hail likelihood λ _{r —} eyelid is the same as the accuracy of the left _hail likelihood λ _{l —} eyelid. Can do.

On the other hand, when the ratio _{(d right_3 /} d _{left_3)} is smaller than 1, the accuracy of the right eyelid likelihood lambda _{R_eyelid} can be considered worse than the precision of the left eyelid likelihood _λ l_eyelid. Then, it is considered that the accuracy of the right _eye likelihood λ _{r_eyelid} becomes worse than the accuracy of the left _eye likelihood λ _{l_eyelid} as the ratio (d _{right — 3} / d _{left — 3} ) becomes smaller than 1.

In the present embodiment, in consideration of these _points , the weight of the overall likelihood is reduced by reducing the weight of the likelihood that the accuracy becomes worse as the ratio ( _{dright_3} / _{dleft_3} ) becomes smaller than 1. The _ratio of the _heel likelihood of the _heel having the lower (bad) accuracy to the likelihood λ _{eyelid_all} is made smaller. As a result, the overall likelihood λ _{eyelid_all} can be calculated with better accuracy.

In step _320, if it is _{d left_3} ≧ _{d right_3,} as shown in FIG. 21 (B), the distance _{d Right_3} the inner corner and the corner of the eye of the right eye, divided by the distance _{d Left_3} the left eye inside corner and corner of the eye A ratio (d _{right — 3} / d _{left — 3} ), which is a value, is calculated to determine the corresponding weight w _{right — 6} and weight w _{left — 6} . More specifically, for example, when the ratio ( _{dright_3} / _{dleft_3} ) in the case of _{dleft_3} ≧ _{dright_3} is 0.75, the weight _{wright_6} is 0.25, and the weight _{wleft_6} is 0.75. Determine. Then, the sum λ _eyelid is calculated as described above. In the example of FIG. 21B as well, the sum of the weight w _{right — 6} and the weight w _{left — 6} is 1.0.

As described above, the eyelid likelihood computation device according to this embodiment, depending on the distance d _{Left_3} between the distance d _{Right_3} and left eye inner canthus and crow's feet and the detected right eye inner canthus and eye corner, by the camera 12 The size corresponding to the likelihood of the eyelid farther to the position where the image was photographed by the camera 12 among the starboard and the left eyelid of the captured image is larger than the size corresponding to the eye likelihood of the closer eyelid. determined to be smaller, the weight _{w Right_6} corresponding to the right of the eyelid of the weights _{w Right_6} and _{w Left_6} corresponding to each of the right and left eyelids, right comparison eyelid models and comparison of the left eyelid opening is the same The right comparison _瞼 model in the combination of _瞼 models The right _瞼 likelihood λ _{r_eyelid} of the model is weighted (w _right — ₆ · λ _{r_eylid} ) and the weight corresponding to the left _heel Only _{w left_6,} the sum of the left eyelid likelihood λ value was weighted to _{l_eyelid} of the left comparison eyelids model of _{(w left_6 · λ l_eyelid) (} (w right_6 · λ r_eyelid + w left_6 · λ l_eyelid) / (w right_6 + w _{Left_6} ))) is calculated for each combination as left and right _heel likelihood λ _eyelid in this combination. The eyelids likelihood computation device according to the present embodiment, among the left and right eyelid likelihood lambda _eyelid of each combination in which a plurality computation, calculates the largest right and left eyelids likelihood lambda _{Eyelid_max} the overall Mabutayudo lambda _{Eyelid_all} To do.

According to the heel likelihood calculating apparatus according to the present embodiment, as in the first to fifth embodiments, the ratio of the heel likelihood of the _heel having a lower accuracy with respect to the overall heel likelihood λ _{eyelid_all} is smaller. Thus, the overall heel likelihood λ _{eyelid_all} can be calculated with better accuracy.

Note that the overall heel likelihood λ _{eyelid_all} calculated by the heel likelihood calculating device of the present embodiment can be used as a likelihood when estimating the shape of the _heel , for example. In addition, for example, the eyelid opening degree can be detected from the shape of the eyelid estimated in this way.

In addition, the heel likelihood calculating apparatus according to the present embodiment estimates a _heel model based on the calculated total heel likelihood λ _{eyelid_all} , and from the estimated _heel model, _cleaves the heel _represented by the heel model. It is good also as a structure provided with the eaves opening degree detection function which detects a degree. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Seventh Embodiment]
Next, a seventh embodiment will be described. In addition, about the structure similar to each 1st-6th embodiment and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted.

In this embodiment, first, similarly to the first to sixth embodiments described in detail above, a set of weight w _right and weight w _left, a set of weight w _{right_2} and weight w _{left_2} , and weight w _{A set of right_3} and weight w _{left — 3,} a set of weight w _{right — 4} and weight w _{left — 4,} a set of weight w _{right — 5} and weight w _{left — 5, and} a set of weight w _{right — 6} and weight w _{left — 6} are defined.

Hereinafter, the weight _w. Right corresponding to the right of the _{eyelid, w right_2 ~w right_6,} and weights _w left corresponding to the left _eyelid, w _{Left_2} when determining the _{~w left_6} (i.e., first to sixth embodiments described above The case where all the weights (6 sets of weights) are determined) will be described as an example. In this case, the wrinkle likelihood calculation apparatus according to the present embodiment performs detection means (first detection means) for detecting the face orientation angle θ of the face in the same manner as in the first embodiment, and the second implementation. form and in the same manner as detection means for detecting a distance _{d left} distance to _{d. right} and left to right (second detection means), a third to distance _{d Right_2} and to the right ear as in the embodiments It detects the distance d _{Left_2} to the left ear detecting means (third detection unit), a sixth embodiment similar manner the distance d _{Right_3} and left the corner of the eye of the right eye of the inner corner of the eye and right eye and the inner corner and the left eye corner of the eye Starring for calculating a contrast _{Iw left} detecting means (the fourth detection means), the contrast _{Iw. right} and left eye of the eye region of the fourth right-eye of the eye region in the same manner as the embodiment of detecting the distance _{d Left_3} the A calculation means (first calculation means) and a calculation means (second calculation means) for calculating the edge intensity E w _{right of the right} eye region and the edge intensity E w _left of the _left eye region in the same manner as the fifth embodiment. It is comprised including the processing means.

Then, when calculating the weight of 6 pairs, as shown in the following equation (13), in this embodiment, define the weights _{w Right_7} for weighting the right eyelid likelihood _λ r_eyelid.

However,
_{sum: a 1 · w right ×} a 2 · w right_2 × a 3 · w right_3 × a 4 · w right_4 × a 5 · w right_5 × a 6 · w right_6 + a 1 · w left × a 2 · w left_2 × a ₃・ w _{left_3} × a ₄・ w _{left_4} × a ₅・ w _{left_5} × a ₆・ w _{left_6}
It is.

It will now be described weighting factor _a 1 ~a _6. When six sets of weights are defined, the condition of a ₁ + a ₂ + a ₃ + a ₄ + a ₅ + a ₆ = 1 is set, and when the weighting factors a _{1 to} a ₆ are changed variously, the lid opening degree detection performance Select the value that maximizes. Note that the maximum lid opening detection performance means, for example, a case where the average lid opening error average is minimized. For example, in the above description, the case where six sets of weights are defined has been described, but when only two sets of weights are defined (for example, when the weights w _right , w _{right_2} , and the weights w _left , w _{left — 2} are defined). For example, the condition of a ₁ + a ₂ = 1 is set. The value of the weighting factor a ₁ in this case, the relationship between the eyelid opening error average becomes the relationship shown in FIG. 22. In the example of FIG. 22, since MabutaHiraki degree average error at the time of the weighting factor _a 1 = 0.6 is minimized, the weight coefficient _{a 1 = 0.6, a 1 =} 0.4. When six sets of weights are defined, selection is made for each value of the weight coefficients a _{1 to} a _{6 on} the same principle.

Then, when the weight of 6 pairs have been established, as shown in the following equation (14), in this embodiment, define the weights _{w Left_7} for weighting the left eyelid likelihood _λ l_eyelid.

However, sum and weighting factors a _{1 to} a ₆ are the same as those described above.

The weights w _{right — 7} and w _{left — 7} determined in this way correspond to the likelihood of the farther farther to the position where the image was captured by the camera 12, of the _starboard and _port side of the image captured by the camera 12. The weight corresponding to each of the left and right eyelids is determined so that the size to be smaller than the size corresponding to the eyelid likelihood of the near eyelid.

Then, the weight w _{Right_7} corresponding to the right of the eyelid, similar to the first to sixth embodiments described above, the right in the combination of the right comparison eyelid model and left comparison eyelid model eyelid opening is the same a comparison eyelids model of the right of the eyelids likelihood _λ r_eyelid to the weighting was value _{(w right_7} · _λ r_eyelid), the weights _{w left_7} corresponding to the left of the eyelids, weighted to the left of the eyelids likelihood _λ l_eyelid of the left comparison eyelids model of value _{(w left_7} · λ _{l_eyelid)} the sum of the _{((w right_7 · λ r_eyelid +} w left_7 · λ l_eyelid) / (w right_7 + w left_7)) multiple operations for each combination as eyelid likelihood lambda _eyelid of the left and right in this combination To do.

In other words, the distance between the detected face direction angle theta, the distance to the distance _{d. Right} and left to right _{d left,} the distance to the distance _{d Right_2} and left ear to the right ear _{d Left_2,} the right inner corner and the right corner of the eye the distance between d _{Right_3} and left eye inner canthus and left in eye corner _{d Left_3,} computed right eye of the eye region in contrast _{Iw. right} and contrast _{Iw left} of the left eye of the eye region, and the right eye region of the edge strength _{Ew. right} and the edge of the left eye area Depending on the intensity E w _left , the size corresponding to the likelihood of the eyelid farther from the position where the image was photographed by the camera 12 out of the starboard and portside of the image photographed by the camera 12 is closer. Weights corresponding to each of the left and right wings, determined to be smaller than the size corresponding to the wing likelihood of the heel Among w right — ₇ and w _{left — 7} , the weight w right — 7 corresponding to the right eyelid is set to the right eyelid _likelihood of the right comparison eyelid model in the combination of the right eyelid opening model and the left eyelid eyelid model having the same eyelid opening degree. the degree _λ r_eyelid to the weighting was value _{(w right_7} · _λ r_eyelid), the weights _{w left_7} corresponding to the left of the eyelids, weighted values to the left of the eyelids likelihood _λ l_eyelid of the left comparison eyelids model of _{(w left_7} · _λ l_eyelid ) ((W _right — ₇ · λ _r — _eyelid + w _left — ₇ • λ _{l — eyelid} ) / (w _right — ₇ + w left — ₇ )) is calculated for each combination as a plurality of left and right likelihoods λ _eyelid .

Similarly to the first to sixth embodiments described above, the heel likelihood calculating apparatus according to the present embodiment has the largest left and right _heel likelihoods among the left and right _heel likelihoods λ _eyelid for each of a plurality of combinations. calculating a likelihood lambda _{Eyelid_max} the overall Mabutayudo _λ eyelid_all.

According to the heel likelihood calculating apparatus according to the present embodiment, as in the first to sixth embodiments, the ratio of the heel likelihood of the _cocoon having the lower accuracy with respect to the overall heel likelihood λ _{eyelid_all} is smaller. Thus, the overall heel likelihood λ _{eyelid_all} can be calculated with better accuracy.

In the present embodiment, an example of calculating six sets of weights has been described. However, by configuring the 瞼 likelihood calculation device of the present embodiment as described below, at least one set of weights is calculated. You may make it do. That is, the heel likelihood calculating apparatus of the present embodiment includes the first detecting means, the second detecting means, the third detecting means, the fourth detecting means, the first calculating means, and The processing means including at least one means of the second calculation means includes a detected face orientation angle θ, a distance d _{right to the right} eye, a distance d _{left to the left} eye, and a distance to the right ear. d _{Right_2} and distance to the left ear _{d left_2,} the distance between the distance _{d Right_3} and left eye inner canthus and left of crow's feet and crow's feet right eye inner canthus and right _{d left_3,} contrast _{Iw. right} and left of the computed right eye area contrast _{Iw left} in the eye region, and in response to at least one of the edge intensities _{Ew left} edge intensity _{Ew. right} and left regions of the right eye area The size corresponding to the likelihood of the eyelid farther from the position where the image was taken by the camera 12 among the starboard and the left eyelid of the image taken by the camera 12 is the eye likelihood of the closer eyelid. determined to be smaller than the corresponding size, of the weights _{w Right_7} and _{w Left_7} corresponding to each of the right and left eyelids, the weights _{w Right_7} corresponding to the right of the eyelid, the right comparison eyelid eyelid opening is the same The right comparison likelihood model λ _{r_eyelid} with the weight (w _right — ₇ · λ _{r —} eyelid) and the weight w left — ₇ corresponding to the left _eye in the combination of the model and the left comparison _eye model with the left comparison the sum of the weighted values to the left of the eyelids likelihood _λ l_eyelid of the eyelid model _{(w left_7 · λ l_eyelid) (} (w right_7 · λ r_ey _{lid + w left_7 · λ l_eyelid)} / (w right_7 + w left_7)) may be more operations to each combination as left and right eyelid likelihood lambda _eyelid in this combination.

In the case of calculating the weight of the n (n ≧ 1) _sets, set a condition that _{a 1 + ··· + a n =} 1, each of the weight coefficients _a 1 ~a _n similar to the above principle To decide. At this time, the "sum" is _{"a 1 · w right × ··· ×} a n · w right_n + a 1 · w left × ··· × a n · w left_n ".

Note that the overall heel likelihood λ _{eyelid_all} calculated by the heel likelihood calculating device of the present embodiment can be used as a likelihood when estimating the shape of the _heel , for example. In addition, for example, the eyelid opening degree can be detected from the shape of the eyelid estimated in this way.

In addition, the heel likelihood calculating apparatus according to the present embodiment estimates a _heel model based on the calculated total heel likelihood λ _{eyelid_all} , and from the estimated _heel model, _cleaves the heel _represented by the heel model. It is good also as a structure provided with the eaves opening degree detection function which detects a degree. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Eighth Embodiment]
Next, an eighth embodiment will be described. In addition, about the structure similar to each 1st-7th embodiment and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted.

  In the present embodiment, the difference from the first embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 23 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and the CPU 14d executes the wrinkle likelihood calculation process shown in FIG. As shown in the figure, when an affirmative determination is made in step 120, step 322 is executed, step 322 is executed after step 322, and step 326 is executed after step 324.

First, the process at step 322 will be described. In step 322, the right _瞼 likelihood λ _{r_eyelid_max} having the maximum value is selected from the right _瞼 likelihoods λ _{r_eyelid} calculated in step 112 and the left _瞼 likelihood calculated in steps 118 is selected. Among the degrees λ _{l_eyelid} , the left maximum likelihood λ _{l_eyelid_max} is selected.

Next, the process at step 324 will be described. In step 324, as in the first embodiment, the weight w _right and the weight w _left are determined, and the weight w _right is weighted to the right _heel likelihood λ _{r_eyelid_max} (w _right · λ _{r_eyelid_max} ), and the weight A value _obtained by weighting w _left to the left eyelid likelihood λ _{l_eyelid_max} (w _left · λ _{l_eyelid_max} ) is compared, and the larger value is selected.

  Next, the process at step 326 will be described. In step 326, the value selected in step 324 is used as the 瞼 likelihood of the selected heel (right heel or left heel), thereby calculating the heel likelihood of the selected heel. To do.

  According to the heel likelihood calculating apparatus according to the present embodiment, only the heel likelihood of the cocoon with higher accuracy is calculated, so that the heel likelihood can be calculated with better accuracy.

In this embodiment, an example has been described for determining the first weight _{w. Right} and weights _{w left} similarly to the embodiment, similarly to the second embodiment defines the weights _{w Right_2} and weights _{w Left_2} You may do it. Similarly, among the third to seventh embodiments, weights corresponding to the left and right wings (瞼 likelihood) may be determined in the same manner as any one of the embodiments. For example, the heel likelihood calculation device of the present embodiment may be configured as described below. That is, the heel likelihood calculating apparatus of the present embodiment includes the first detecting means, the second detecting means, the third detecting means, the fourth detecting means, the first calculating means, and The processing means including at least one means of the second calculation means includes a detected face orientation angle θ, a distance d _{right to the right} eye, a distance d _{left to the left} eye, and a distance to the right ear. d _{Right_2} and distance to the left ear _{d left_2,} the distance between the distance _{d Right_3} and left eye inner canthus and left of crow's feet and crow's feet right eye inner canthus and right _{d left_3,} contrast _{Iw. right} and left of the computed right eye area According to at least one of the contrast Iw _{left of} the eye region, the edge strength Ew _{right of the right} eye region, and the edge strength Ew _left of the _left eye region The size corresponding to the likelihood of the eyelid farther from the position where the image was taken by the camera 12 among the starboard and the left eyelid of the image taken by the camera 12 is the eye likelihood of the closer eyelid. determined to be smaller than the corresponding size, of the weights _{w Right_7} and _{w Left_7} corresponding to each of the right and left eyelids, weighted values of weights _{w Right_7} the right eyelid likelihood _{λ r_eyelid_max (w right_7 · λ r_eyelid_max} ) , And the weight w left — ₇ weighted to the left _heel likelihood λ _{l_eyelid_max} (w _left — ₇ · λ _{l_eylid_max} ), the larger value is selected, and the selected value is compared with the selected _瞼 ( Calculate the likelihood of the selected heel by taking the heel likelihood of the right heel or the left heel) Unishi may be.

  The cocoon likelihood of the cocoon with higher accuracy calculated in this way can be used, for example, as the likelihood when estimating the shape of the cocoon with higher accuracy. Further, for example, it is possible to detect the eyelid opening with high accuracy from the shape of the eyelid thus estimated.

  Further, the heel likelihood calculating apparatus according to the present embodiment estimates a heel model based on the calculated heel likelihood, and detects the heel opening degree of the heel represented by the heel model from the estimated heel model. It is good also as a structure provided with the eaves opening degree detection function. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Ninth Embodiment]
Next, a ninth embodiment will be described. In addition, about the structure similar to each 1st-8th embodiment and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted. The heel likelihood calculating device of the present embodiment has a heel opening degree detecting function for detecting the heel opening degree, and can be used as a heel opening degree detecting device.

  In the present embodiment, the difference from the eighth embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 23 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and a program for executing the processing routine of the eyelid opening degree determination process shown in FIG. 24 is stored, and the CPU 14d of the eyelid likelihood calculating device according to the present embodiment performs the operation shown in FIG. This is the point at which the likelihood calculation process is executed and the heel opening determination process shown in FIG. 24 is executed.

  With reference to FIG. 24, the lid opening degree determination process will be described. First, in step 600, the face orientation angle θ detected in step 106 of the heel likelihood calculation process is 0 degrees (the angle when facing the front is 0 degrees) or (0 ± β) degrees. It is determined whether it is within the range. Thereby, it can be determined whether or not the face is facing the front (when θ = 0 degrees) or the face is almost facing the front (when θ is in the range of 0 ± β degrees).

  If it is determined in step 600 that the face angle θ is 0 degrees (or in the range of 0 ± β degrees), the process proceeds to the next step 606.

  On the other hand, if it is determined in step 600 that the face angle θ is not 0 degree (or not within the range of 0 ± β degrees), the process proceeds to the next step 602.

  Step 600 may be omitted, and the eyelid opening determination process may be started from the next step 602.

In the next step 602, it is Meyudo processing the weighted value in step _{324 (w right} · λ r_eyelid_max) (see FIG. 23) is greater than a predetermined reference value alpha, and was weighted at step 324 It is determined whether or not the value (w _left · λ _{1_eyelid_max} ) is equal to or greater than a predetermined reference value α. By determining in this way, it is determined whether or not the (comparison) wing model corresponding to each of the weighted (ie, corrected) starboard likelihood and portside likelihood is reliable. Judgment can be made.

If it is determined in step 602 that (w _right · λ _{r_eyelid_max} ) is greater than or equal to the predetermined reference value α and (w _left · λ _{l_eyelid_max} ) is greater than or equal to the predetermined reference value α, the corresponding both It is determined that each comparison model is reliable, and the process proceeds to the next step 604.

In step 604, the comparison model is _{cleaved from the starboard} comparison model corresponding to the weighted _right model likelihood ( _wright · _{λr_eyelid_max} ) determined to be equal to or greater than the predetermined reference value α. Degree (opening degree of starboard) o _right is detected. Similarly, from the left comparison _瞼 model corresponding to the weighted _{left heel} likelihood (w _left · λ _{l_eyelid_max} ) determined to be equal to or greater than the predetermined reference value α, the comparison _瞼 model _瞼 Opening degree (left side opening degree) o _left is detected. Then, it calculates the eyelid opening _{o. Right} and eyelid opening _{o left} of the ratio of Hidarimabuta of the detected right eyelid _{(e.g., o} right _{/ o left).} The calculated ratio is stored in the RAM 14e or the HDD 14c. Thereby, it is possible to calculate the ratio of the left and right eyelid opening detected when both the right and left eyelid models are highly reliable. The ratio of the left and right eyelid opening is, for example, as described later, when only one of the starboard eyelid model and the left eyelid eyelid model is highly reliable, the eyelid opening degree of the unreliable eyelid model Can be used when it is desired to accurately detect. More specifically, the heel opening degree of the heel represented by this heel model is detected from the heel model that is considered to be highly reliable, and the reliability is considered to be low from the detected heel opening degree and ratio. By calculating the eyelid opening of the other eyelid, the eyelid opening degree of the eyelid corresponding to the eyelid model with low reliability can be detected accurately.

  In the next step 606, it is determined whether or not the ratio is stored in the RAM 14e or the HDD 14c.

  If it is determined in step 606 that the ratio is stored, the process proceeds to the next step 608.

  In step 608, the ratio is read from the RAM 14e or the HDD 14c. Then, the eyelid opening degree of the eyelid represented by the eyelid model is detected from the eyelid model of the eyelid on which the eyelid likelihood is calculated in step 326 of the eyelid likelihood calculation process. Then, from the detected saddle opening degree and the read ratio, the saddle opening degree of the lower saddle likelihood is detected by calculating the saddle opening degree of the saddle whose side is not detected. To do.

  In the next step 610, the opening degree of the both eyelids detected in step 608 is output to an output device (not shown) (for example, a display device or an audio output device). Then, the lid opening degree detection process is terminated. Thereby, the saddle opening degree of both sides with good accuracy is outputted from the output device. For example, when the output device is a display device, the eyelid opening degree of both sides is displayed, and when the output device is an audio output device, the eyelid opening degree of both sides is output by voice.

  On the other hand, if it is determined in step 606 that the ratio is not stored, the process proceeds to the next step 612.

  In step 612, the eyelid opening degree of the eyelid represented by the eyelid model is detected from the eyelid model of the eyelid on which the eyelid likelihood is calculated in step 326 of the eyelid likelihood calculation process. Then, the detected heel opening is output to an output device (not shown) (for example, a display device or a sound output device). Thereby, the eaves opening degree is output from the output device. Then, the lid opening degree detection process is terminated.

The heel likelihood calculating device having the heel opening degree detection function of the present embodiment is a value _obtained by weighting the weight w _right corresponding to the right _heel to the maximum right _heel likelihood λ _{r_eyelid_max} (w _right · λ _{r_eyelid_max} ) , And the weight w _left corresponding to the left _heel is weighted to the maximum left _heel likelihood λ _{l_eyelid_max} (w _left · λ _{l_eyelid_max} ) is equal to or greater than a predetermined reference value, the maximum right heel likelihood _{Detect and detect starboard heel} opening o _right represented by right comparison _heel model with degree λ _{r_eylid_max} and left _heel opening o _left represented by left comparison _heel model with maximum left _heel likelihood λ _{l_eyelid_max} have been the right eyelid eyelid opening _{o. right} and eyelid opening _{o left} in the ratio of Hidarimabuta _{(e.g., o} right _{/ o left)} And the heel opening degree of the heel represented by the comparison heel model of the heel likelihood calculated in step 326 is detected, and the heel opening degree is not detected based on the detected heel opening degree and ratio. By calculating the eyelid opening degree of the one eyelid, the eyelid opening degree of the eyelid having the lower eyelid likelihood is detected.

  Thereby, the eyelid opening degree of the eyelid corresponding to the eyelid model with high accuracy and low reliability can be detected.

In this embodiment, an example has been described for determining the first weight _{w. Right} and weights _{w left} similarly to the embodiment, similarly to the second embodiment defines the weights _{w Right_2} and weights _{w Left_2} You may do it. Similarly, among the third to eighth embodiments, a weight corresponding to each of the left and right wings (瞼 likelihood) may be determined as in any of the embodiments. For example, the heel likelihood calculation device of the present embodiment may be configured as described below. That is, the heel likelihood calculating apparatus of the present embodiment includes the first detecting means, the second detecting means, the third detecting means, the fourth detecting means, the first calculating means, and The processing means including at least one means of the second calculation means includes a detected face orientation angle θ, a distance d _{right to the right} eye, a distance d _{left to the left} eye, and a distance to the right ear. d _{Right_2} and distance to the left ear _{d left_2,} the distance between the distance _{d Right_3} and left eye inner canthus and left of crow's feet and crow's feet right eye inner canthus and right _{d left_3,} contrast _{Iw. right} and left of the computed right eye area According to at least one of the contrast Iw _{left of} the eye region, the edge strength Ew _{right of the right} eye region, and the edge strength Ew _left of the _left eye region The size corresponding to the likelihood of the eyelid farther from the position where the image was taken by the camera 12 among the starboard and the left eyelid of the image taken by the camera 12 is the eye likelihood of the closer eyelid. determined to be smaller than the corresponding size, of the weights _{w Right_7} and _{w Left_7} corresponding to each of the right and left eyelids, weighted values of weights _{w Right_7} the right eyelid likelihood _{λ r_eyelid_max (w right_7 · λ r_eyelid_max} ) , And the weight w left — ₇ weighted to the left _heel likelihood λ _{l_eyelid_max} (w _left — ₇ · λ _{l_eylid_max} ), the larger value is selected, and the selected value is compared with the selected _瞼 ( Calculate the likelihood of the selected heel by taking the heel likelihood of the right heel or the left heel) Unishi may be. Then, the _瞼 likelihood calculation device according to the present embodiment has a weight ( _{wright_7} · _{λr_eyelid_max} ) _obtained by weighting the weight _{wright_7} corresponding to the right _heel to the maximum right _heel likelihood _{λr_eyelid_max} , and the left _瞼 When the value (w _left — ₇ · λ _{l — eyelid_max} ) _obtained by weighting the weight w left — ₇ corresponding to the maximum left _eye likelihood λ _{l_eyelid_max} is equal to or greater than a predetermined reference value, the maximum right _eye likelihood λ _{r_eyelid_max} The _starboard eyelid opening o _right indicated by the right comparison eyelid model and the left eyelid opening degree o _left indicated by the left eyelid likelihood λ _{l_eyelid_max} of the maximum left eyelid likelihood λ _{l_eyelid_max} are detected, and the detected _starboard The ratio of the heel opening degree o _right of the heel and the left heel opening degree o _left (eg, o _right / o _left) ) And the heel opening degree of the heel represented by the comparison heel model of the heel likelihood calculated in step 326 is detected, and the heel opening degree is detected based on the detected heel opening degree and ratio. It is also possible to detect the eyelid opening degree of the eyelid with the lower eyelid likelihood by calculating the eyelid opening degree of the eyelid that is not.

[Tenth embodiment]
Next, a tenth embodiment will be described. In addition, about the structure similar to each 1st-9th embodiment and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted. For example, as shown in FIG. 25, when a person 50 who is a subject to be photographed by the camera 12 wears spectacles 52, the heel likelihood calculating apparatus according to the present embodiment reflects light by the lens 54 of the spectacles 52. Thus, when the luminance of the image 58 of the eye region portion of the photographed person image 56 is increased, the heel likelihood can be calculated accurately when the heel likelihood of the heel cannot be calculated accurately. Only the 瞼 likelihood of 瞼 is calculated. For example, in the example of FIG. 25, light is reflected from the lens 54 on the left side of the glasses 52, and the luminance of the left eye region is high. In such a case, only the starboard likelihood likelihood is calculated.

  In this embodiment, the difference from the eighth embodiment is that a program for executing the processing routine of the 瞼 likelihood calculation process shown in FIG. 26 is stored in the HDD 14c of the 瞼 likelihood calculation device according to this embodiment. Is stored, and the CPU 14d executes the likelihood likelihood calculation process shown in FIG. As shown in the figure, step 328 is executed after step 108, step 330 is executed after step 328, and if an affirmative determination is made in step 330, the process proceeds to step 110, and negative in step 330. If the determination and affirmative determination is made in step 114, step 332 is executed. If an affirmative determination is made in step 332, the process proceeds to step 116. On the other hand, if a negative determination is made in step 332 and an affirmative determination is made in step 120, step 334 is executed. If an affirmative determination is made in step 334, the next step 322 is executed. On the other hand, if a negative determination is made in step 334, step 336 is executed. If an affirmative determination is made in step 336, then step 338 is executed, step 338 is followed by step 340, and the process proceeds to step 326. On the other hand, when a negative determination is made in step 336, step 342 is executed next. When an affirmative determination is made in step 342, step 344 is executed next, and after step 344, step 346 is executed. And proceed to step 326. If a negative determination is made in step 342, the heel likelihood calculation process ends.

  First, the process at step 328 will be described. In step 328, high brightness pixel region area ratio calculation processing is executed.

  With reference to FIG. 27, the high-luminance pixel region area ratio calculation process will be described. First, in step 700, the number of pixels in the right eye region among the eye regions extracted in step 104 of the heel likelihood calculation process (see FIG. 26) is specified.

  In the next step 702, as shown in FIG. 28, for the image data of the image 58 of the right eye region among the eye regions extracted in step 104, a predetermined threshold value K (for example, K = 200) is used. A binarization process is performed in which the pixel value of a pixel having a large pixel value is set to 255, and the pixel value of a pixel having a pixel value equal to or less than a predetermined threshold K is set to 0. As a result, each pixel of the image 58 in the right eye region becomes a black pixel (pixel having a pixel value of 255) and a white pixel (pixel having a pixel value of 0).

In the next step 704, the ratio of the number of white pixels obtained by the binarization process in step 702 to the number of pixels in the right eye area specified in step 700 is set to a first ratio (the right eye area height). (Luminance pixel area area ratio) _Rright is calculated as a first ratio (ratio of high-luminance pixel area area of the right eye area) that is a ratio of the number of pixels equal to or higher than a predetermined luminance (that is, threshold value K) to the number of pixels of the right eye area. ) Calculate R _right . In the example of FIG. 28, 0 (0%) is calculated as the high luminance pixel region area ratio R _right of the _right eye region.

  In the next step 706, the number of pixels of the left eye region among the eye regions extracted in step 104 is specified.

  In the next step 708, among the eye regions extracted in step 104, the pixel data having a pixel value larger than a predetermined threshold K (for example, K = 200) with respect to the image data of the image 58 of the left eye region. A binarization process is performed in which the pixel value is set to 255 and the pixel value of a pixel having a pixel value equal to or less than a predetermined threshold K is set to 0. As a result, the image 58 in the left eye region becomes a black pixel (pixel having a pixel value of 255) and a white pixel (pixel having a pixel value of 0).

In the next step 710, the ratio of the number of white pixels obtained by the binarization process in step 708 to the number of pixels in the left eye area specified in step 706 is set to a second ratio (the left eye area height). (Luminance pixel area area ratio) R _left , by calculating the second ratio (high luminance pixel area area ratio of the left eye region), which is a ratio of the number of pixels equal to or higher than a predetermined luminance (that is, threshold value K) to the number of pixels of the left eye region ) Calculate R _left . In the example of FIG. 28, 0.54 (54%) is calculated as the high luminance pixel region area ratio R _left of the _left eye region. Then, the high luminance pixel region area ratio calculation process is terminated.

  The high brightness pixel region area ratio calculation process executed in step 328 has been described above.

In the next step 330, the high luminance pixel region area ratio R _{right of the right} eye region calculated in step 704 is compared with a predetermined value γ (eg, 0.4 (40%)), and the high luminance pixel of the right eye region is compared. It is determined whether or not the area ratio R _right is equal to or less than a predetermined value γ. If it is determined that the high-luminance pixel region area ratio R _{right of the right} eye region is equal to or less than the predetermined value γ, it can be determined that the heel likelihood can be accurately calculated from the image 58 of the right eye region. . On the other hand, if it is determined that the high-luminance pixel region area ratio R _right of the right eye region is larger than the predetermined value γ, it can be determined that it is impossible to calculate the wrinkle likelihood with accuracy from the image 58 of the right eye region.

If it is determined in step 330 that the high luminance pixel region area ratio R _right of the right eye region is equal to or smaller than the predetermined value γ, the process proceeds to step 110. On the other hand, if it is determined in step 330 that the high-luminance pixel region area ratio R _right of the right eye region is larger than the predetermined value γ, the process proceeds to step 332. By proceeding to step 332, the processing of steps 110 to 114 can be omitted without executing.

In step 332, by comparing the high-brightness pixel region area ratio of the computed eye region in Step 710 _{R left} and the predetermined value gamma (e.g. 0.4 (40%)), high-brightness pixel region area of the left eye area It is determined whether the ratio R _left is less than or equal to a predetermined value γ. If the high-brightness pixel region area ratio R _left in the eye region is determined to be equal to or less than the predetermined value γ, it can be determined from the image 58 of the left eye region and it is possible to calculate accurately Mabutayu degree. On the other hand, when a high-luminance pixel region area ratio R _left in the eye region is determined to be greater than the predetermined value γ, it can be determined that it is not possible to computed accurately Mabutayu degree from the image 58 of the eye region.

In step 332, if the high-brightness pixel region area ratio R _left in the eye region is determined to be equal to or less than the predetermined value γ, the routine proceeds to step 116. On the other hand, if it is determined in step 332 that the high-luminance pixel region area ratio R _leftt of the left eye region is greater than the predetermined value γ, the process proceeds to step 334. By proceeding to step 334, the processing of steps 116 to 120 can be omitted without executing.

In step 334, it is determined whether or not the high-luminance pixel region area ratio R _{right of the right} eye region is less than or equal to a predetermined value γ and the high-luminance pixel region area ratio R _{left of the left} eye region is less than or equal to a predetermined value γ.

If it is determined in step 334 that the high-brightness pixel region area ratio R _{right of the right} eye region is equal to or less than the predetermined value γ and the high-brightness pixel region area ratio R _{leftt of the left} eye region is equal to or less than the predetermined value γ, Proceed to step 322.

On the other hand, if it is determined in step 334 that the high luminance pixel area area ratio R _{right of the right} eye area is not more than the predetermined value γ and the high luminance pixel area area ratio R _{left of the left} eye area is not less than the predetermined value γ. Go to step 336.

In step 336, it is determined whether or not the high-luminance pixel region area ratio R _{right of the right} eye region is equal to or less than a predetermined value γ. If it is determined in step 336 that the high-luminance pixel region area ratio R _right of the right eye region is equal to or smaller than the predetermined value γ, the process proceeds to step 338.

In step 338, among the plurality computed right eyelid likelihood lambda _{R_eyelid} in step 112, it selects the right eyelid likelihood lambda _{R_eyelid_max} maximum value.

In the next step 340, as in the eighth embodiment (1), the weight w _right is determined, and the value (w _right · λ _{r_eylid_max} ) _obtained by weighting the weight w _right to the right _＿ likelihood λ _{r_eyelid_max} is selected. To do. Then, the process proceeds to step 326.

On the other hand, if it is determined in step 336 that the high-luminance pixel region area ratio R _right of the right eye region is greater than the predetermined value γ, the process proceeds to step 342.

In step 342, it is determined whether or not the high-luminance pixel region area ratio R _{left of the left} eye region is equal to or less than a predetermined value γ. If it is determined in step 342 that the high-luminance pixel region area ratio R _leftt of the left eye region is equal to or smaller than the predetermined value γ, the process proceeds to step 344.

In step 344, among the eyelid likelihood lambda _{L_eyelid} left in which a plurality calculated at step 118, selects the left eyelid likelihood lambda _{L_eyelid_max} maximum value.

In the next step 346, as in the eighth embodiment (1), the weight w _left is determined, and the value _obtained by weighting the weight w _left to the left _heel likelihood λ _{l_eyelid_max} (w _left · λ _{l_eyelid_max} ) is selected. To do. Then, the process proceeds to step 326.

On the other hand, in step 342, if the high-brightness pixel region area ratio R _left in the eye region is determined to be greater than the predetermined value γ it ends the Mabutayudo processing.

As described above, the eyelid likelihood calculation device of this embodiment, (high-brightness pixel region area ratio of the right eye region) first rate high-brightness pixel area of the region _{R. Right} and a second fraction (left eye area The ratio R _left ) is calculated, and the right _heel likelihood λ _{r_eyelid} is calculated only when the first ratio is less than or equal to the predetermined value γ, and the _{left heel} likelihood only when the second ratio is less than or equal to the predetermined value γ. The degree λ _{l_eyelid} is calculated. Then, the heel likelihood calculation device according to the present embodiment is the farther to the position where the image is captured by the camera 12 among the starboard and port side of the image captured by the camera 12 according to the face orientation angle θ. magnitude corresponding to the eyelid likelihood of eyelid, was determined to be smaller than the size corresponding to the eyelid likelihood of eyelid closer, among the weights _{w. right} and w _left corresponding to each of the left and right eyelid, the When the ratio of 1 is less than or equal to the predetermined value γ and the second ratio is less than or equal to the predetermined value γ, the weight w _right corresponding to the right eyelid is weighted to the right eyelid likelihood λ _{r_eyelid_max} (w _{right) ·} λ r_eyelid_max), and the magnitude of the weights _{w left} and was weighted to the left of the eyelid likelihood lambda _{L_eyelid_max} value corresponding to the left eyelid _{(w left} · _λ l_eyelid_max) When only one of the first ratio and the second ratio is equal to or less than the predetermined value γ, the 瞼 (first ratio or first ratio) for which the 瞼 likelihood is calculated is selected. The maximum _瞼 likelihood (right _瞼 likelihood λ _{r_eyelid_max} or left _瞼 likelihood λ _{l_eyelid_max} ) of the _{な る} whose ratio of 2 is less than or equal to a predetermined value is _assigned a weight (weight w _right or weight) corresponding to this _瞼 A value obtained by weighting w _left ) ((w _right · λ _{r_eyelid_max} ) or (w _left · λ _{l_eyelid_max} )) is selected. As a result, it is possible to calculate the heel likelihood more easily without calculating the heel likelihood of the heel that cannot calculate the heel likelihood with high accuracy by a complicated operation.

In this embodiment, an example has been described for determining the first weight _{w. Right} and weights _{w left} similarly to the embodiment, similarly to the second embodiment defines the weights _{w Right_2} and weights _{w Left_2} You may do it. Similarly, among the third to eighth embodiments, a weight corresponding to each of the left and right wings (瞼 likelihood) may be determined as in any of the embodiments. For example, the heel likelihood calculation device of the present embodiment may be configured as described below. That is, the heel likelihood calculating device of the present embodiment includes the first detecting means, the second detecting means, the third detecting means, the fourth detecting means, the first calculating means, and The processing means including at least one means of the second calculation means includes the detected face orientation angle θ, the distance d _{right to the right} eye, the distance d _{left to the left} eye, and the distance to the right ear. d _{Right_2} and distance to the left ear _{d left_2,} the distance between the distance _{d Right_3} and left eye inner canthus and left of crow's feet and crow's feet right eye inner canthus and right _{d left_3,} contrast _{Iw. right} and left of the computed right eye area According to at least one of the contrast Iw _{left of} the eye region, the edge strength Ew _{right of the right} eye region, and the edge strength Ew _left of the _left eye region The size corresponding to the likelihood of the eyelid farther to the position where the image was imaged by the camera 12 out of the starboard and portside of the image taken by the camera 12 is the eye likelihood of the closer eyelid. Of the weights w _{right — 7} and w _{left — 7} corresponding to each of the left and right eyelids that are determined to be smaller than the corresponding size, the first ratio is equal to or less than the predetermined value γ and the second ratio is equal to or less than the predetermined value γ. In some cases, the weight w right — ₇ corresponding to the right eyelid is weighted to the right _eye likelihood λ _{r_eyelid_max} (w _right — ₇ · λ _{r_eyelid_max} ), and the weight w left — ₇ corresponding to the left eyelid is _set to the left _eye likelihood. along with selecting the value of the larger one of the weighting was value _{(w left_7} · _λ l_eyelid_max) to _λ l_eyelid_max, the first split When only one ratio of the second ratio and the second ratio is less than or equal to the predetermined value γ, the one for which the likelihood is calculated (the first ratio or the second ratio that is less than the predetermined value) ) Of the maximum _heel likelihood (right _heel likelihood λ _{r_eyelid_max} or left _heel likelihood λ _{l_eyelid_max} ) with a weight (weight w _{right — 7} or weight w _{left — 7} ) corresponding to this _heel ((w _right — ₇ · (λ _{r_eylid_max} ) or (w _left — ₇ · λ _{l_eyelid_max} )) may be selected.

  In addition, the high likelihood accuracy calculated in step 326 in the present embodiment can be used, for example, as the likelihood when estimating the shape of the eyelid. Further, for example, it is possible to detect the eyelid opening with high accuracy from the shape of the eyelid thus estimated.

  Further, the heel likelihood calculating apparatus according to the present embodiment estimates a heel model based on the calculated heel likelihood, and detects the heel opening degree of the heel represented by the heel model from the estimated heel model. It is good also as a structure provided with the eaves opening degree detection function. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

[Eleventh embodiment]
Next, an eleventh embodiment will be described. In addition, about the structure similar to each 1st-10th embodiment and the same process, the same code | symbol is attached | subjected and description is abbreviate | omitted. For example, as shown in FIG. 25, when a person 50 who is a subject to be photographed by the camera 12 wears spectacles 52, the heel likelihood calculating apparatus according to the present embodiment reflects light by the lens 54 of the spectacles 52. If the brightness of the image 58 in the eye region of the photographed person image 56 is increased, the eyelid likelihood cannot be calculated accurately when the eyelid likelihood cannot be accurately calculated. The left and right wrinkle likelihoods are calculated by setting the wrinkle likelihood of the wrinkles to a predetermined first value (for example, 0). For example, in the example of FIG. 25, light is reflected from the lens 54 on the left side of the glasses 52, and the luminance of the left eye region is high. In such a case, for example, the port heel likelihood is calculated by setting the port heel likelihood to a predetermined first value.

  The present embodiment is different from the tenth embodiment in that a program for executing the processing routine of the heel likelihood calculation process shown in FIG. 29 is stored in the HDD 14c of the heel likelihood calculation apparatus according to the present embodiment. Is stored, and the CPU 14d executes the heel likelihood calculation process shown in FIG. As shown in the figure, when a negative determination is made at step 330, step 348 is executed, and after step 348 and when an affirmative determination is made at step 114, step 332 is executed. When a negative determination is made at step 332, step 350 is executed, after step 350 and when an affirmative determination is made at step 120, step 352 is executed, and step 324 is executed next to step 352. Is done.

First, the process of step 348 will be described. In step 348, a predetermined first value (e.g. 0), by a right eyelid likelihood lambda _{R_eyelid,} calculates the right eyelid likelihood _λ r_eyelid. The predetermined first value is not limited to 0, but is preferably a value that makes it difficult to select in step 326.

Next, the process of step 350 will be described. In step 350, a predetermined first value (e.g. 0), by the left eyelid likelihood lambda _{L_eyelid,} calculates the left eyelid likelihood _λ l_eyelid. The predetermined first value is not limited to 0 as described above, but is preferably a value that is difficult to be selected in step 326.

Next, the process of step 352 will be described. In step 352, among the computed right eyelid likelihood lambda _{R_eyelid,} while selecting the maximum right eyelid likelihood lambda _{R_eyelid_max,} among eyelid likelihood lambda _{L_eyelid} of the computed left, up left Mabutayu Select the degree λ _{l_eyelid_max} .

As described above, the eyelid likelihood calculation apparatus according to the present embodiment uses the first ratio (the high luminance pixel area area ratio of the right eye region) R _right and the second ratio (the high luminance pixel area area ratio of the left eye region). ) calculates the _{R left,} when a first rate is greater than a predetermined value gamma, by the right eyelid likelihood lambda _{R_eyelid} a predetermined first value, while calculating the right eyelid likelihood lambda _{R_eyelid} second ratio of is larger than a predetermined value gamma, by a predetermined first value and the left eyelid likelihood lambda _{L_eyelid,} calculates the left eyelid likelihood _λ l_eyelid. Then, the heel likelihood calculation device according to the present embodiment is the farther to the position where the image is captured by the camera 12 among the starboard and port side of the image captured by the camera 12 according to the face orientation angle θ. Of the weights w _right and w _left corresponding to each of the left and right wings, the right corresponding to the heel likelihood of the heel is set to be smaller than the size corresponding to the heel likelihood of the near heel. A value _obtained by weighting the weight w _right corresponding to _瞼 of the _right weight (w _right · λ _{r_eyelid_max} ) to the _{right heel} likelihood λ _{r_eyelid_max,} and a value weighting the weight w _left corresponding to the _{left heel} to the left _heel likelihood λ _{l_eyelid_max} The larger value of (w _left · λ _{l_eylid_max} ) is selected. As a result, the heel likelihood of the moth that cannot accurately calculate the heel likelihood is set to the predetermined first value without being calculated by a complicated operation, so that the heel likelihood can be calculated more easily. be able to.

In this embodiment, an example has been described for determining the first weight _{w. Right} and weights _{w left} similarly to the embodiment, similarly to the second embodiment defines the weights _{w Right_2} and weights _{w Left_2} You may do it. Similarly, among the third to eighth embodiments, a weight corresponding to each of the left and right wings (瞼 likelihood) may be determined as in any of the embodiments. For example, the heel likelihood calculation device of the present embodiment may be configured as described below. That is, the heel likelihood calculating apparatus of the present embodiment includes the first detecting means, the second detecting means, the third detecting means, the fourth detecting means, the first calculating means, and The processing means including at least one means of the second calculation means includes a detected face orientation angle θ, a distance d _{right to the right} eye, a distance d _{left to the left} eye, and a distance to the right ear. d _{Right_2} and distance to the left ear _{d left_2,} the distance between the distance _{d Right_3} and left eye inner canthus and left of crow's feet and crow's feet right eye inner canthus and right _{d left_3,} contrast _{Iw. right} and left of the computed right eye area According to at least one of the contrast Iw _{left of} the eye region, the edge strength Ew _{right of the right} eye region, and the edge strength Ew _left of the _left eye region The size corresponding to the likelihood of the eyelid farther from the position where the image was taken by the camera 12 among the starboard and the left eyelid of the image taken by the camera 12 is the eye likelihood of the closer eyelid. Of weights w right — ₇ and w _{left — 7} corresponding to each of the left and right eyelids that are determined to be smaller than the corresponding size, a value _obtained by weighting the weight w right — ₇ corresponding to the right eyelid to the right eyelid likelihood λ _{r_eyelid_max} ( _{wright_7} · λ _{r_eyelid_max} ) and a value _obtained by weighting the weight w _{left_7} corresponding to the left eyelid to the left eyelid likelihood λ _{l_eyelid_max} (w _left — ₇ · λ _{l_eyelid_max} ) may be selected. .

  In addition, the high likelihood accuracy calculated in step 326 in the present embodiment can be used, for example, as the likelihood when estimating the shape of the eyelid. Further, for example, it is possible to detect the eyelid opening with high accuracy from the shape of the eyelid thus estimated.

  Further, the heel likelihood calculating apparatus according to the present embodiment estimates a heel model based on the calculated heel likelihood, and detects the heel opening degree of the heel represented by the heel model from the estimated heel model. It is good also as a structure provided with the eaves opening degree detection function. In this case, the heel likelihood calculation device of the present embodiment can be used as a heel opening degree detection device.

It is the schematic which shows the structure of the heel likelihood calculating apparatus which concerns on the 1st Embodiment of this invention. It is a figure for demonstrating the face area | region which concerns on 1st Embodiment. It is a figure for demonstrating the eyelid model which concerns on 1st Embodiment. It is a figure which shows the flowchart of the process routine of the heel likelihood calculation process which the heel likelihood calculation apparatus which concerns on 1st Embodiment performs. It is a figure which shows the flowchart of the process routine of the comparison model creation process which the hail likelihood calculating apparatus which concerns on 1st Embodiment performs. It is a figure for demonstrating the weight which concerns on 1st Embodiment. It is a figure for demonstrating the subject which this invention tends to solve. It is a figure which shows the flowchart of the processing routine of the wrinkle likelihood calculation process which the wrinkle likelihood calculation apparatus which concerns on 2nd Embodiment performs. It is a figure for demonstrating the weight which concerns on 2nd Embodiment. It is a figure which shows the flowchart of the processing routine of the wrinkle likelihood calculation process which the wrinkle likelihood calculation apparatus which concerns on 3rd Embodiment performs. It is a figure for demonstrating the weight which concerns on 3rd Embodiment. It is a figure which shows the flowchart of the process routine of the heel likelihood calculation process which the heel likelihood calculation apparatus which concerns on 4th Embodiment performs. It is a figure which shows the flowchart of the process routine of the eye area | region contrast calculation process which the eyelid likelihood calculating apparatus which concerns on 4th Embodiment performs. It is a figure for demonstrating eye area contrast calculation processing. It is a figure for demonstrating the weight which concerns on 4th Embodiment. It is a figure which shows the flowchart of the processing routine of the wrinkle likelihood calculation process which the wrinkle likelihood calculation apparatus which concerns on 5th Embodiment performs. It is a figure which shows the flowchart of the process routine of the eye area | region edge intensity | strength calculation process which the eyelid likelihood calculating apparatus which concerns on 5th Embodiment performs. It is a figure for demonstrating eye area | region edge intensity | strength calculation processing. It is a figure for demonstrating the weight which concerns on 5th Embodiment. It is a figure which shows the flowchart of the process routine of the heel likelihood calculation process which the heel likelihood calculating apparatus which concerns on 6th Embodiment performs. It is a figure for demonstrating the weight which concerns on 6th Embodiment. It is a figure which shows the relationship between the value of a weighting coefficient, and the eyelid opening degree error average. It is a figure which shows the flowchart of the processing routine of the wrinkle likelihood calculation process which the wrinkle likelihood calculation apparatus which concerns on 8th Embodiment and 9th Embodiment performs. It is a figure which shows the flowchart of the processing routine of the wrinkle opening degree determination process which the wrinkle likelihood calculating apparatus which concerns on 9th Embodiment performs. When a person to be photographed by the camera wears glasses, light is reflected by the lens of the glasses, and the brightness of the image of the eye area portion of the photographed person's image is increased, so It is a figure for demonstrating that the likelihood of 瞼 cannot be calculated. It is a figure which shows the flowchart of the processing routine of the wrinkle likelihood calculation process which the wrinkle likelihood calculation apparatus which concerns on 10th Embodiment performs. It is a figure which shows the flowchart of the process routine of the high-intensity pixel area area ratio calculation process which the haze likelihood calculating apparatus which concerns on 10th Embodiment performs. It is a figure for demonstrating a high-intensity pixel area area ratio calculation process. It is a figure which shows the flowchart of the process routine of the heel likelihood calculation process which the heel likelihood calculation apparatus which concerns on 11th Embodiment performs.

Explanation of symbols

10 瞼 Likelihood Calculation Device 12 Camera 14 Display Device 14 Computer 14c HDD
14d CPU

Claims (14)

Photographing means for photographing an image including a face image and outputting image data;
Detecting means for detecting a face orientation angle of the face based on the image data;
A starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left in which the shapes of the left upper eyelid and the left lower eyelid are defined by a plurality of second model parameters Storage means for storing the heel model;
Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison models,
The degree to which the shapes of the right upper eyelid and right lower eyelid represented by the right comparative eyelid model match the shapes of the right upper eyelid and right lower eyelid of the image photographed by the photographing means. The left heel likelihood and the left lower heel shape represented by the left comparative heel model and the left upper heel shape and the left lower heel shape of the image coincide with each other. First computing means for computing a left heel likelihood indicating a degree;
A starboard including a right upper eyelid and a right upper eyelid and a left upper eyelid and a left lower eyelid of the image photographed by the photographing means according to the face orientation angle detected by the detecting means. Left and right, the size corresponding to the heel likelihood of the heel farther to the position where the image was taken by the photographing means is smaller than the size corresponding to the heel likelihood of the closer heel The weight corresponding to the right eyelid among the weights corresponding to each of the eyelids of the right, the right comparison eyelid model in the combination of the right comparison eyelid model and the left comparison eyelid model having the same eyelid opening degree Of the weight corresponding to the left eyelid likelihood and the value corresponding to the left eyelid likelihood of the left comparative eyelid model among the weights corresponding to the left and right eyelids. For each combination as the left and right heel likelihood in the combination. A second computing means for computing,
Third computing means for computing the largest right and left eyelid likelihood as the overall eyelid likelihood among the left and right eyelid likelihoods for each combination calculated by the second computing means;
瞼 Likelihood calculation device. Photographing means for photographing an image including a face image and outputting image data;
Based on the image data, the distance to the right eye and the left eye of the face, the distance to the right ear of the face and the distance to the left ear, or the distance from the right eye to the right eye and the right eye Detection means for detecting a distance from the left eye to the left eye;
A starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left in which the shapes of the left upper eyelid and the left lower eyelid are defined by a plurality of second model parameters Storage means for storing the heel model;
Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison models,
The degree to which the shapes of the right upper eyelid and right lower eyelid represented by the right comparative eyelid model match the shapes of the right upper eyelid and right lower eyelid of the image photographed by the photographing means. The left heel likelihood and the left lower heel shape represented by the left comparative heel model and the left upper heel shape and the left lower heel shape of the image coincide with each other. First computing means for computing a left heel likelihood indicating a degree;
The distance to the right eye and the distance to the left eye detected by the detection means, the distance to the right ear and the distance to the left ear, or the distance from the right eye to the right eye and the left eye According to the distance from the top of the eye to the center of the left eye, the starboard including the right upper eyelid and the right lower eyelid and the left upper eyelid and the left lower eyelid of the image photographed by the photographing means Of the left and right sides, the size corresponding to the heel likelihood of the farther heel is smaller than the size corresponding to the heel likelihood of the closer heel Among the weights corresponding to each of the heels, the weight corresponding to the right heel is set to the right comparison heel model in the combination of the right comparison heel model and the left comparison heel model having the same heel opening.値 Likelihood weighted value and previous Of the weights corresponding to the left and right eyelids, the weight corresponding to the left eyelid is combined with the value obtained by weighting the left eyelid likelihood of the left comparison eyelid model as the left and right eyelid likelihood in the combination. A second computing means for performing a plurality of computations for each;
Third computing means for computing the largest right and left eyelid likelihood as the overall eyelid likelihood among the left and right eyelid likelihoods for each combination calculated by the second computing means;
瞼 Likelihood calculation device. Photographing means for photographing an image including a face image and outputting image data;
Based on the image data, the contrast of the right eye region of the facial image and the contrast of the left eye region of the facial image, or the edge strength of the right eye region of the facial image and the edge strength of the left eye region of the facial image. First computing means for computing;
A starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left in which the shapes of the left upper eyelid and the left lower eyelid are defined by a plurality of second model parameters Storage means for storing the heel model;
Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison models,
The degree to which the shapes of the right upper eyelid and right lower eyelid represented by the right comparative eyelid model match the shapes of the right upper eyelid and right lower eyelid of the image photographed by the photographing means. The left heel likelihood and the left lower heel shape represented by the left comparative heel model and the left upper heel shape and the left lower heel shape of the image coincide with each other. Second computing means for computing the left heel likelihood indicating the degree;
Depending on the contrast of the right eye region and the contrast of the left eye region calculated by the first calculating unit, or the edge strength of the right eye region and the edge strength of the left eye region, the right of the image photographed by the photographing unit The size corresponding to the likelihood of the heel with the smaller contrast or edge strength of the left heel and the left heel including the upper heel and the lower heel Or, the weight corresponding to the right eyelid among the weights corresponding to each of the left and right eyelids is set to be smaller than the size corresponding to the eyelid likelihood of the eyelid with the larger edge strength, and the eyelid opening degree is the same. A value weighted to the right heel likelihood of the right comparative heel model in a combination of the right comparative heel model and the left comparative heel model, and a weight corresponding to each of the left and right heel models The weights corresponding to the eyelid, and a third calculating means for a plurality calculating the sum of the value the was weighted to the left of the eyelid likelihood comparison eyelid model left for each combination as an eyelid likelihood of the right and left in the combination,
A fourth computing means for computing the largest left and right heel likelihood as the overall heel likelihood among the left and right heel likelihoods for each combination calculated by the third computing means;
瞼 Likelihood calculation device. Photographing means for photographing an image including a face image and outputting image data;
First detection means for detecting a face orientation angle of the face based on the image data; second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data; Third detection means for detecting the distance to the right ear and the left ear of the face based on the data; the distance from the right eye to the right eye based on the image data; and the left eye Fourth detection means for detecting a distance from the eye to the eye of the left eye, and first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data , And at least one means for calculating the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image based on the image data. And processing means,
A starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left in which the shapes of the left upper eyelid and the left lower eyelid are defined by a plurality of second model parameters Storage means for storing the heel model;
Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison models,
The degree to which the shapes of the right upper eyelid and right lower eyelid represented by the right comparative eyelid model match the shapes of the right upper eyelid and right lower eyelid of the image photographed by the photographing means. The left heel likelihood and the left lower heel shape represented by the left comparative heel model and the left upper heel shape and the left lower heel shape of the image coincide with each other. Third computing means for computing the left heel likelihood indicating the degree;
The detected face orientation angle, the distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, the distance from the right eye to the right eye, and the left eye Depending on at least one of the distance from the left eye to the center of the left eye, the calculated contrast of the right eye region and the contrast of the left eye region, and the edge strength of the right eye region and the edge strength of the left eye region. Of the captured image, the right farther including the right upper arm and the right lower arm, and the left upper arm and the left corridor including the left lower arm, the farther farther from the position where the image was captured by the imaging means. The weight corresponding to the right eyelid among the weights corresponding to the left and right eyelids is determined such that the size corresponding to the eyelid likelihood of the near eyelid is smaller than the magnitude corresponding to the eyelid likelihood of the near eyelid. , Cleavage A value weighted to the right heel likelihood of the right comparison heel model in the combination of the right comparison heel model and the left comparison heel model, and the weights corresponding to each of the left and right heels A fourth computing means for performing a plurality of computations for each combination as a right and left eyelid likelihood in the combination of a weight corresponding to the left eyelid and a value obtained by weighting the left eyelid likelihood of the left comparative eyelid model ,
A fifth computing means for computing the largest left and right eyelid likelihood as the overall eyelid likelihood among the left and right eyelid likelihoods for each combination calculated by the fourth computing means;
瞼 Likelihood calculation device. Photographing means for photographing an image including a face image and outputting image data;
First detection means for detecting a face orientation angle of the face based on the image data; second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data; Third detection means for detecting the distance to the right ear and the left ear of the face based on the data; the distance from the right eye to the right eye based on the image data; and the left eye Fourth detection means for detecting a distance from the eye to the eye of the left eye, and first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data , And at least one means for calculating the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image based on the image data. And processing means,
A starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left in which the shapes of the left upper eyelid and the left lower eyelid are defined by a plurality of second model parameters Storage means for storing the heel model;
Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison models,
The degree to which the shapes of the right upper eyelid and right lower eyelid represented by the right comparative eyelid model match the shapes of the right upper eyelid and right lower eyelid of the image photographed by the photographing means. The left heel likelihood and the left lower heel shape represented by the left comparative heel model and the left upper heel shape and the left lower heel shape of the image coincide with each other. Third computing means for computing the left heel likelihood indicating the degree;
The detected face orientation angle, the distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, the distance from the right eye to the right eye, and the left eye Depending on at least one of the distance from the left eye to the center of the left eye, the calculated contrast of the right eye region and the contrast of the left eye region, and the edge strength of the right eye region and the edge strength of the left eye region. Of the captured image, the right farther including the right upper arm and the right lower arm, and the left upper arm and the left corridor including the left lower arm, the farther farther from the position where the image was captured by the imaging means. The weight corresponding to the right eyelid among the weights corresponding to the left and right eyelids is determined such that the size corresponding to the eyelid likelihood of the near eyelid is smaller than the magnitude corresponding to the eyelid likelihood of the near eyelid. Said The third computing means computes a value weighted to the maximum right likelihood of the right computed by the computing means and a weight corresponding to the left heel among the weights corresponding to each of the left and right heels. Selecting means for selecting the larger value among the values weighted to the maximum likelihood of the left heel,
Fourth calculating means for calculating the value selected by the selecting means as the likelihood of the selected eyelid;
瞼 Likelihood calculation device. A value obtained by weighting the weight corresponding to the right heel to the maximum right heel likelihood and a value weighting the weight corresponding to the left heel to the maximum left heel likelihood are both predetermined. If it is greater than or equal to a reference value, the right eyelid opening degree represented by the right comparison eyelid model of the maximum right eyelid likelihood and the left eyelid likelihood model represented by the left eyelid likelihood model represented by the maximum left eyelid likelihood A fifth computing means for detecting a saddle opening and calculating a ratio between the detected starboard saddle opening and the left saddle saddle opening;
The heel opening degree of the heel represented by the comparative heel model of the heel likelihood calculated by the fourth computing means is detected, and the heel opening degree is detected based on the detected heel opening degree and the ratio. A fifth detecting means for detecting the eyelid opening degree of the eyelid having the lower eyelid likelihood by calculating the eyelid opening degree of the eyelid not having one;
The heel likelihood calculating apparatus according to claim 5, further comprising: Photographing means for photographing an image including a face image and outputting image data;
First detection means for detecting a face orientation angle of the face based on the image data; second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data; Third detection means for detecting the distance to the right ear and the left ear of the face based on the data; the distance from the right eye to the right eye based on the image data; and the left eye Fourth detection means for detecting a distance from the eye to the eye of the left eye, and first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data , And at least one means for calculating the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image based on the image data. And processing means,
A starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left in which the shapes of the left upper eyelid and the left lower eyelid are defined by a plurality of second model parameters Storage means for storing the heel model;
Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison models,
Based on the image data, a first ratio that is a ratio of the number of pixels equal to or higher than a predetermined luminance to the number of pixels of a right-eye area image that is an image of a right-eye area included in the face image, and the face image Third calculating means for calculating a second ratio that is a ratio of the number of pixels equal to or higher than a predetermined luminance to the number of pixels of the left eye area image that is an image of the left eye eye area included in
Only when the first ratio calculated by the third calculating means is less than or equal to a predetermined value, the shapes of the right upper eyelid and the right lower eyelid represented by the right comparative eyelid model and the photographing means The right eyelid likelihood indicating the degree to which the shapes of the right upper eyelid and the right lower eyelid of the captured image coincide with each other is calculated, and the second ratio calculated by the third calculating means is The degree to which the shapes of the left upper eyelid and left lower eyelid represented by the left comparative eyelid model only match the shapes of the left upper eyelid and left lower eyelid of the image only when they are equal to or less than a predetermined value A fourth computing means for computing the left heel likelihood indicating
The detected face orientation angle, the distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, the distance from the right eye to the right eye, and the left eye Depending on at least one of the distance from the left eye to the center of the left eye, the calculated contrast of the right eye region and the contrast of the left eye region, and the edge strength of the right eye region and the edge strength of the left eye region. Of the captured image, the right farther including the right upper arm and the right lower arm, and the left upper arm and the left corridor including the left lower arm, the farther farther from the position where the image was captured by the imaging means. The weight corresponding to each of the left and right wings is determined so that the size corresponding to the heel likelihood of the near heel is smaller than the size corresponding to the heel likelihood of the near heel, and the first ratio is predetermined. Below value And if the second ratio is less than or equal to a predetermined value, the weight corresponding to the right heel is weighted to the maximum right heel likelihood computed by the fourth computing means, and Of the weights corresponding to the left and right eyelids, the larger one of the values obtained by weighting the weight corresponding to the left eyelid to the maximum left eyelid likelihood calculated by the fourth arithmetic means is selected. In addition, when only one of the first ratio and the second ratio is less than or equal to a predetermined value, the first ratio or the second ratio calculated by the fourth calculation means Selection means for selecting a value obtained by weighting the maximum heel likelihood of the cocoon having a value equal to or less than a predetermined value with a weight corresponding to the cocoon;
Fifth calculating means for calculating the value selected by the selecting means as the likelihood of the selected eyelid;
瞼 Likelihood calculation device. Photographing means for photographing an image including a face image and outputting image data;
First detection means for detecting a face orientation angle of the face based on the image data; second detection means for detecting a distance to the right eye and a distance to the left eye of the face based on the image data; Third detection means for detecting the distance to the right ear and the left ear of the face based on the data; the distance from the right eye to the right eye based on the image data; and the left eye Fourth detection means for detecting a distance from the eye to the eye of the left eye, and first calculation means for calculating the contrast of the right eye region of the face image and the contrast of the left eye region of the face image based on the image data , And at least one means for calculating the edge strength of the right eye region of the face image and the edge strength of the left eye region of the face image based on the image data. And processing means,
A starboard model in which the shapes of the right upper eyelid and the right lower eyelid are defined by a plurality of first model parameters, and a left in which the shapes of the left upper eyelid and the left lower eyelid are defined by a plurality of second model parameters Storage means for storing the heel model;
Creating a plurality of right comparison models by changing at least one model parameter of the first model parameter a plurality of times, and changing at least one model parameter of the second model parameter a plurality of times Creating means for creating a plurality of left comparison models,
Based on the image data, a first ratio that is a ratio of the number of pixels equal to or higher than a predetermined luminance to the number of pixels of a right-eye area image that is an image of a right-eye area included in the face image, and the face image Third calculating means for calculating a second ratio that is a ratio of the number of pixels equal to or higher than a predetermined luminance to the number of pixels of the left eye area image that is an image of the left eye eye area included in
When the first ratio calculated by the third calculating means is less than or equal to a predetermined value, the shape of the right upper eyelid and the right lower eyelet represented by the right comparative eyelid model and the image taking means A right heel likelihood indicating a degree of matching between the shape of the right upper eyelid and the right lower eyelid of the obtained image, and when the first ratio is greater than a predetermined value, a predetermined first When the right heel likelihood is calculated by setting the value as the right heel likelihood and the second ratio calculated by the third calculation means is equal to or less than a predetermined value, the left comparative heel model The left upper eyelid and left lower eyelid shapes represented by the left upper eyelid and left lower eyelid shapes represented by When the ratio of 2 is greater than a predetermined value, the left first A fourth calculating means for calculating a Mabutayudo,
The detected face orientation angle, the distance to the right eye and the distance to the left eye, the distance to the right ear and the distance to the left ear, the distance from the right eye to the right eye, and the left eye Depending on at least one of the distance from the left eye to the center of the left eye, the calculated contrast of the right eye region and the contrast of the left eye region, and the edge strength of the right eye region and the edge strength of the left eye region. Of the captured image, the right farther including the right upper arm and the right lower arm, and the left upper arm and the left corridor including the left lower arm, the farther farther from the position where the image was captured by the imaging means. The weight corresponding to the right eyelid among the weights corresponding to the left and right eyelids, so that the size corresponding to the eyelid likelihood of the near eyelid is smaller than the size corresponding to the eyelid likelihood of the near eyelid The above Among the weights corresponding to each of the left and right eyelids, the weight corresponding to the left eyelid among the values weighted to the maximum right eyelid likelihood computed by the operation means 4 and the fourth computing means. A selection means for selecting a larger value among the weighted values of the calculated maximum left likelihood of the left,
Fifth calculating means for calculating the value selected by the selecting means as the likelihood of the selected eyelid;
瞼 Likelihood calculation device.   A computer is caused to function as the detection means, the creation means, the first calculation means, the second calculation means, and the third calculation means in the heel likelihood calculation device according to claim 1 or 2. Program for.   A computer is caused to function as the first calculation means, the creation means, the second calculation means, the third calculation means, and the fourth calculation means in the heel likelihood calculation device according to claim 3. Program.   A program for causing a computer to function as the processing means, the creation means, the third calculation means, the fourth calculation means, and the fifth calculation means in the heel likelihood calculation device according to claim 4.   A program for causing a computer to function as the processing means, the creation means, the third calculation means, the selection means, and the fourth calculation means in the heel likelihood calculation device according to claim 5.   The program according to claim 11 for causing a computer to further function as the fifth calculation means and the fifth detection means in the heel likelihood calculation device according to claim 6.   9. The computer according to claim 7 or claim 8, wherein said processing means, said creating means, said third computing means, said fourth computing means, said selecting means, and said fifth computation. Program to function as a means.