JP2016152029A - Face authentication device, image processing device, and living body determination device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a face authentication device high in living body determination accuracy and strong against surreptitious photographing, and provide an image processing device and a living body determination device applicable to the face authentication device.SOLUTION: A face authentication device 20 is provided with: an exposed mucosa detection part 24 that detects exposed mucosae, which are parts where mucosae of a face are intentionally exposed by a person; a living body determination unit 25 that detects presence/absence of a change, in an image of each exposed mucosa detected by the exposed mucosa detection unit 24, caused due to biological activity of the person so as to determine whether a person to be authenticated is a living body; a database 26 that stores registered person data indicating features of faces of registered persons; a person-in-question determination unit 27 that determines whether a face of the person to be authenticated corresponds to any of the faces of the registered persons; and an authentication determination unit 28 that authenticates the person to be authenticated when the living body determination unit 25 determines that the person to be authenticated is a living body and the person-in-question determination unit 27 determines that the person to be authenticated corresponds to any of the registered persons.SELECTED DRAWING: Figure 6

Description

  The present invention relates to a face authentication device that performs authentication on a human face detected from an image, and an image processing device and a biological determination device that can be applied to the face authentication device.

  In recent years, various authentication systems have been used as security measures for preventing unauthorized use of information devices and intrusion of suspicious persons. In particular, a face authentication device that uses a human face as an authentication key does not cause problems such as theft, loss or forgetting of the authentication key because the authentication key is a part of the human body, or a person who wants to receive authentication Since it has various advantages such as that it is possible to authenticate in a non-contact and simple way that the face (hereinafter referred to as "authenticated person") faces the camera for authentication, it is attracting attention Yes.

  However, in the face authentication apparatus, the face of a person whose face is registered in the face authentication apparatus as a person to be authenticated (hereinafter referred to as “registrant”) is voyeurized, and a photograph of the face of the registrant created thereby is taken. There is a problem that authentication succeeds by an unauthorized method of pointing to the authentication camera.

  In view of this, Japanese Patent Laid-Open No. 2004-228620 proposes a face authentication apparatus that prevents the authentication from succeeding by an unauthorized method as described above by performing face authentication after confirming that the person to be authenticated is a living body. Specifically, in this face authentication device, whether or not a change in blood flowing through capillaries in a specific area of the face is detected as an image change indicated by a series of image data obtained by capturing a moving image of the person to be authenticated. Whether or not the person to be authenticated is a living body is determined.

JP 2014-184002 A

  However, in the face authentication device proposed in Patent Document 1, a surface layer (typically the skin epidermis that is strong and thick to be exposed to the outside world. Note that the lips are the outer part exposed to the outside world. The skin has a stratum corneum like the skin, and is a kind of skin.) It must detect a very fine change in appearance, that is, a change in blood flow through the capillary below. Therefore, although the face authentication device proposed in Patent Document 1 can in principle determine whether or not it is a living body, it is practically difficult to put it to practical use because the determination accuracy of the living body is low. , It becomes a problem.

  Furthermore, in the face authentication device proposed in Patent Document 1, even if it can be determined whether or not it is a living body, it is not a living body when a still image such as a photograph is directed to the authentication camera. Although it can be determined, if the display device that reproduced the moving image is directed to the authentication camera, it is erroneously determined to be a living body. In other words, in the face authentication device proposed in Patent Document 1, the registrant's face is voyeurized as a moving image, and an illegal method of directing the display device that reproduces the moving image obtained thereby to the authentication camera, Authentication succeeds. Therefore, the face authentication device proposed in Patent Document 1 is problematic because it is vulnerable to voyeurism.

  SUMMARY OF THE INVENTION An object of the present invention is to provide a face authentication device that has a high accuracy of biometric determination and is robust against voyeurism. It is another object of the present invention to provide an image processing apparatus and a biometric determination apparatus that can be applied to the face authentication apparatus.

  In order to achieve the above object, the present invention provides an exposed mucous membrane, which is a portion where a human has intentionally exposed the mucous membrane of a face from among images indicated by image data generated by imaging the face of the person to be authenticated. About each exposed mucous membrane detected by the exposed mucous membrane detection unit from among each of the plurality of images indicated by the plurality of image data generated by temporally continuous imaging to detect the exposed mucosa, By detecting the presence or absence of an image change caused by a person's life activity, a biometric determination unit for determining whether or not the person to be authenticated is a living body, and registrant data indicating the characteristics of the registrant's face By comparing the database to be recorded with the facial features of the person to be authenticated in the image indicated by the image data, and the facial features of the registrant indicated by the registrant data recorded by the database. , The person to be authenticated An identity determination unit that determines whether or not the registered person corresponds, and the biometric determination unit determines that the person to be authenticated is a living body, and the person determination that the to-be-authenticated person corresponds to the registrant. A face authentication apparatus comprising: an authentication determination unit configured to authenticate the person to be authenticated when the unit determines.

  According to this face recognition device, the surface layer is thinner than the part that is constantly exposed to the outside, such as the skin, and the capillaries and the blood flowing through them are exposed to the exposed mucous membrane clearly visible from the outside. Based on this, it can be determined whether the person to be authenticated is a living body. In addition, it is determined whether or not the person to be authenticated is a registered person based on a facial expression intentionally performed by a person and a special expression rarely seen in the public, that is, an exposed mucosal expression. Can do.

  The face authentication device having the above characteristics further includes a face part detection unit that detects a face part, which is a predetermined part existing in a human face, from the image indicated by the image data, and the database records the data. The registrant data to indicate the face part of the registrant, and the identity determination unit indicates the face part of the person to be authenticated detected by the face part detection unit by the registrant data. It is preferable to determine whether or not the person to be authenticated corresponds to the registrant based on whether or not it corresponds to the face part of the registrant.

  According to this face authentication device, it is possible to accurately determine whether or not the person to be authenticated corresponds to the registrant by comparing different characteristics of each person such as a face part.

  Furthermore, in the face authentication device having the above characteristics, the exposed mucous membrane detection unit includes an appearance prediction region that is a region where the appearance of the exposed mucous membrane is predicted based on the position of the face part detected by the face part detection unit. It is preferable to set and detect the exposed mucous membrane from the set appearance prediction region.

  According to this face authentication device, an exposed mucous membrane that is difficult to exhaustively detect because it is an unspecified area is accurately detected while suppressing erroneous detection from a non-face area such as a background. It becomes possible. Furthermore, according to the face authentication device, the exposed mucous membrane is detected from a limited area called the appearance prediction area, so that the calculation amount necessary for detecting the exposed mucosa can be reduced and the detection can be quickly performed. become.

  Furthermore, in the face authentication device having the above characteristics, it is preferable that the exposed mucous membrane detection unit detects the exposed mucous membrane based on a red component of the image.

  According to this face authentication device, it is possible to detect an exposed mucous membrane that is conspicuously red in the face while preventing erroneous detection of a red portion of a non-face area such as a background as an exposed mucosa. . Therefore, it is possible to detect the exposed mucous membrane with high accuracy.

  In the face authentication device having the above characteristics, it is preferable that the face part detection unit detects the face part based on a luminance component of the image.

  According to this face authentication apparatus, it is possible to detect a face part with high accuracy by utilizing the existence of a light and dark pattern peculiar to a human face.

  In the face authentication device having the above characteristics, the registrant data recorded in the database indicates the exposed mucous membrane of the registrant, and the identity determination unit is configured to detect the subject detected by the exposed mucous membrane detection unit. Preferably, it is determined whether or not the person to be authenticated corresponds to the registrant based on whether or not the exposed mucous membrane of the certifier corresponds to the exposed mucous membrane of the registrant indicated by the registrant data. .

  According to this face authentication apparatus, it is possible to accurately determine whether or not the person to be authenticated corresponds to the registrant by comparing different characteristics of the exposed mucous membrane for each person. Further, by comparing the facial region in addition to the exposed mucous membrane, it becomes possible to determine with high accuracy whether or not the person to be authenticated corresponds to the registrant.

  Moreover, in the face authentication device having the above characteristics, the registrant data recorded in the database indicates an identification code for identifying an individual set for each registrant, and the identity determination unit includes: The identification code input by the person to be authenticated identifies the registrant that matches the identification code indicated by the registrant data, and the facial features of the registrant who have been identified as the facial features of the person to be authenticated. It is preferable to compare the characteristics.

  According to this face authentication apparatus, by specifying in advance a registrant to be compared based on an identification code, an authenticated person who originally does not have a corresponding registrant accidentally falls into one of the registrants. It is possible to prevent the determination. Therefore, it can be accurately determined whether or not the person to be authenticated corresponds to the registrant.

  In addition, the image processing apparatus of the present invention detects an exposed mucous membrane, which is a portion where a human intentionally exposes the mucous membrane of a face, from an image indicated by image data generated by imaging a subject's face. An exposed mucous membrane detection unit; and a facial part detection unit that detects a facial part, which is a predetermined part existing on a human face, from the image, and the exposed mucous membrane detection part includes: An appearance prediction area that is an area where the appearance of the exposed mucous membrane is predicted is set based on the detected position of the face part, and the exposed mucosa is detected from the set appearance prediction area. .

  According to this image processing apparatus, an exposed mucous membrane that is difficult to exhaustively detect because it is an unspecified area is detected with high accuracy while suppressing erroneous detection from a non-face area such as a background. It becomes possible. Furthermore, according to this image processing apparatus, since the exposed mucous membrane is detected from a limited area called the appearance prediction area, it is possible to reduce the amount of calculation necessary for detecting the exposed mucous membrane and perform detection quickly. become.

  In addition, the living body determination device of the present invention detects an exposed mucous membrane, which is a portion where a human has intentionally exposed the mucous membrane of the face, from an image indicated by image data generated by imaging the subject's face. For each of the exposed mucous membranes detected by the exposed mucosa detecting unit from among each of the plurality of images indicated by the exposed mucous membrane detecting unit and the plurality of image data generated by temporally continuous imaging, And a living body determination unit that determines the state of the subject's life activity by detecting an image change caused by the life activity.

  According to this living body judgment device, the surface layer is thinner than the part that is constantly exposed to the outside such as the skin, and the capillary blood vessel and the blood flowing through it are exposed to the exposed mucous membrane clearly visible from the outside. Based on this, it becomes possible to accurately determine the state of the life activity of the subject.

  In the face authentication apparatus having the above characteristics, it is possible to accurately determine whether the person to be authenticated is a living body because the person to be authenticated is determined based on the exposed mucous membrane. Furthermore, a face recognition device that is robust against voyeurism is realized in order to determine whether or not the person to be authenticated corresponds to a registrant based on a special expression that rarely appears in the public, such as a facial expression with exposed mucous membranes. It becomes possible.

The block diagram shown about an example of a structure of a registrant data production | generation apparatus. The flowchart which shows an example of operation | movement of the registrant data generation apparatus of FIG. The figure shown about an example of operation | movement of a face part detection part and an exposed mucous membrane detection part. The table | surface which shows an example of a structure of registrant data. The figure shown about another example of operation | movement of a face part detection part and an exposed mucous membrane detection part. The block diagram shown about an example of the structure of the face authentication apparatus which concerns on embodiment of this invention. The flowchart which shows an example of operation | movement of the face authentication apparatus of FIG. The figure which shows another example of operation | movement of an exposed mucous membrane detection part.

<Registrant data generation device>
Before describing a face authentication apparatus according to an embodiment of the present invention, a registrant data generation apparatus for generating registrant data used in the face authentication apparatus will be described with reference to the drawings. First, the configuration of the registrant data generation device will be described with reference to the drawings. FIG. 1 is a block diagram illustrating an example of a configuration of a registrant data generation apparatus.

  As shown in FIG. 1, the registrant data generation device 10 includes an identification code acquisition unit 11, an imaging unit 12, a face part detection unit 13, an exposed mucosa detection unit 14, a registrant data generation unit 15, and a database. 16.

  The identification code acquisition unit 11 acquires an identification code input by the registrant, and provides the registrant data generation unit 15 with identification code data indicating the identification code. For example, the identification code acquisition unit 11 includes an input device such as a keypad or a keyboard, or a reading device such as a contact type or non-contact type card reader. The identification code is a code for identifying an individual set for each registrant, such as an employee ID or the name of a registrant.

  The imaging unit 12 captures the registrant's face and generates image data. For example, the imaging unit 12 is configured by an imaging device including an imaging element such as a complementary metal oxide semiconductor (CMOS) image sensor or a charge coupled device (CCD) image sensor. The image indicated by the image data generated by the imaging unit 12 is a color image including color components such as RGB (red, green, and blue).

  The face part detection unit 13 detects a face part, which is a predetermined part existing in a human face, from the image indicated by the image data generated by the imaging unit 12, and generates face part data indicating the face part. To do.

  A face part is each part which comprises a face which can be distinguished from other parts by shapes, colors, etc., such as eyes, mouth, nose, ears, and eyebrows. For example, the face part data is data representing the coordinates of one or a plurality of feature points (for example, the center position, the left and right end positions, the upper and lower end positions, etc.) in each face part in the image.

  The exposed mucous membrane detection unit 14 detects an exposed mucous membrane, which is a portion where a human has intentionally exposed the mucous membrane from the image indicated by the image data generated by the imaging unit 12, and exposes the mucous membrane data indicating the exposed mucous membrane. Is generated.

  The exposed mucous membrane is the outside world in a normal state (a state of expressionlessness with weakened muscles of the face and no intentional deformation of the facial expression from the outside, such as pulling the skin with a hand). It is not exposed to water, but it is a mucous membrane that is exposed to the outside world by actions such as moving facial muscles or pulling skin with hands. For example, the exposed mucous membrane includes the conjunctiva on the lower side of the eye (jaw side), the conjunctiva on the upper side of the eye (the parietal side), the conjunctiva on the back side of the eyelid, the tongue, the inside of the lips (oral side), the oral mucosa, etc. . Further, for example, the exposed mucosa data is data representing the coordinates of each vertex of the polygon when the outline of the exposed mucosa region in the image is approximated by a polygon.

  The registrant data generation unit 15 associates the identification code data given from the identification code acquisition unit 11, the facial part data generated by the facial part detection unit 13, and the exposed mucous membrane data generated by the exposed mucous membrane detection unit 14 ( Registrant data is generated.

  The face part detection unit 13, the exposed mucous membrane detection unit 14, and the registrant data generation unit 15 are units for processing various data, and can be realized by one or a plurality of arithmetic devices and storage devices as a whole. For example, a CPU (Central Processing Unit) or a GPU (Graphic Processing Unit) may be used as the arithmetic device. Further, for example, a RAM (Random Access Memory) may be used as the storage device.

  The database 16 records registrant data generated by the registrant data. For example, the database 16 is configured by a non-volatile recording device such as a hard disk or a flash memory.

  Next, the operation of the registrant data generation device 10 will be described with reference to the drawings. FIG. 2 is a flowchart showing an example of the operation of the registrant data generation apparatus of FIG. FIG. 3 is a diagram illustrating specific operations of the face part detection unit and the exposed mucous membrane detection unit. FIG. 3A is a diagram schematically illustrating a specific example of a face included in an image indicated by image data generated by the imaging unit. FIG. 3B is a diagram schematically showing a specific example of the operation of the face part detection unit in the case of detecting a face part from the image including the face shown in FIG. 3 (c) and 3 (d) schematically show specific examples of the operation of the exposed mucosa detection unit when the exposed mucosa is detected from the image including the face shown in FIG. 3 (a). It is a figure.

  As shown in FIG. 2, first, the identification code acquisition unit 11 acquires the registrant's identification code, and provides the registrant data generation unit 15 with identification code data indicating the identification code (step # 10). At this time, the registrant performs a predetermined operation on the identification code acquisition unit 11 (for example, hitting the keypad, holding the employee card over the card reader, etc.), thereby causing the identification code acquisition unit 11 to Enter the identification code.

  Next, the imaging unit 12 captures the registrant's face and generates image data (step # 11). At this time, the registrant creates an expression in which the exposed mucous membrane intentionally appears when the image is captured by the imaging unit 12.

  For example, as shown in FIG. 3 (a), the registrant exposes the lower conjunctiva of the right eye by pulling the lower skin of the right eye with the finger of the right hand, and the muscles and tongue around the mouth. Expose the upper surface of the tongue using muscles (so-called “glance”).

  Next, the face part detection unit 13 detects a face part from the image indicated by the image data generated by the imaging unit 12 (step # 12). Furthermore, the face part detection unit 13 generates face part data indicating the detected face part.

  The human face has lightness and darkness due to the unevenness of the facial part (the part that is convex from the periphery of the nose becomes brighter and the part that is concave from the periphery of the eye socket becomes darker) and the color of the facial part There are distinctive light and dark patterns, such as the resulting light and dark (white areas are brighter than the surroundings such as white eyes, and black parts are darker than the surroundings such as the iris and pupil and eyebrows). Therefore, the face part detection unit 13 acquires the luminance component of each pixel constituting the image (for example, converts the color image data generated by the imaging unit 12 into a grayscale image in which each pixel is represented by the luminance component). In addition, by applying the above-described unique light / dark pattern to the distribution of the luminance component, the face area is detected from the image indicated by the image data in distinction from the background area. To detect the face part.

  For example, as shown in FIG. 3B, the face part detection unit 13 selects the left and right eyes (pupil or center position) EL and ER and the left and right eyes from the image including the face shown in FIG. Each of mouth angles ML and MR is detected. In FIGS. 3B and 3C, for convenience of illustration, the left and right mouth corners ML and MR are represented by black rectangles, and the left and right eyes EL and ER are represented by white rectangles. Represents. Further, here, in order to simplify the description, a total of four feature points, that is, one feature point (pupil or center position) in each of the left and right eyes EL and ER and two feature points (left and right end positions) in the mouth. Although only feature points are illustrated, many feature points of 5 or more can be detected in actual processing. In particular, in order to perform face authentication with high accuracy, it is preferable to detect a large number of feature points for a large number of face parts.

  Further, the face part data generated by the face part detection unit 13 at this time is data indicating the coordinates of the respective feature points. For example, the face part data may be data in which the coordinates of a specific position such as the lower left corner of the image are set as the reference coordinates (0, 0) and the positions of the respective feature points are represented by the coordinates (x, y). . Further, for example, the facial part data is data in which the coordinates of specific feature points such as the center position of the left eye are set as reference coordinates (0, 0), and the positions of the respective feature points are represented by coordinates (x, y). Also good.

  By the way, as in the example shown in FIG. 3A, when the skin under the right eye is pulled downward with the right hand, the right hand overlaps the face. However, this right hand only overlaps the cheek with few features, and does not overlap with a region rich in features such as eyes, nose and mouth. Therefore, as shown in FIG. 3B, the face part detection unit 13 can detect the face and the face part by using parts such as eyes, nose and mouth.

  Next, the exposed mucosa detection unit 14 detects the exposed mucous membrane from the image indicated by the image data generated by the imaging unit 12 (step # 13). Further, the exposed mucosa detection unit 14 generates exposed mucosa data indicating the detected exposed mucosa.

  The exposed mucous membrane has a thin surface layer as compared with a part that is constantly exposed to the outside, such as the skin, and the capillaries and the blood flowing therethrough are clearly visible from the outside. In other words, it can be said that the exposed mucous membrane is a portion of the face that is markedly red. Therefore, the exposed mucous membrane detection unit 14 detects, from the image indicated by the image data generated by the imaging unit 12, a region where the red component of the image is large (for example, a region where the red component is greater than or equal to a predetermined threshold). Detecting exposed mucous membranes.

  Further, due to the structure of the human face, the exposed mucous membrane appears at a position adjacent to a specific face part such as the eyes and mouth. Therefore, the exposed mucous membrane detection unit 14 detects the exposed mucous membrane based on the position of the facial part indicated by the facial part data generated by the facial part detection unit 13. Specifically, the exposed mucous membrane detection unit 14 sets an appearance prediction area that is an area where the appearance of the exposed mucosa is predicted based on the position of the facial part, and detects the exposed mucosa from the appearance prediction area.

  Specifically, for example, as shown in FIG. 3C, the exposed mucosa detection unit 14 is perpendicular to a straight line connecting the left and right eyes EL and ER, and the left and right eyes ER Line segments SL and SR extending to a straight line MLR connecting the corner angles ML and MR are defined. Here, the line segment SL is a line segment extending from the left eye EL to the straight line MLR, and the line segment SR is a line segment extending from the right eye ER to the straight line MLR.

  Next, the exposed mucous membrane detection unit 14 detects a portion where the red component of the image is equal to or greater than a threshold value for the line segments SL and SR. In the case of the example shown in FIG. 3C, it is assumed that the portion of the line segment SR close to the right eye ER (the portion where the line segment SR overlaps the conjunctiva CR below the right eye ER) has a red component equal to or greater than the threshold value. It is detected by the exposed mucosa detection unit 14.

  Next, the exposed mucous membrane detection unit 14 sets the appearance prediction region PR around the portion where the red component of the image in the line segment SR is equal to or greater than the threshold value. At this time, the size of the appearance prediction region PR set by the exposed mucous membrane detection unit 14 may be a fixed size determined in advance, or the position and size of the eyes, the size of the face, and the line segment. The size may be variable according to the length of the portion where the red component of the image in SR is equal to or greater than the threshold.

  Then, the exposed mucous membrane detection unit 14 detects, as the exposed mucous membrane, a region in which the red component of the image is equal to or greater than the threshold in the set appearance prediction region PR. In the example shown in FIG. 3C, the conjunctival CR below the right eye ER is detected as an exposed mucosa. When the registrant exposes the lower conjunctiva CL of the left eye EL instead of (or in addition to) the lower conjunctiva CR of the right eye ER, the lower side of the right eye ER shown in FIG. Similar to the case where the conjunctival CR is exposed, an appearance prediction region is set around the left eye EL, and the entire conjunctiva below the left eye EL is detected as an exposed mucosa.

  Further, for example, as shown in FIG. 3D, the exposed mucosa detection unit 14 sets the start points of the line segments SL and SR shown in FIG. 3C from the left and right eyes EL and ER to the left and right mouth corners ML and MR. A rectangular region surrounded by the line segments TL and TR obtained by translation, the line segments connecting the left and right mouth angles ML and MR, and the line segments connecting the end points of the line segments TL and TR is defined as an appearance prediction region Q. Set as. Then, the exposed mucous membrane detection unit 14 detects, as the exposed mucous membrane, a region in the set appearance prediction region Q where the red component of the image is equal to or greater than a threshold value. In the example shown in FIG. 3D, the tongue U is detected as an exposed mucosa.

  In addition, the exposed mucosa data generated by the exposed mucosa detection unit 14 at this time is data indicating the coordinates of the respective exposed mucosa regions. For example, the exposed mucous membrane data includes a plurality of contour lines of the exposed mucosa area detected from the appearance prediction area with the coordinates of a specific feature point in the face part where the appearance prediction area is set as reference coordinates (0, 0). Data that represents the position of each vertex of the polygon in coordinates (x, y) when approximated by a square may be used. Specifically, for example, the conjunctival CR below the right eye ER shown in FIG. 3C is an area of the exposed mucosa data with the coordinates of the right eye (pupil or center position) ER as reference coordinates (0, 0). Defined as position coordinates. Further, for example, the region of the tongue U shown in FIG. 3D is the position of the region in the exposed mucous membrane data where the coordinates of the mouth corner center MC, which is the center of the left and right mouth angles ML, MR, are used as the reference coordinates (0, 0). Is defined as the coordinates of

  By the way, the exposed mucous membrane data can also define the coordinates of a specific position such as the lower left corner of the image as the reference coordinates (0, 0), similarly to the facial part data. However, with regard to the exposed mucosa data, if the coordinates of a specific feature point in the face part are defined as the reference coordinates (0, 0), the position of the registrant's exposed mucous membrane can be quickly and accurately specified in the processing in the face authentication device described later. This is preferable.

  Next, the registrant data generation unit 15 includes the identification code data generated by the identification code acquisition unit 11, the facial part data generated by the face part detection unit 13, the exposed mucous membrane data generated by the exposed mucous membrane detection part 14, Are associated (linked) to generate registrant data (step # 14).

  A specific example of the registrant data generated by the registrant data generation unit 15 will be described with reference to the drawings. FIG. 4 is a table showing an example of the configuration of registrant data. Note that the registrant data shown in FIG. 4 is a configuration example of the registrant data for one registrant.

  As shown in FIG. 4, the registrant data indicates identification code data indicating the registrant's identification code, face part data indicating the position of the registrant's face part, and the position of the exposed mucous membrane of the registrant. And exposed mucosa data. As described above, for example, the facial part data is data representing the position (absolute position) of the feature point of the facial part in the image. Further, for example, the exposed mucous membrane data is data representing the position (relative position) of the exposed mucous membrane region based on the coordinates of a specific feature point of the facial part.

  Then, the registrant data generated by the registrant data generation unit 15 is recorded in the database 16 (step # 15), and the operation of the registrant data generation apparatus 10 ends.

  In the operation example shown in FIG. 2, acquisition of the identification code (step # 10) is performed first, but it may be performed at any timing as long as it is before step # 14.

  FIG. 2 only illustrates the basic operation of the registrant data generation apparatus 10. Therefore, various modifications may be made from the viewpoint of obtaining registrant data suitable for face authentication.

  For example, the face part detection unit 13 detects the exposed mucous membrane in the exposed mucosa detection unit 14 or registers in the registrant data generation unit 15 until a sufficiently large face facing the front is continuously detected for a predetermined time or more. You may wait for generation of person data. Whether or not the registrant is facing the front can be determined based on the score at the time of detection of the face and the facial part (a value indicating the degree to which the contrast pattern for the front face matches). Further, generation of registrant data in the registrant data generation unit 15 may be waited until the exposed mucous membrane detection unit 14 continuously detects a sufficiently large exposed mucous membrane for a predetermined time or more.

  In this case, the imaging unit 12 continuously generates a plurality of image data. Further, the face part detection unit 13 and the exposed mucous membrane detection part 14 detect the face part and the exposed mucous membrane for each of the plurality of images indicated by the plurality of image data generated by the imaging unit 12.

  In addition, FIG. 3 illustrates a case where the front is faced and the center line of the face (a straight line passing through the middle point of the left and right eyes and the center of the mouth corner) and the vertical axis of the image (vertical direction in the figure) are parallel. However, as shown in FIG. 5, the face center line and the vertical axis of the image are not parallel (that is, the face does not rotate in the yaw direction and pitch direction but rotates in the roll direction). However, it is possible to generate registrant data as in FIG. Here, FIG. 5 is a diagram showing another example of the operation of the face part detection unit and the exposed mucosa detection unit. When the face in FIG. 3 is tilted (specifically, the left eye EL and the right eye ER are in the horizontal direction). When the neck is tilted so that the left eye EL is lowered and the right eye ER is raised from the state of being lined up along.

  As described with reference to FIG. 3, the face part detection unit 13 detects a face part by applying a specific light / dark pattern in a human face to the luminance component distribution of the image. Therefore, the face of the registrant can be detected from the image regardless of the direction of the vertical axis or horizontal axis of the image. Similarly, the exposed mucous membrane detection unit 14 sets an appearance prediction region based on the positional relationship between the facial parts (in the above example, the positional relationship between the eyes and the mouth), and detects the exposed mucous membrane from the appearance prediction region. Therefore, the exposed mucous membrane can be detected regardless of the direction of the vertical axis or horizontal axis of the image. Therefore, as shown in FIG. 5, even if the registrant's face is tilted in the image, registrant data can be generated.

<Face recognition device>
Next, a face authentication apparatus according to an embodiment of the present invention will be described with reference to the drawings. First, the configuration of the face authentication device will be described with reference to the drawings. FIG. 6 is a block diagram illustrating an example of the configuration of the face authentication apparatus.

  As shown in FIG. 6, the face authentication device 20 includes an identification code acquisition unit 21, an imaging unit 22, a face part detection unit 23, an exposed mucosa detection unit 24, a living body determination unit 25, a database 26, and the person himself / herself. The determination part 27 and the authentication determination part 28 are provided.

  Similar to the identification code acquisition unit 11 in the registrant data generation device 10, the identification code acquisition unit 21 is configured by an input device such as a keypad or a keyboard, or a reading device such as a contact type or non-contact type card reader. The However, the identification code acquisition unit 21 is operated by the person to be authenticated, acquires the identification code of the person to be authenticated, and gives identification code data indicating the identification code of the person to be authenticated to the person determination unit 27.

  Similar to the imaging unit 22 in the registrant data generation device 10, the imaging unit 22 is configured by an imaging device including an imaging element such as a CMOS image sensor or a CCD image sensor, and is a color image including color components such as RGB. Is generated. However, the imaging unit 22 performs temporally continuous imaging (moving image imaging) on the person to be authenticated.

  Similar to the face part detection unit 23 in the registrant data generation device 10, the face part detection unit 23 detects a face part from the image indicated by the image data generated by the imaging unit 22, and indicates the face part. Generate site data. However, the face part detection unit 23 detects a face part for each of a plurality of images indicated by a plurality of image data generated by the imaging unit 22 by temporally continuous imaging.

  Similar to the exposed mucosa detection unit 24 in the registrant data generation apparatus 10, the exposed mucosa detection unit 24 detects the exposed mucous membrane from the image indicated by the image data generated by the imaging unit 22, and exposes the exposed mucous membrane. Generate mucosal data. However, the exposed mucosa detection unit 24 detects the exposed mucous membrane for each of a plurality of images indicated by a plurality of image data generated by the imaging unit 22 by temporally continuous imaging. The exposed mucosa data generated by the exposed mucosa detection unit 24 is data that defines the coordinates of the position of the exposed mucosa area (absolute position in the image) using the same reference coordinates (0, 0) as the face part data. It is.

  The biometric determination unit 25 detects the exposure detected by the exposed mucosa detection unit 24 from each of a plurality of image data generated by the imaging unit 22 by temporally continuous imaging and a plurality of images indicated by the plurality of image data. Based on the exposed mucosa data indicating the position of the mucosal region, an area representing the exposed mucosa in each of the plurality of images is specified. Furthermore, the biometric determination unit 25 determines whether or not the person to be authenticated is a living body by detecting the presence or absence of an image change caused by a human life activity for the area representing the specified exposed mucous membrane, Biological determination result data indicating the determination result is generated.

  As described above, the exposed mucous membrane is a portion that is conspicuously red in the face because the capillaries and the blood flowing therethrough are clearly visible from the outside. Therefore, if the person to be authenticated is a living body, the pulsation corresponding to the heartbeat of the blood flowing in the capillary blood vessel appears as a change in the red component of the image. Therefore, if the pulsation of the exposed mucous membrane is detected from the image, the living body determination unit 25 determines that the person to be authenticated is a living body. Specifically, for example, the living body determination unit 25 has a human average heart rate of about 60 to 90 times / minute (in terms of conversion, about 1 Hz to 1.5 Hz or less), and at least one exposed mucous membrane. If the red component changes, it is determined that the person to be authenticated is a living body.

  In this case, the pulsation is about once per second. Therefore, from the viewpoint of accurately determining whether or not the person to be authenticated is a living body, it is preferable that the living body determination unit 25 is configured to acquire image data continuously captured for several seconds or more. Specifically, for example, when the frame rate of the imaging unit 22 is 30 fps, the biometric determination unit 25 is preferably configured to acquire image data of about 300 frames (imaging time of about 10 seconds).

  Similar to the database 16 in the registrant data generation device 10, the database 26 is configured by a non-volatile recording device such as a hard disk or a flash memory, and records registrant data generated by the registrant data generation device 10. Note that the database 26 included in the face authentication device 20 and the database 16 included in the registrant data generation device 10 may be configured by the same recording device.

  The identity determination unit 27 first identifies a registrant corresponding to the authenticated person based on the identification code of the authenticated person indicated by the identification code data given from the identification code acquisition unit 21, and the registrant of the registrant. Data is read from the database 26. Specifically, the identity determination unit 27 includes the registrant associated with the identification code data indicating the identification code of the registrant that matches the identification code of the person to be authenticated among the registrant data recorded in the database 26. Data (especially face part data and exposed mucosa data) is selectively read out.

  Next, the person determination unit 27 determines that the face part of the person to be authenticated indicated by the face part data generated by the face part detection unit 23 is the face of the registrant indicated by the face part data included in the registrant data read from the database 26. It is judged whether it corresponds to a region.

  For example, at this time, the person determination unit 27 determines the coordinate mutual positional relationship (the mutual positional relationship between the coordinates of each feature point) of the feature points of the face portion of the person to be authenticated indicated by the face portion data generated by the face portion detection unit 23. The coordinate mutual positional relationship of the feature points of the registrant's face part indicated by the face part data included in the registrant data read from the database 26 is compared, and if it matches a predetermined degree or more, the face of the person to be authenticated It is determined that the part corresponds to the registrant's face part.

  Further, at this time, in order to eliminate the deviation in the roll direction of the face of the registrant and the person to be authenticated in the image, the identity determination unit 27 applies to the coordinate mutual positional relationship between the feature points of the face parts of the person to be authenticated and the person to be registered. You may compare both, accept | permitting rotation. In addition, the identity determination unit 27 detects a difference in the size of the face of the registrant and the person to be authenticated in the image (the difference caused by the difference between the distance of the registrant to the imaging unit 12 and the distance of the person to be authenticated from the imaging unit 22). ) May be compared with each other after allowing similar deformation of the coordinate positions of the feature points of the face portions of the person to be authenticated and the registrant. Further, in order for the identity determination unit 27 to eliminate the deviation in the yaw direction and the pitch direction of the face of the registrant and the person to be authenticated in the image, the coordinate mutual positional relationship of the feature points of the face parts of the person to be authenticated and the registrant The two may be compared after allowing deformation in the left-right direction and the up-down direction.

  Further, the identity determination unit 27 determines whether the exposed mucosa of the person to be authenticated indicated by the exposed mucosa data generated by the exposed mucosa detection unit 24 is the exposed mucous membrane of the registrant indicated by the exposed mucosa data included in the registrant data read from the database 26. It is determined whether it corresponds to.

  For example, at this time, the identity determination unit 27 uses the coordinates of the feature point of the face part of the person to be authenticated indicated by the face part data generated by the face part detection unit 23 and the coordinates of the specific feature point of the face part as reference coordinates (0 , 0) based on the coordinates of the exposed mucous membrane area of the registrant indicated by the exposed mucosa data included in the registrant data defined as the relative position coordinates, The coordinates of the position (absolute position in the image) of the region of the exposed mucous membrane of the registrant as the reference coordinates (0, 0) are calculated. The identity determination unit 27 then authenticates the registrant's exposed mucous membrane area position (absolute position in the image) calculated as described above and the exposed mucosa data generated by the exposed mucosa detection unit 24. The coordinates of the position of the person's exposed mucosa (absolute position in the image) are compared, and if they match more than a predetermined degree, it is determined that the person's exposed mucous membrane corresponds to the person's exposed mucous membrane.

  Further, at this time, as in the case where the identity determination unit 27 compares the coordinate mutual positional relationship between the feature points of the face portions of the person to be authenticated and the registrant, the rotation and similarity of the exposed mucous membrane of the person to be authenticated and the registrant. A comparison between the two may be performed after allowing deformation, lateral deformation, and vertical deformation.

  For example, when the person determination unit 27 determines that the face portion of the person to be authenticated corresponds to the face portion of the registrant and determines that the exposed mucous membrane of the person to be authenticated corresponds to the exposed mucous membrane of the registrant. It is determined that the certifier corresponds to the registrant, and personal identification result data indicating the determination result is generated.

  The authentication determination unit 28 determines whether or not authentication is possible for the person to be authenticated based on the person determination result data and the biometric determination result data, and generates authentication result data indicating the determination result. Specifically, the authentication determination unit 28 indicates a determination result that the person to be authenticated corresponds to the registrant, and the biometric determination result data indicates a determination result that the person to be authenticated is a living body. If so, authenticate the person to be authenticated.

  The face part detection unit 23, the exposed mucous membrane detection unit 24, the living body determination unit 25, the person determination unit 27, and the authentication determination unit 28 are units that process various data. Therefore, it can be realized by one or a plurality of arithmetic devices and storage devices as a whole. For example, a CPU or a GPU may be used as the arithmetic device. Further, for example, a RAM may be used as the storage device.

  Next, the operation of the face authentication device 20 will be described with reference to the drawings. FIG. 7 is a flowchart showing an example of the operation of the face authentication apparatus of FIG.

  As shown in FIG. 7, first, the identification code acquisition unit 21 acquires the identification code of the person to be authenticated, and gives identification code data indicating the identification code to the identity determination unit 27 (step # 20).

  Next, the identity determination unit 27 identifies the identification code of the person to be authenticated indicated by the identification code data generated by the identification code acquisition unit 21, and identifies the registrant indicated by the identification code data of the registrant data recorded in the database 26. The code is collated (step # 21).

  At this time, if there is a registrant with an identification code that matches the identification code of the person to be authenticated, the identity determination unit 27 identifies the registrant as a registrant corresponding to the person to be authenticated. On the other hand, if there is no registrant whose identification code matches the identification code of the person to be authenticated, the person determination unit 27 determines that the registrant corresponding to the person to be authenticated cannot be identified.

  When the person determination unit 27 determines that the registrant corresponding to the person to be authenticated cannot be identified (step # 22, NO), the person determination result indicating the determination result that the person to be authenticated does not correspond to the registrant Generate data. Then, the authentication determination unit 28 that has acquired the personal identification result data generates authentication result data indicating that the person to be authenticated is not authenticated (authentication failure) (step # 23), and the operation of the face authentication device 20 is completed. To do.

  On the other hand, when the identity determination unit 27 identifies a registrant corresponding to the person to be authenticated (step # 23, YES), the registrant data of the identified registrant (particularly facial part data and exposed mucous membrane data) is stored in the database 26. Are selectively read out (step # 24).

  Next, the imaging unit 22 captures the face of the person to be authenticated and generates image data (step # 25). At this time, the to-be-authenticated person makes the same facial expression as the registrant's facial expression when imaged by the imaging unit 12 of the registrant data generation device 10 when being imaged by the imaging unit 22. Specifically, the person to be authenticated creates a facial expression in which the exposed mucosa appears intentionally (see, for example, FIG. 3A). Further, the imaging unit 22 performs temporally continuous imaging (for example, moving image imaging for 10 seconds or more) for the person to be authenticated.

  Next, similarly to the face part detection unit 13 of the registrant data generation apparatus 10, the face part detection unit 23 detects a face part from the image indicated by the image data generated by the imaging unit 22 (step # 26). ).

  At this time, the face part detection unit 23 detects each face part from a plurality of images indicated by a plurality of image data generated by temporally continuous imaging. Further, the face part detection unit 23 generates face part data indicating the detected face part, similarly to the face part detection unit 13 of the registrant data generation apparatus 10.

  Next, the person determination unit 27 performs the face part of the person to be authenticated indicated by the face part data generated by the face part detection unit 23 and the face of the registrant indicated by the face part data included in the registrant data read from the database 26. The site is compared (step # 27).

  As described above, the identity determination unit 27 is included in the coordinate mutual positional relationship between the feature points of the face parts of the person to be authenticated indicated by the face part data generated by the face part detection unit 23 and the registrant data read from the database 26. Compared with the coordinate mutual positional relationship of the feature points of the registrant's face part indicated by the face part data to be identified, if it matches a predetermined degree or more, it is determined that the face part of the person to be authenticated corresponds to the registrant's face part To do.

  Here, the identity determination unit 27 may compare the face parts of the plurality of authentication targets detected by the face part detection unit 23 from each of the plurality of images, or from any one of the images. A comparison may be made with respect to the face part of one person to be authenticated detected by the face part detection unit 23. Further, when the person determination unit 27 compares the face parts of the plurality of authentication targets detected by the face part detection unit 23 from among each of the plurality of images, for example, the characteristics of the face parts of the plurality of authentication subjects The final comparison result may be calculated by averaging the comparison results of the coordinate mutual positional relationship of the points and the coordinate mutual positional relationship of the feature points of the registrant's face part, or the face of the person to be authenticated The comparison result may be calculated by averaging the coordinate mutual positional relationship of the feature point of the part and comparing it with the coordinate mutual positional relation of the feature point of the registrant's face. Of the comparison results between the coordinate mutual positional relationship of the feature points of the region and the coordinate mutual positional relationship of the feature points of the registrant's facial region, the most similar may be calculated as the final comparison result.

  When the person determination unit 27 determines that the face part of the person to be authenticated does not correspond to the face part of the registrant (step # 28, NO), the person determination that indicates the determination result that the person to be authenticated does not correspond to the registrant Generate result data. Then, the authentication determination unit 28 that has acquired the personal identification result data generates authentication result data indicating that the person to be authenticated is not authenticated (authentication failure) (step # 23), and the operation of the face authentication device 20 is completed. To do.

  On the other hand, when the person determination unit 27 determines that the face part of the person to be authenticated corresponds to the face part of the registrant (step # 28, YES), the exposed mucous membrane detection part 24 uses the exposed mucous membrane of the registrant data generation apparatus 10. Similar to the detection unit 14, the exposed mucous membrane is detected from the image indicated by the image data generated by the imaging unit 22 (step # 29).

  At this time, the exposed mucous membrane detection unit 24 detects each exposed mucous membrane from among a plurality of images indicated by a plurality of image data generated by temporally continuous imaging. Further, the exposed mucosa detection unit 24 indicates the coordinates of the position of the exposed mucosa region (absolute position in the image) using the same reference coordinates (0, 0) as the face part data generated by the face part detection unit 23. Generate exposed mucosa data.

  Next, the person determination unit 27 exposes the exposed mucous membrane of the person to be authenticated indicated by the exposed mucosa data generated by the exposed mucosa detecting unit 24 and the exposure of the registrant indicated by the exposed mucous membrane data included in the registrant data read from the database 26. The mucous membrane is compared (step # 30).

  As described above, the person determination unit 27 uses the coordinates of the feature point of the face part of the person to be authenticated indicated by the face part data generated by the face part detection unit 23 and the coordinates of the specific feature point of the face part as reference coordinates ( The specific feature point of the person to be authenticated based on the coordinates of the exposed mucous membrane area of the registrant indicated by the exposed mucous membrane data included in the registrant data defined as the relative position coordinates of 0,0) The coordinates of the position (absolute position in the image) of the exposed mucous membrane area of the registrant are calculated with reference coordinates (0, 0). The identity determination unit 27 then authenticates the registrant's exposed mucous membrane area position (absolute position in the image) calculated as described above and the exposed mucosa data generated by the exposed mucosa detection unit 24. The coordinates of the position of the person's exposed mucosa (absolute position in the image) are compared, and if they match more than a predetermined degree, it is determined that the person's exposed mucous membrane corresponds to the person's exposed mucous membrane.

  Here, the identity determination unit 27 may compare the exposed mucous membranes of the plurality of authentication targets detected by the exposed mucosa detection unit 24 from among each of the plurality of images, or from any one of the images. A comparison may be made with respect to the exposed mucous membrane of one person to be authenticated detected by the exposed mucosa detecting unit 24. Further, when the identity determination unit 27 compares the exposed mucous membranes of the plurality of authentication subjects detected by the exposed mucosa detection unit 24 from among each of the plurality of images, for example, the positions of the exposed mucous membranes of the plurality of authentication subjects. The final comparison result may be calculated by averaging the respective comparison results of the coordinates of the registrant and the position of the exposed mucous membrane of the registrant, or the coordinates of the position of the exposed mucous membrane of the authentication subject may be averaged. Then, the final comparison result may be calculated by comparing with the coordinates of the position of the exposed mucous membrane of the registrant. The most similar result of the respective comparisons with the coordinates may be calculated as the final comparison result.

  When the person determination unit 27 determines that the exposed mucous membrane of the person to be authenticated does not correspond to the exposed mucous membrane of the registrant (step # 31, NO), the person determination indicating the determination result that the person to be authenticated does not correspond to the registrant Generate result data. Then, the authentication determination unit 28 that has acquired the personal identification result data generates authentication result data indicating that the person to be authenticated is not authenticated (authentication failure) (step # 23), and the operation of the face authentication device 20 is completed. To do.

  On the other hand, when the person determination unit 27 determines that the exposed mucous membrane of the person to be authenticated corresponds to the exposed mucous membrane of the registrant (step # 31, YES), the person indicating the determination result that the person to be authenticated corresponds to the registrant. Judgment result data is generated.

  Next, the exposed mucous membrane detection unit 24 selects each of the plurality of image data generated by the imaging unit 22 through temporal imaging and the plurality of images indicated by the plurality of image data. Based on the exposed mucous membrane data indicating the position of the detected exposed mucosa region, the region representing the exposed mucosa in each of the plurality of images is specified. Furthermore, the biometric determination unit 25 determines whether or not the person to be authenticated is a living body by detecting the presence or absence of an image change caused by a human life activity for the area representing the specified exposed mucous membrane, Biological determination result data indicating the determination result is generated (step # 32).

  At this time, the living body determination unit 25 determines whether or not the image change detected from the region representing the exposed mucous membrane is due to a human life activity. For example, the living body determination unit 25 determines that the person to be authenticated is a living body if the variation of the red component in the region representing the exposed mucous membrane is 60 times / minute or more and 90 times / minute or less (1 Hz or more and 1.5 Hz or less). If this condition is not satisfied, the person to be authenticated is determined not to be a living body.

  When the biometric determination unit 25 determines that the person to be authenticated is not a living body (step # 33, NO), biometric determination result data indicating a determination result that the person to be authenticated is not a living body is generated. The authentication determination unit 28 that has acquired the biometric determination result data generates authentication result data indicating that the person to be authenticated is not authenticated (authentication failure) (step # 23), and the operation of the face authentication device 20 is completed. To do.

  On the other hand, when the biometric determination unit 25 determines that the person to be authenticated is a living body (step # 33, YES), biometric determination result data indicating a determination result that the person to be authenticated is a living body is generated. Then, the authentication determination unit 28 that has acquired the person determination result data indicating the determination result that the above-mentioned person to be authenticated corresponds to the registrant and the biometric determination result data authenticates the person to be authenticated (authentication). Authentication result data indicating (success) is generated (step # 34), and the operation of the face authentication apparatus 20 is completed.

  As described above, in the face authentication device 20 according to the embodiment of the present invention, the surface layer is thinner than the part that is constantly exposed to the outside, such as the skin, and the capillaries and the blood flowing therein are external. It is possible to determine whether or not the person to be authenticated is a living body based on the exposed mucous membrane that is clearly visually recognized. Therefore, it can be accurately determined whether or not the person to be authenticated is a living body.

  Furthermore, in the face authentication apparatus 20 according to the embodiment of the present invention, the person to be authenticated is based on a facial expression intentionally performed by a person and a special expression rarely seen in public, that is, an exposed mucosal expression. It can be determined whether or not the user is a registered person. Therefore, it is possible to realize the face authentication device 20 that is robust against voyeurism.

  FIG. 7 merely illustrates the basic operation of the face authentication device 20. Therefore, various modifications may be made from the viewpoint of performing an appropriate operation according to the required security level.

  For example, it is not necessary to specify the registrant based on the identification code (steps # 20 to # 24). In this case, it is determined whether or not the face part of the person to be authenticated corresponds to the face part of the registrant (steps # 27 and # 28), and whether or not the exposed mucous membrane of the person to be authenticated corresponds to the exposed mucous membrane of the registrant. When making such determination (steps # 30 and # 31), it is only necessary to make an exhaustive determination for all registrants.

  However, in this method, there is a possibility that the person determination unit 27 erroneously determines that the person to be authenticated who originally does not have the corresponding registrant falls under any registrant. Therefore, when the required security level is high and it is necessary to accurately determine whether the person to be authenticated corresponds to the registrant, the registrant is identified based on the identification code (steps # 20 to # 24). ) Is preferable.

  In addition, it is determined whether or not the face part of the person to be authenticated corresponds to the face part of the registrant (steps # 27 and # 28), and whether or not the exposed mucous membrane of the person to be authenticated corresponds to the exposed mucous membrane of the registrant. (Steps # 30 and # 31) may not be performed.

  However, if the required security level is high and it is necessary to accurately determine whether or not the person to be authenticated falls under the category of registrant, the person to be authenticated will be asked to registrant for both the facial area and the exposed mucous membrane. It is preferable to determine whether this is the case.

  When determining whether or not the exposed mucous membrane of the person to be authenticated corresponds to the exposed mucous membrane of the registrant (steps # 30 and # 31), the exposed mucosa existing on the left and right of the human face such as the conjunctiva. In addition, the left and right sides of a person's face may be ignored. Specifically, for example, when the person determination unit 27 determines that the person to be authenticated does not correspond to the registrant, the person-to-be-authenticated person and the registrant's exposed mucous membrane are reversed left and right, and then compared again. May be.

  In this case, for example, the person to be authenticated has exposed the lower conjunctiva of the left eye, but the exposed mucous membrane data included in the registrant data of the registrant who is the same person as the person to be authenticated is the lower side of the right eye of the registrant. Even if the person's conjunctiva is shown, the person determination unit 27 can determine that the person to be authenticated corresponds to the registrant.

  However, if the required security level is high and the selection of the exposed mucosa itself (that is, which mucosa the registrant has exposed to create registrant data) is also used as the authentication key, such processing is performed. Preferably not.

  Further, in the determination of whether or not the exposed mucous membrane of the person to be authenticated corresponds to the exposed mucous membrane of the registrant (steps # 30 and # 31), the exposed mucous membrane of the authenticated person and the exposed mucous membrane of the registrant do not substantially match The user may be notified that the mucous membrane to be exposed is not correct, or may be instructed to the user to be exposed.

  Specifically, for example, the person to be authenticated has exposed the lower conjunctiva of the left eye, but the exposed mucous membrane data included in the registrant data of the registrant who is the same person as the person to be authenticated is the lower side of the right eye of the registrant. If it indicates a conjunctiva of the subject, a warning such as "Exposed mucosa is different" or "Expose the conjunctiva under the right eye" is notified to the subject by voice or image display. Also good.

  However, when the required security level is high and the selection of the exposed mucosa is also used as an authentication key, it is preferable not to perform such processing.

<Deformation, etc.>
[1] In FIGS. 3 and 5, the conjunctiva and tongue below the eyes have been described as specific examples of the exposed mucosa, but other mucosa can also be detected as the exposed mucosa. Here, a case where a mucous membrane different from those shown in FIGS. 3 and 5 is detected as an exposed mucosa will be described with reference to the drawings. FIG. 8 is a diagram illustrating another example of the operation of the exposed mucosa detection unit.

  As shown in FIG. 8A, when the exposed mucosa detection units 14 and 24 detect the conjunctiva on the upper side of the eye, the line segments SL and SR and the start points (left eye EL and right eye ER) in FIG. The appearance prediction region PR1 may be set based on the line segments SL1 and SR1 that are common but extend in the opposite direction. In the example shown in FIG. 8A, the appearance prediction region PR1 is set above the right eye ER, and the conjunctiva CR1 above the right eye ER is detected as an exposed mucosa.

  Further, as shown in FIG. 8B, when the exposed mucosa detection units 14 and 24 detect the entire mucous membrane in the oral cavity (the mucous membrane exposed only with the mouth open), The appearance prediction region Q2 may be set based on the line segments TL2 and TR2 in which the centers of the line segments TL and TR are aligned with the left and right mouth angles ML and MR. In the case of the example shown in FIG. 8B, the appearance prediction region Q2 is set inside and outside the opened mouth, and the entire mucous membrane U2 in the oral cavity is detected as the exposed mucosa.

  Further, when the registrant and the person to be authenticated do not limit which mucous membrane should be exposed, such as when the selection of the exposed mucosa itself is also used as an authentication key, the exposed mucosa detection units 14 and 24 detect all the exposed mucosa. It is preferable to set the line segment and the appearance prediction area so that For example, the exposed mucous membrane detection units 14 and 24 may set the appearance prediction regions based on the line segments SL and SR in FIG. 3C and the line segments SL1 and SR1 in FIG. Further, the exposed mucous membrane detection units 14 and 24 may set the appearance prediction regions based on the line segments TL and TR in FIG. 3D and the line segments TL2 and TR2 in FIG.

[2] The detection method of the facial part by the facial part detection units 13 and 23 and the detection method of the exposed mucous membrane by the exposed mucous membrane detection units 14 and 24 shown in FIGS. 3 and 5 are merely examples, and the face is detected by another detection method. You may detect a site | part and an exposed mucous membrane.

  For example, when the exposed mucosa detection units 14 and 24 detect the exposed mucous membrane, it is not necessary to set an appearance prediction region based on the position of the face part. In this case, the exposed mucosa detection units 14 and 24 comprehensively detect the exposed mucosa from the entire image.

  However, since the exposed mucous membrane is an unspecified area, comprehensive detection is difficult. In particular, if an exposed mucosa is detected from the entire image based on the red component of the image, there is a possibility that a red portion of a non-face area such as a background is erroneously detected as an exposed mucosa.

  Therefore, as illustrated in FIGS. 3 and 5, when the exposed mucosa detection units 14 and 24 set the appearance prediction region based on the position of the face part and detect the exposed mucosa, the exposed mucosa is detected with high accuracy. Is preferable. Furthermore, in this case, the exposed mucous membrane detection units 14 and 24 detect the exposed mucous membrane from a limited region called the appearance prediction region, thereby reducing the amount of calculation necessary for detecting the exposed mucous membrane and quickly detecting it. Since it can be performed, it is preferable.

[3] In the above-described embodiment, the registrant data generation device 10 and the face authentication device 20 are illustrated in FIGS. 1 and 6 as if they are separate devices. Common device ”). For example, in this case, when the common device operates in the “registration mode”, the operation of each block of the registrant data generation device 10 shown in FIG. 1 is realized, and when the common device operates in the “authentication mode”, The operation of each block of the face authentication device 20 shown in FIG.

  For example, in this common device, the identification code acquisition units 11 and 21 can be configured by the same input device or reading device. Further, for example, the imaging units 12 and 22 can be configured by the same imaging device. Further, for example, the databases 16 and 26 can be configured by the same recording device. Further, for example, the face part detection units 13 and 23, the exposed mucosa detection units 14 and 24, the registrant data generation unit 15, the living body determination unit 25, and the person determination unit 27 can be configured by the same arithmetic device and storage device.

[4] The exposed mucous membrane detection unit 24 and the biometric determination unit 25 of the face authentication device 20 can be applied not only to the face authentication device 20 but also to other devices that determine the state of life activity of the subject. For example, in a living body determination apparatus for medical use that measures the pulse of a target person in an image or determines the presence or absence of an abnormality in the measured pulse, the exposed mucosa detection unit 24 and the living body determination unit 25 are used. Also good.

  As described above, the exposed mucous membrane has a thin surface layer as compared with a part that is constantly exposed to the outside, such as skin, and the capillaries and the blood flowing therethrough are clearly visible from the outside. Therefore, in the living body determination apparatus as described above, when the state of the life activity is determined by paying attention to the exposed mucous membrane, the state of the life activity of the subject can be accurately determined.

[5] The face part detection unit 13 and the exposed mucous membrane detection part 14 of the registrant data generation device 10 and the face part detection unit 23 and the exposed mucous membrane detection part 24 of the face authentication device 20 are limited to devices for performing face authentication. It can also be applied to other image processing apparatuses that detect exposed mucous membranes. For example, in an image processing apparatus for medical use that detects mucous membrane portions such as conjunctiva and tongue from an image, the facial part detection units 13 and 23 and the exposed mucosal detection units 14 and 24 may be used.

  As described above, the exposed mucous membrane is an area having an unspecified extent, so that comprehensive detection is difficult, and there is a concern that it may be erroneously detected from a non-face area such as a background. In addition, when detecting the exposed mucous membrane from the entire image, the amount of calculation necessary for detecting the exposed mucosa becomes enormous, making it difficult to quickly detect it. Therefore, in the image processing apparatus as described above, it is possible to accurately detect the exposed mucous membrane by setting the appearance prediction region based on the position of the face part and detecting the exposed mucosa from the appearance prediction region. In addition, the amount of calculation necessary for detecting the exposed mucous membrane can be reduced and detection can be performed quickly.

  INDUSTRIAL APPLICABILITY The present invention can be used for a face authentication device that performs authentication on a human face detected from an image. Further, it can be used for an image processing apparatus and a biometric determination apparatus applicable to the face authentication apparatus.

DESCRIPTION OF SYMBOLS 10: Registrant data generation apparatus 11: Identification code acquisition part 12: Imaging part 13: Face part detection part 14: Exposed mucosa detection part 15: Registrant data generation part 16: Database 20: Face authentication apparatus 21: Identification code acquisition part 22: Imaging unit 23: Face part detection unit 24: Exposed mucosa detection unit 25: Biometric determination unit 26: Database 27: Identity determination unit 28: Authentication determination unit EL, ER: Eye ML, MR: Mouth corner MC: Mouth corner center MLR: Straight line connecting mouth corners SL, SR, SL1, SR1: line segment TL, TR, TL2, TR2: line segment PR, PR1: appearance prediction region (conjunctiva)
Q, Q2: Appearance prediction area (tongue, oral cavity)
CR, CR1: Conjunctiva U: Tongue U2: Whole mucous membrane in the oral cavity

Claims (9)

An exposed mucous membrane detecting unit that detects an exposed mucous membrane, which is a portion where a human has intentionally exposed the mucous membrane of the face, from an image indicated by image data generated by imaging the face of the person to be authenticated;
Of each of the plurality of images indicated by the plurality of image data generated by temporally continuous imaging, each of the exposed mucosa detected by the exposed mucosa detection unit is an image resulting from human life activity. A living body determination unit that determines whether the person to be authenticated is a living body by detecting the presence or absence of a general change;
A database that records registrant data indicating the registrant's facial features;
By comparing the facial features of the person to be authenticated in the image indicated by the image data with the facial features of the registrant indicated by the registrant data recorded in the database, A person determination unit that determines whether or not the user corresponds to the registrant;
An authentication determination unit that authenticates the person to be authenticated when the living body determination unit determines that the person to be authenticated is a living body and the person determination unit determines that the person to be authenticated corresponds to the registrant. When,
A face authentication apparatus comprising: A face part detection unit for detecting a face part, which is a predetermined part existing on a human face, from the image indicated by the image data;
The registrant data recorded in the database indicates the facial part of the registrant,
The identity determination unit, based on whether the face part of the person to be authenticated detected by the face part detection unit corresponds to the face part of the registrant indicated by the registrant data. 2. The face authentication apparatus according to claim 1, wherein it is determined whether or not a person corresponds to the registrant.   The exposed mucous membrane detection unit sets an appearance prediction region, which is a region where the appearance of the exposed mucous membrane is predicted, based on the position of the facial part detected by the face part detection unit, and The face authentication apparatus according to claim 2, wherein the exposed mucous membrane is detected from the inside.   The face authentication apparatus according to claim 3, wherein the exposed mucous membrane detection unit detects the exposed mucous membrane based on a red component of the image.   The face authentication apparatus according to claim 2, wherein the face part detection unit detects the face part based on a luminance component of the image. The registrant data recorded by the database indicates the exposed mucous membrane of the registrant,
The identity determination unit, based on whether or not the exposed mucous membrane of the person to be authenticated detected by the exposed mucous membrane detection unit corresponds to the exposed mucous membrane of the registrant indicated by the registrant data. The face authentication apparatus according to claim 1, wherein it is determined whether or not a person corresponds to the registrant. The registrant data recorded in the database indicates an identification code for identifying an individual set for each registrant,
The identity determination unit identifies the registrant whose identification code input by the person to be authenticated matches the identification code indicated by the registrant data, and then identifies the facial feature of the person to be authenticated. The face authentication apparatus according to claim 1, wherein the face authentication apparatus compares the face characteristics of the registrant. An exposed mucous membrane detection unit that detects an exposed mucous membrane, which is a portion where a human has intentionally exposed the mucous membrane of the face, from an image indicated by image data generated by imaging the face of the subject,
A face part detection unit for detecting a face part, which is a predetermined part existing on a person's face, from the image,
The exposed mucous membrane detection unit sets an appearance prediction region, which is a region where the appearance of the exposed mucous membrane is predicted, based on the position of the facial part detected by the face part detection unit, and An image processing apparatus for detecting the exposed mucous membrane from the inside. An exposed mucous membrane detection unit that detects an exposed mucous membrane, which is a portion where a human has intentionally exposed the mucous membrane of the face, from an image indicated by image data generated by imaging the face of the subject,
Of each of the plurality of images indicated by the plurality of image data generated by temporally continuous imaging, each of the exposed mucosa detected by the exposed mucosa detection unit is an image resulting from human life activity. A living body determination unit that determines a state of the life activity of the subject by detecting a local change,
A living body determination apparatus comprising:

Images