JP2012003686A - Authentication device, authentication method, authentication program, and recording medium - Google Patents

Abstract

PROBLEM TO BE SOLVED: To improve the accuracy of personal authentication with a small processing load for any change of the state of a face region.SOLUTION: When performing authentication according to a face region included in image data, feature data extraction units 102, 103, and 104 receive image data including a plurality of frames, detect a face region in each of the frames, and extract, from the face region, feature data of the face region as extraction feature data. Feature data used in the authentication is registered in a personal database unit 108 as registration feature data, and an authentication unit 106 performs the authentication by comparing the registration feature data and the extraction feature data. A tracking unit 105 identifies the same face region in successive frames. If the authentication is not performed in spite of the identification of the same face region in different frames by the tracking unit, the authentication unit and a registration unit 109 register extraction feature data corresponding to the face region in the different frames in the personal database unit as additional feature data.

Description

  The present invention relates to an authentication device, an authentication method, an authentication program, and a recording medium, and more particularly to a technique for detecting a face area from image data and authenticating an individual or an individual.

  In general, a technique for performing personal authentication using a face detection function is known. In this technique, feature data of different face regions is extracted from face image data for each individual, and feature data registered in advance is compared with the extracted feature data. Then, based on the comparison result, it is identified whether or not the face image data is a registered person.

  By the way, the feature data is influenced by the facial expression, orientation, lighting conditions, and other conditions. For this reason, if the state when the feature data is registered and the state when the feature data is extracted from the face image data are different, the accuracy of the personal authentication may be lowered.

  In order to avoid such a decrease in the accuracy of personal authentication, for example, a facial expression is detected and authentication may be performed using a personal database in which feature data corresponding to the facial expression is registered. If it is an unregistered facial expression, the feature data is additionally registered (see Patent Document 1).

  Further, a plurality of face images obtained by continuously photographing a person and a plurality of face images of the same person registered in advance in the personal database are compared with a reference image selected from these face images. Then, there is one that selects a plurality of face images from the comparison result and updates the face image in the personal database (see Patent Document 2).

JP 2010-27035 A JP 2007-179224 A

  However, in the configuration disclosed in Patent Document 1, additional registration is performed only when a person is already authenticated and a facial expression whose feature data is not registered can be detected from the person. . As a result, there is a problem that the authentication accuracy cannot be increased with respect to changes in facial expressions and face orientations that cannot be detected, and changes in other conditions such as lighting conditions.

  On the other hand, in the configuration disclosed in Patent Document 2, the face image selected changes depending on how the reference image is selected. For this reason, the authentication accuracy is not stable. In addition, each time a person (that is, a face) is photographed, there is a problem that the processing load increases as a result of the personal database update process being performed.

  Accordingly, an object of the present invention is to provide an authentication device, an authentication method, an authentication program, and a recording medium that can improve the accuracy of authentication with a small processing load for any change in the state of the face area. It is in.

  In order to solve the above-described problem, an authentication apparatus according to the present invention receives an image data having a plurality of frames in an authentication apparatus that performs authentication according to a face area included in image data, and detects a face area in each of the frames. Feature data extraction means for detecting and extracting the feature data from the face area as extracted feature data, storage means for registering the feature data used in the authentication, and features registered in the storage means Authentication means for performing authentication by comparing data with the extracted feature data, tracking means for specifying the same face area in the successive frames, and specifying the same face area in different frames by the tracking means If authentication is not performed by the authentication unit despite being performed, the face corresponding to the face area in the different frame And having a registration means for registering in the storage means the characteristic data as additional feature data out.

  An authentication method according to the present invention is an authentication method used in an authentication apparatus for performing authentication in accordance with a face area included in image data. The authentication apparatus includes the feature data of the face area used in the authentication. Is stored in the storage device, receives image data having a plurality of frames, detects a face area in each of the frames, and extracts feature data from the face area as extracted feature data An extraction step; an authentication step for performing authentication by comparing the feature data registered in the storage device with the extracted feature data; a tracking step for identifying the same face region in successive frames; and the tracking step Although the same face area is specified in different frames depending on the authentication step, And having a registration step of registering in the storage device the extracted feature data corresponding to the face region as additional characteristic data in not to take the different frames.

  An authentication program according to the present invention is an authentication program used in an authentication apparatus for performing authentication in accordance with a face area included in image data. The authentication apparatus includes the facial area feature data used in the authentication. Is stored in the storage device, and the computer included in the authentication device receives image data having a plurality of frames, detects a face area in each of the frames, and extracts feature data from the face area. A feature data extraction step for extracting as extracted feature data; an authentication step for performing authentication by comparing the feature data registered in the storage device with the extracted feature data; and identifying the same face region in the successive frames The tracking step and the identification of the same face region in different frames depending on the tracking step. However, if authentication is not performed in the authentication step, a registration step of registering the extracted feature data corresponding to the face area in the different frame as additional feature data in the storage device is executed. To do.

  A recording medium according to the present invention is a computer-readable recording medium in which the authentication program is recorded.

  According to the present invention, even when image data having a plurality of frames is received and authentication is performed, it is possible to improve the accuracy of authentication when the photographing condition of the subject to be authenticated changes. is there.

It is a block diagram which shows an example of the personal authentication apparatus by the 1st Embodiment of this invention. It is a figure which shows an example of the feature data used with the personal authentication apparatus shown in FIG. 1, (a) is a figure which shows the coordinate of the feature point which comprises the feature data at the time of facing the front, (b) is a facial expression of a person FIG. 4C is a diagram showing the coordinates of feature points constituting the feature data when the angle changes, and FIG. 5C is a diagram showing the coordinates of feature points constituting the feature data when the orientation of the person is changed. It is a figure which shows an example of the characteristic data for personal authentication registered into the personal database part shown in FIG. It is a flowchart for demonstrating an example of operation | movement of the personal authentication apparatus shown in FIG. It is a figure which shows an example of the moving image data processed with the personal authentication apparatus shown in FIG. 1, (a) is a figure which shows 1 frame in moving image data, (b) is an example of the flame | frame continuous to (a). The figure shown, (c) is a figure which shows the other example of the flame | frame continuous to (a). It is a flowchart for demonstrating in detail the process in the personal authentication demonstrated in FIG. It is a figure which shows the change of the personal authentication result in the personal authentication apparatus by the 2nd Embodiment of this invention.

  Hereinafter, a personal authentication device according to an embodiment of the present invention will be described with reference to the drawings.

[First Embodiment]
FIG. 1 is a block diagram showing an example of a personal authentication device according to the first embodiment of the present invention.

  With reference to FIG. 1, the illustrated personal authentication apparatus has an image input unit 101. Image data obtained as a result of photographing at least a person's face is input from the image input unit 101. The image data is given from the image input unit 101 to the face detection unit 102 and the normalization unit 103.

  The face detection unit 102 detects the position and size of the person's face area from the image data. For example, the face detection unit 102 extracts shapes corresponding to components of the face area such as the nose, mouth, and eyes from the input image data. Then, the face detection unit 102 detects a region where the nose and the mouth exist on an extension line passing through the middle of both eyes. Then, the face detection unit 102 estimates the size of the face from the size of both eyes and the distance between the eyes, and uses the position corresponding to the center of the nose as a reference and the area surrounded by the estimated size area as the face area. To detect. This face area is given as face area detection data from the face detection unit 102 to the normalization unit 103 and the tracking unit 105.

  The normalizing unit 103 cuts out the face area from the image data in accordance with the face area detection data, and when the face area is inclined, for example, rotates the face area so as to eliminate this inclination. Further, the normalization unit 103 normalizes the face area as face image data having a predetermined angle and size by scaling the face area to a predetermined size with a predetermined distance between both eyes (hereinafter referred to as normal image). This is called converted face image data). Then, the normalization unit 103 provides the normalized face image data to the feature data generation unit 104 and the registration unit 109.

  The feature data generation unit 104 extracts feature data from the normalized face image data. For example, as described in Japanese Patent Application Laid-Open No. 2005-266981, the feature data includes information regarding specific shapes of facial components such as mouth, eyes, eyebrows, and nose, and positions of these components. including. The feature data is given from the feature data generation unit 104 to the authentication unit 106.

  Here, the feature data can be extracted from the normalized face image data by using a technique such as edge detection using a neural network or a spatial filter, for example. Note that not only information regarding the shape and position but also information regarding saturation and hue may be included in the feature data. The more feature data in a face, the more detailed the state of the face can be analyzed, and the accuracy of personal authentication using this feature data is improved.

  By the way, in image data having a plurality of frames such as a moving image, a face area is detected for each frame at a predetermined cycle. That is, the face detection unit 102 detects a face area for each frame of a predetermined cycle. The tracking unit 105 determines whether the face area detected in one frame and the area in another frame are the face area of the same person.

  For example, when a plurality of faces are detected from image data of one frame and one or more faces are detected from image data of another frame, the tracking unit (tracking unit) 105 detects each face. Those with similar size and position are considered to be the same person. Furthermore, when a face area similar to the face area detected in one frame cannot be detected in another frame, the tracking unit 105 selects a peripheral area similar in brightness and color difference pattern to the former face area. Search and track. Then, the determination result in the tracking unit 105 is given to the authentication unit 106 and the result output unit 107.

  In the personal database unit (storage means or storage device) 108, a plurality of feature data for personal authentication is registered as registered feature data together with normalized face image data. The authentication unit (authentication unit) 106 accesses the personal database unit 108 to read out feature data for personal authentication (registered feature data). Then, the authentication unit 106 performs personal authentication by comparing and collating the registered feature data with the feature data of the face area extracted by the feature data generation unit 104 (hereinafter referred to as extracted feature data). This collation result is given from the authentication unit 106 to the result output unit 107. The result output unit 107 outputs an authentication result according to the collation result.

  When the authentication unit 106 determines whether or not to perform additional registration at the time of comparison / collation, and determines that additional registration should be performed, the authentication unit 106 gives the extracted feature data to the registration unit 109. Then, the registration unit 109 additionally registers the feature data determined to be additionally registered by the authentication unit 106 in the personal database unit 108 together with the normalized face image data.

  The authentication unit 106 performs authentication according to the determination result given from the tracking unit 105, and the result output unit 107 outputs the authentication result according to the determination result given from the tracking unit 105.

  By the way, the illustrated personal authentication device may be configured as a single device or may be configured as a system including a plurality of devices. For example, all the constituent elements of the illustrated personal authentication apparatus may be provided inside a single digital camera or digital video camera.

  On the other hand, only the image input unit 101 may be disposed inside the digital camera or digital video camera, and other components of the personal authentication device may be provided in an external computer that can communicate with the digital camera or digital video camera.

  Further, the constituent elements of the illustrated personal authentication apparatus are allocated to a plurality of computers on the network, and the computer including the image input unit 101 is configured to receive image data from another external device or a recording medium. May be.

  FIG. 2 is a diagram showing an example of feature data used in the personal authentication device shown in FIG. FIG. 2A is a diagram showing the coordinates of feature points constituting the feature data when facing the front, and FIG. 2B is a feature point constituting the feature data when the facial expression of the person changes. FIG. FIG. 2C is a diagram showing the coordinates of feature points constituting the feature data when the orientation of the person changes.

  Referring to FIG. 2, here, as feature data for personal authentication, coordinates of a total of 23 feature points are used as shown in FIG. In order to actually perform personal authentication, a large number of feature points are required, but here, in order to simplify the description, description will be made assuming that a total of 23 feature points are used.

  The coordinates of these 23 feature points are calculated using the face image data normalized with the distance between the eyes for the extracted positions of the eyes, nose, mouth, and eyebrows. These coordinates are based on the position of the end point of the nose, for example. The authentication unit 106 sets Pi (i = 1, 2,..., 23) as the coordinates of each feature point calculated from the input image data. Then, the authentication unit 106 obtains an absolute value sum S = Σ | Pi−P′i | of the difference between the coordinates P′i and the coordinates Pi, which are feature points of the face area registered in the personal database unit 108 in advance.

  As the absolute value sum S is smaller, the authentication unit 106 determines that the person who is the detection target (that is, the face area) and the person registered in advance are more likely to be the same person. The authentication unit 106 determines that the person is the person when the absolute value sum S regarding the person who is determined to be most likely in the comparison / matching of the feature data is equal to or less than a preset threshold value. On the other hand, when the absolute value sum S is larger than the threshold value, the authentication unit 106 determines that there is no corresponding person.

  Note that the above-described method using the sum of absolute values S is one example of a method for performing personal authentication, and personal authentication may be performed using another method. For example, the individual may be identified from the positions of the eyes and mouth when the facial expression changes and the pattern of the shape change.

  Further, a final personal authentication result may be obtained comprehensively in accordance with the comparison / collation results for a plurality of feature data. In other words, as long as the configuration is such that the matching is performed using feature data registered in the personal database unit 108 in advance and it is determined that the person is the most likely person, the same method as in the above example can be used with other methods. An effect can be obtained.

  By the way, since the personal authentication is performed using the feature points extracted from the image data in the above-described personal authentication, as shown in FIG. 2 (b) or FIG. 2 (c), as shown in FIG. If the orientation changes, the coordinates of the feature points will change. As a result, the value of the absolute value sum S greatly fluctuates, and the accuracy of personal authentication decreases.

  In addition, changes in various shooting conditions such as lighting conditions and background affect the accuracy of personal authentication. Therefore, if a plurality of feature data obtained by photographing the same person under various conditions is registered in the personal database unit 108 in advance, and the absolute value sum S is obtained for each of the plurality of feature data, the photographing condition is satisfied. Even if it changes, the person can be authenticated with higher accuracy.

  However, taking a picture of a person under various conditions in advance takes time and effort, and in some cases, equipment such as lighting is also required. On the other hand, in the example shown in the figure, if the feature data of a person photographed under one condition is registered in the personal database, personal authentication and additional registration of feature data are automatically performed as follows. Authentication accuracy can be improved.

  FIG. 3 is a diagram showing an example of feature data for personal authentication registered in the personal database unit 108 shown in FIG.

  In FIG. 3, a plurality of feature data is registered for an individual in the personal database unit 108. A personal ID is assigned to each individual registered in the personal database unit 108. Then, a plurality of feature data is registered for the personal ID, and a registration ID is assigned to each feature data.

  In the illustrated example, the registration ID = 1 is feature data registered in advance by the user, and the registration ID = 0 indicates that the feature data is not registered. The other registration IDs indicate feature data automatically added and registered by the personal authentication device shown in FIG.

  As described above, here, the facial image data normalized by the normalizing unit 103 is also registered in the personal database unit 108 for each feature data, and the user confirms the facial image data and stores unnecessary feature data. Can be erased. For example, the normalized face image data in the registration ID = 1 of “Kotarou” with the personal ID = 1 is A11, and the feature data extracted from the face image data A11 is C11.

  Note that personal authentication is possible if any one of the face image data and the feature data is registered, and the personal authentication can be performed while generating the feature data from the face image data. In the illustrated example, the upper limit number of registration IDs is “3” for each personal ID, but this upper limit number may be “2” or more. Furthermore, the upper limit number may be set not for each individual ID but for the entire individual database unit 108.

  FIG. 4 is a flowchart for explaining an example of the operation of the personal authentication device shown in FIG.

  Referring to FIGS. 1 and 4, in the personal authentication apparatus, when image data is input to image input unit 101, the operation shown in FIG. 4 is started. If the image input unit 101 is a camera, image data obtained as a result of shooting by the camera or image data read from a recording medium attached to the camera is input to the personal authentication device. If the image input unit 101 is a personal computer, an image read from a recording medium or image data received via a network is input to the personal authentication device.

  In the following description, it is assumed that moving image data is input to the image input unit 101 and personal authentication is continuously performed at frame intervals corresponding to the time required for personal authentication.

  First, the face detection unit 102 receives image data for one frame of moving image data from the image input unit 101, and detects a human face area (step S201). Then, the face detection unit 102 determines whether or not the face area has been detected (step S202). When at least one face area can be detected (YES in step S202), normalization unit 103 normalizes each of the face areas cut out from the image data based on the detection result in face detection unit 102 (step S203). .

  Subsequently, the feature data generation unit 104 calculates feature data including the coordinates of the feature points described with reference to FIG. 5 according to the normalized face image data (step S204). The tracking unit 105 receives the detection result of the face area from the face detection unit 102, and whether there is a face that can be regarded as the same person based on the center position and size of the face among the face areas detected or determined in different frames. (Face tracking: Step S205).

  For this reason, the tracking unit 105 compares the center position and size of each face area detected in each frame and has the smallest sum of changes in the distance and size of the center position of the face area between consecutive frames. Are estimated to be face areas of the same person. For example, the tracking unit 105 compares the face areas detected in the first frame and a second frame different from the first frame to specify the face area of the same person (individual).

  However, even if the total sum is the smallest, if the value does not fall within the preset threshold range, it is determined that the tracking unit 105 is not the same person.

  If the face area cannot be detected in step S202 (NO in step S202), the process proceeds from step S202 to step S205.

  In this case, the tracking unit 105 selects either the face area detected by the face detection unit 102 in the previous frame or the area determined as the face area by the tracking unit 105 in the previous frame from the frame to be processed. A similar area is detected and determined as the face area of the same person. Specifically, an area in which the similarity between the luminance and color difference patterns is the highest and the value falls within a preset threshold range is the face area of the same person as the face area in the previous frame. Is determined.

  FIG. 5 is a diagram showing an example of moving image data processed by the personal authentication device shown in FIG. FIG. 5A is a diagram showing one frame in the moving image data, and FIG. 5B is a diagram showing an example of a frame continuous to FIG. 5A. FIG. 5C is a diagram showing another example of the continuous frames in FIG.

  Now, assume that the face area of the person 501 is detected in FIG. 5A, and the face areas of the persons 501 and 502 are detected in FIG. 5B, respectively. The tracking unit 105 compares the face area of the person 501 and the person 502 detected in FIG. 5B with the face area of the person 501 detected in FIG. As a result, the tracking unit 105 has almost no change in the center position and size of the face area, and the face area of the person 501 shown in FIG. It is determined that the face area of the person 501 shown is the same person.

  On the other hand, when the face area is not detected or the face area determined to be the same person is not found, the tracking unit 105 has similar patterns of brightness and color difference to the face area detected by the face detection unit 102 in different frames. Search the surrounding area.

  Now, assume that only the face area of the person 501 is detected in FIG. 5A and only the face area of the person 502 is detected in FIG. 5C. The tracking unit 105 compares the face area of the person 502 shown in FIG. 5C with the face area of the person 501 shown in FIG. As a result, both the face center position and size change greatly, and both do not fall within the preset threshold range, and therefore the tracking unit 105 is determined to be a different person.

  Therefore, the tracking unit 105 searches the frame in FIG. 5C for a peripheral region similar in brightness and color difference pattern to the face region of the person 501 detected in FIG. As a result of the search, the tracking unit 105 estimates that the face area of the same person is in the area having the highest similarity. However, if the similarity is not within the preset threshold range, the tracking unit 105 determines that the same person does not exist. Note that the person 501 in FIG. 5C is facing backward, and the similarity with the person 501 in FIG. 5A is out of the threshold range, so the person 501 does not exist in FIG. 5C. It will be determined.

  Furthermore, the tracking unit 105 assigns a unique person ID to each person determined to be the same person through a plurality of consecutive frames. This person ID is newly assigned every time a new person is detected, and the person ID of a person that no longer exists is deleted.

  For example, in the continuous frames shown in FIGS. 5A and 5B, a person ID is already assigned to the person 501 shown in FIG. 5A, and the face of the person 502 detected in FIG. A new person ID different from the person 1 is assigned to the area as a new person.

  Referring to FIGS. 1 and 4 again, authentication unit 106 determines whether or not a person exists based on the detection result and tracking result of the face area (step S206). If it is determined that there is at least one person in the image data (YES in step S206), the authentication unit 106 uses the facial region feature data and the feature registered in the personal database unit 108 for each person ID assigned by the tracking unit 105. Compare and collate with data. Then, the authentication unit 106 authenticates the person who is each face area (step S207).

  As will be described later, the authentication unit 106 holds a correspondence relationship between a person ID and a personal ID. The authentication unit 106 checks each person ID for each personal ID registered in the personal database unit 108 and for each registered ID, and updates the personal ID for the person ID when the corresponding individual is specified. .

  Further, the authentication unit 106 determines whether or not the feature data of the person ID should be additionally registered in the personal database unit 108. The personal authentication process in step S207 will be described later.

  Subsequently, the result output unit 107 receives the result of personal authentication from the authentication unit 106, and displays the authentication result superimposed on the image generated from the image data to be authenticated (step S208).

  In addition, about the display of an authentication result, it is not restricted to said display method, Various methods can be used. Further, when personal authentication is simultaneously performed for a plurality of face areas, it is desirable to clarify which face area each authentication result corresponds to.

  Further, the authentication unit 106 determines whether there is a person ID having feature data to be additionally registered in the personal database unit 108 (step S209). If there is a person ID having feature data to be additionally registered (YES in step S209), the registration unit 109 additionally registers the feature data determined to be additionally registered in the personal database unit 108 as additional feature data ( Step S210).

  Subsequently, the image input unit 101 determines whether another frame exists in the image data (step S211). If image data of another frame exists (YES in step S211), the image input unit 101 updates the image data to another frame (step S212). Then, the process returns to step S201, and face detection is performed on the image data updated by the face detection unit 102. On the other hand, if there is no image data of another frame, the process ends.

  If it is determined in step S206 that there is no person in the image data (NO in step S206), the process proceeds to step S211. If there is no person ID having feature data to be additionally registered in step S209 (NO in step S209), the process proceeds to step S211.

  FIG. 6 is a flowchart for explaining in detail the processing in the personal authentication described in FIG.

  Referring to FIGS. 1 and 6, authentication unit 106 associates the person ID assigned by tracking unit 105 with the personal ID registered in personal database unit 108, and associates the correspondence relationship over successive frames. to manage. Further, the authentication unit 106 determines whether or not feature data corresponding to the person ID should be additionally registered in the personal database unit 108.

  First, the authentication unit 106 determines whether or not feature data for the person ID is detected in the frame being processed (step S301). Here, the case where the feature data is not detected refers to a case where the face detection unit 102 cannot detect the face area and the tracking unit 105 determines the area with the face.

  When the feature data is detected (YES in step S301), the authentication unit 106 includes the feature data (extracted feature data) of the face area and the feature data (registered feature data) registered in the personal database unit 108. Are compared and verified to authenticate the individual (step S302). Then, the authentication unit 106 determines whether personal authentication has been performed (whether or not possible), that is, whether or not an individual has been identified (step S303).

  If the individual can be specified (YES in step S303), authentication unit 106 updates the correspondence between the person ID and the personal ID based on the personal authentication result (step S304). Then, the authentication unit 106 determines whether there is any person ID that has not been subjected to authentication processing among the person IDs assigned by the tracking unit 105 to the frame being processed (step S305).

  If there is a person ID that has not been subjected to the authentication process (YES in step S305), the authentication unit 106 updates the person ID to be processed (step S306), returns to step S301, and whether there is feature data for the person ID. Judgment is made. On the other hand, when the authentication process for all the person IDs is completed (NO in step S305), authentication unit 106 ends the process. Then, the process proceeds to step S208 described in FIG.

  If the individual cannot be identified in step S303 (NO in step S303), the authentication unit 106 determines whether there is a personal ID associated with the person ID (step S307). If there is a personal ID associated with the person ID (YES in step S307), authentication unit 106 determines that feature data relating to the person ID in the frame being processed should be additionally registered in personal database unit 108. (Step S308). That is, the authentication unit 106 determines that the feature data is a registration target. Then, the authentication unit 106 proceeds to step S305.

  The presence of a personal ID associated with a person ID means that the personal ID has already been successfully authenticated in a frame before the frame being processed. Nevertheless, in the frame being processed, the fact that the authentication unit 106 has failed in personal authentication for the person ID means that the extracted feature data in the frame being processed cannot be covered by the registered feature data of the person. It shows that. Therefore, the accuracy of personal authentication can be improved by additionally registering the extracted feature data in the frame being processed in the personal database unit 108 as new registered feature data corresponding to the personal ID.

  Therefore, every time the authentication unit 106 fails in personal authentication for a person ID that has already been successfully authenticated in the previous frame due to changes in lighting conditions and face orientation, a new data is stored in the personal database unit 108. Feature data is additionally registered. Therefore, feature data corresponding to various lighting conditions and face orientations are additionally registered in the personal database unit 108, so that the accuracy of personal authentication can be easily improved even if the photographing conditions change. become.

  In step S308, for example, when additionally registering the feature data of the person ID associated with the personal ID = 2 in the personal database unit 108 described in FIG. 3, the registration unit 109 registers the personal ID = 2. The feature data of the person ID is registered together with the image data normalized by overwriting the registration ID = 2 with one of the elements of ID = 0.

  If there is no personal ID associated with the person ID (NO in step S307), authentication unit 106 proceeds to step S305. The case where the personal ID associated with the person ID does not exist refers to the case where the corresponding individual has not been authenticated by the personal database unit 108 with respect to the person ID. If no feature data is detected in step S301 (NO in step S301), authentication unit 106 proceeds to step S305.

  As described above, according to the first embodiment, it is possible to easily improve the accuracy of personal authentication even if the shooting condition changes.

[Second Embodiment]
Next, as a second embodiment, a case where only the feature data of a person with high personal authentication reliability is additionally registered will be described. Here, the personal authentication device additionally registers only the feature data of a person with higher reliability of personal authentication so as not to register wrong person feature data for the individual. The configuration of the personal authentication device itself is the same as that of the personal authentication device shown in FIG.

  In the personal authentication device according to the second embodiment, the authentication unit 106 uses two different thresholds to change the reliability of the authentication result. That is, the authentication unit 106 performs comparison / matching of feature data using two threshold values, and sets only feature data with a high reliability of the authentication result as an additional registration target.

  Specifically, in step S302 of the personal authentication process described with reference to FIG. 6, the authentication unit 106 performs authentication when comparing / collating the feature data of the face area with the feature data registered in the personal database unit 108. Two thresholds A and B having different reliability of the result are used. Here, threshold A (second threshold) <threshold B (first threshold).

  In the personal authentication apparatus according to the second embodiment, the authentication unit 106 calculates the absolute value sum S of the feature point coordinate differences during personal authentication in the same manner as in the first embodiment. Then, if the absolute value sum S related to the person who is determined to be most likely is smaller than the threshold value B, the authentication unit 106 determines that the person is the person.

  Furthermore, if the absolute value sum S is smaller than the threshold value A (<threshold value B), the authentication unit 106 determines that the person is the person with higher reliability. For a person ID whose person has been authenticated with high reliability even once, the authentication unit 106 maintains high reliability unless the corresponding personal ID changes.

  In step S308 of the personal authentication process described with reference to FIG. 6, the authentication unit 106 has a high person ID when determining that the feature data related to the person ID should be additionally registered in the personal database unit 108. Subject to personal authentication with confidence. If the person ID is not highly personally authenticated, the authentication unit 106 does not instruct the registration unit 109 to perform additional registration in the personal database unit 108.

  FIG. 7 is a diagram showing a change in the personal authentication result in the personal authentication device according to the second embodiment of the present invention.

  Referring to FIG. 1, FIG. 3, and FIG. 7, in FIG. 7, the feature data of the face area of the person ID determined by the tracking unit 105 as the same person and registered in the personal database unit 108 in seven consecutive frames. The change of the absolute value sum S of the difference with the feature data is shown.

  Black squares 701 to 707 indicate the absolute value sum S of differences between the feature data of the face area of the person ID and the feature data C21 (FIG. 3) in seven consecutive frames. That is, the black squares 701 to 707 indicate the absolute value sum S of the difference in the coordinates of the feature points. Here, regarding the black squares 701 to 706, the absolute value sum S with the feature data C21 among the feature data registered in the personal database unit 108 is the smallest.

  When attention is paid to the black square 701, the absolute value sum S of the black square 701 is smaller than the threshold value B. Therefore, the person is authenticated as personal ID = 2 in the frame corresponding to the black square 701. Further, in the black square 702, the absolute value sum S is smaller than the threshold value A. For this reason, in the frames after the frame corresponding to the black square 702, as long as the person is tracked by the tracking unit 105, the person is treated as a person with high reliability of personal authentication.

  On the other hand, since the absolute value sum S of the black square 706 is larger than the threshold value B, the authentication unit 106 uses the personal database unit 108 to store the facial region feature data related to the person detected in the frame corresponding to the black square 706. It is determined that additional registration should be performed. Assuming that the feature data is C22 and the normalized face image data is A22, the registration unit 109 additionally registers the feature data C22 in the personal database unit 108 together with the normalized face image data A22 as the registration ID = 2 of the personal ID = 2. Will do.

  In FIG. 7, black circles 706 ′ and 707 ′ indicate the absolute value sum S of the difference between the feature data of the face area related to the person ID and the feature data C22 additionally registered in the frame corresponding to the black square 706. In the frame corresponding to the black circle 707 ′, instead of the feature data C 21, the absolute value sum S related to the additionally registered feature data C 22 is the smallest among the feature data registered in the personal database unit 108.

  In the second embodiment, two threshold values A and B having different degrees of reliability of personal authentication are used as thresholds for the sum of absolute values S. However, as long as the reliability of personal authentication can be measured, It may be a threshold value.

  In addition, it is determined that the person is the person when the sum of absolute values S is below the threshold B, but in order to avoid the determination result being output as an incorrect person, for the first time when the sum is below the threshold A, The determination result may be output from the result output unit 107 as the person.

  Furthermore, in the second embodiment, the feature data registered by the user's instruction and the additionally registered feature data are treated equally, but in order to avoid being additionally registered as an incorrect person, the former Additional registration may be performed only when personal authentication is performed with high reliability on the feature data.

  As described above, according to the second embodiment, only highly reliable feature data is additionally registered, and the accuracy of personal authentication can be further improved.

[Third Embodiment]
Next, as a third embodiment, a case will be described in which individual authentication is performed by distinguishing feature data registered in advance in the personal database unit 108 by a user's instruction and feature data additionally registered in the personal database unit 108. Here, the personal authentication device performs personal authentication using only feature data (instruction registration data) registered in advance according to the user's instruction, and if no matching person is found, the additionally registered feature data (additional registration) Data) is used for personal authentication. The configuration of the personal authentication device itself is the same as that of the personal authentication device shown in FIG.

  Specifically, the personal authentication device according to the third embodiment includes an instruction unit (not shown in FIG. 1) that instructs the user to register feature data. In response to an instruction from the instruction unit, the authentication unit 106 selects the designated face area from the face areas detected by the face detection unit 102 in the designated frame. Then, the registration unit 109 registers the feature data in the personal database unit 108 together with the personal name and normalized face image data. The registered feature data is registered ID = 1.

  In step S302 of the personal authentication process described with reference to FIG. 6, the authentication unit 106 first calculates the facial area feature data (extracted feature data) and the feature data of registration ID = 1 registered in the personal database unit 108. Compare and collate. Here, only when no matching feature data is found, the authentication unit 106 performs comparison / collation between the feature data of registration ID> 1 additionally registered in the personal database unit 108 and the extracted feature data.

  As described above, according to the third embodiment, when matching feature data is not found, comparison / collation is performed again using feature data that is additionally registered, so that it is authenticated as an incorrect person. It is possible to keep down the frequency of doing so.

[Fourth Embodiment]
Next, a case where the number of feature data that can be registered in the personal database unit 108 has already reached the upper limit will be described as a fourth embodiment. Here, the usage frequency is registered for each feature data registered in the personal database unit 108. Then, the registration unit 109 overwrites the feature data to be newly registered with the feature data with low usage frequency. The configuration of the personal authentication device itself is the same as that of the personal authentication device shown in FIG.

  Specifically, with respect to the feature data for personal authentication registered in the personal database unit 108 described with reference to FIG. 3, the number of times of coincidence with the feature data of the face area detected by the face detection unit 102 (the number of matches) and finally The matching date and time (final matching date and time) is registered for each feature data.

  In step S304 of the personal authentication process described with reference to FIG. 6, the authentication unit 106 updates the usage frequency of the feature data in the personal database unit 108 when updating the personal ID corresponding to the person ID.

  The usage frequency is updated when a new personal ID or a personal ID different from the previous one is assigned to the person ID. In this case, the authentication unit 106 increments the number of matches for the feature data of the personal database unit 108 that matches the feature data of the person ID, and further updates the last match date and time to the current date and time.

  Furthermore, in step S212 of the overall processing described with reference to FIG. 4, when the registration unit 109 additionally registers feature data in the personal database unit 108, if the number of feature data that can be registered has already reached the upper limit, the usage frequency Overwrite the low feature data and perform additional registration. That is, when newly registering additional feature data, the registration unit 109 deletes registered feature data that is less frequently used if the number of feature data that can be registered in the personal database unit 108 has reached a predetermined upper limit. Thus, additional feature data is registered.

  Here, the feature data with the oldest last match date among the feature data whose number of matches is equal to or less than a predetermined threshold is assumed to be feature data with low usage frequency. Note that the weighting of the usage frequency may be determined by giving priority to the last match date over the number of matches, and a history of authentication reliability when matching with the feature data of the detected face area is used. It may be.

  However, the feature data with registration ID = 1 registered by the user is not to be overwritten. For this reason, the update of the usage frequency and the search for the feature data with the low usage frequency need only be performed for the additionally registered feature data with the registration ID> 1.

  As described above, according to the fourth embodiment, even when the number of feature data that can be registered in the personal database unit 108 is limited, the frequency with which a registered individual can be authenticated can be increased.

  As is clear from the above description, the image input unit 101, face detection unit 102, normalization unit 103, and feature data generation unit 104 collectively function as feature data extraction means. Further, the authentication unit 106 and the registration unit 109 collectively function as a registration unit.

  Further, although the description has been made taking personal authentication using a human face area as an example, the present invention is not limited to this. If the function of detecting the face area of an animal that is a pet that has been mounted on some digital cameras is used, the same processing as described above can be executed in the individual authentication of the animal.

  As mentioned above, although this invention was demonstrated based on embodiment, this invention is not limited to these embodiment, Various forms of the range which does not deviate from the summary of this invention are also contained in this invention. .

  For example, the function of the above embodiment may be used as a personal authentication method, and this personal authentication method may be executed by a computer or the like. Further, a program having the functions of the above-described embodiments may be used as a personal authentication program, and this personal authentication program may be executed by a computer included in the personal authentication device.

  At this time, the personal authentication method and the personal authentication program have at least a feature data extraction step, an authentication step, a tracking step, and a registration step.

  The present invention can also be realized by executing the following processing. That is, software (program) for realizing the functions of the above-described embodiments is supplied to a system or apparatus via a network or various recording media, and a computer (or CPU, MPU, etc.) of the system or apparatus reads the program. It is a process to be executed.

DESCRIPTION OF SYMBOLS 101 Image input part 102 Face detection part 103 Normalization part 104 Feature data generation part 105 Tracking part 106 Authentication part 107 Result output part 108 Personal database part 109 Registration part

Claims (10)

In an authentication device that performs authentication according to a face area included in image data,
Feature data extraction means for receiving image data having a plurality of frames, detecting a face area in each of the frames, and extracting the feature data from the face area as extracted feature data;
Storage means in which the feature data used in the authentication is registered;
Authentication means for performing authentication by comparing the feature data registered in the storage means with the extracted feature data;
Tracking means for identifying the same face region in the successive frames;
The extracted feature data corresponding to the face area in the different frame is used as additional feature data if authentication is not performed by the authentication means in spite of the fact that the same face area is specified in different frames by the tracking means. An authentication apparatus comprising registration means for registering in the storage means.   In a case where one of the frames is a first frame and a frame different from the first frame is a second frame, the registration unit may include the face area and the second frame in the first frame. When the tracking means specifies that the face area in the second frame is the same as the face area in the second frame, if the authentication according to the extracted feature data corresponding to the face area in the second frame is disabled, the second area The authentication apparatus according to claim 1, wherein the extracted feature data in a frame is registered in the storage unit as the additional feature data.   If the difference between the extracted feature data and the feature data registered in the storage unit is smaller than a predetermined threshold, the authentication unit and the feature data registered in the storage unit are the same The authentication apparatus according to claim 1, wherein the authentication apparatus authenticates the person or individual.   4. The tracking unit according to claim 1, wherein the tracking unit determines whether or not the face regions are the same based on a center position and a size of the face region in different frames. The authentication device described. If the difference between the extracted feature data and the feature data registered in the storage unit is smaller than a predetermined threshold, the authentication unit and the feature data registered in the storage unit are the same Authenticate that it is an individual or individual,
A first threshold value and a second threshold value smaller than the first threshold value are set as the predetermined threshold values;
When the difference is smaller than the second threshold value in the first frame by the authentication unit, the tracking unit specifies that the face areas in the first frame and the second frame are the same If the authentication means determines that the difference exceeds the first threshold in the second frame, the registration means uses the extracted feature data in the second frame as the additional feature data. The authentication apparatus according to claim 2, wherein the authentication apparatus is registered in the storage unit.   The authentication unit executes the authentication using the feature data registered in the storage unit in advance for the authentication, and uses the additional feature data when the authentication cannot be performed using the feature data registered in the storage unit. The authentication apparatus according to claim 2, further performing the authentication. At the time of the authentication by the authentication means, the registration means registers the use frequency of the feature data registered in the storage means in the storage means,
When the registration means newly registers the additional feature data, if the number of feature data that can be registered in the storage means has reached a predetermined upper limit, the registration means deletes the less frequently used feature data. The authentication apparatus according to claim 1, wherein the additional feature data is registered. In an authentication method used in an authentication device for performing authentication according to a face area included in image data,
The authentication device includes a storage device in which feature data of the face area used for the authentication is registered as registered feature data.
A feature data extraction step of receiving image data having a plurality of frames, detecting a face area in each of the frames, and extracting the feature data from the face area as extracted feature data;
An authentication step of performing authentication by comparing the feature data registered in the storage device with the extracted feature data;
A tracking step of identifying the same face region in the successive frames;
Although the same face area is specified in different frames by the tracking step, the extracted feature data corresponding to the face area in the different frame is used as additional feature data if authentication is not performed by the authentication step. And a registration step of registering in the storage device. In an authentication program used in an authentication apparatus for performing authentication according to a face area included in image data,
The authentication device includes a storage device in which feature data of the face area used for the authentication is registered as registered feature data.
In the computer provided in the authentication device,
A feature data extraction step of receiving image data having a plurality of frames, detecting a face area in each of the frames, and extracting the feature data from the face area as extracted feature data;
An authentication step of performing authentication by comparing the feature data registered in the storage device with the extracted feature data;
A tracking step of identifying the same face region in the successive frames;
Although the same face area is specified in different frames by the tracking step, the extracted feature data corresponding to the face area in the different frame is used as additional feature data if authentication is not performed by the authentication step. An authentication program for executing a registration step of registering in the storage device.   A computer-readable recording medium on which the authentication program according to claim 9 is recorded.

Images