JP2013069155A - Face authentication database construction method, face authentication device, and face authentication program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To correctly and efficiently perform automatic registration of face image data of an authentication target user to a face authentication database when image data obtained by imaging with an imaging device includes face image data of plural users.SOLUTION: A face authentication device includes: a face image detection unit 12 for detecting face image data of a user imaged by a camera 3; a distance calculation unit 13 for calculating the position of the user in image data obtained by imaging with the camera 3 with respect to the camera 3, and calculating a distance from an operation panel 2 to the user on the basis of the calculation result; and a registration processing unit 14 for registering the face image of the user existing at a minimum distance in association with the ID of the user to a face authentication database 5 when operation is performed on the operation panel 2 and the distance from the operation panel 2 to the user is equal to or shorter than a predetermined distance Dth.

Description

  In the present invention, when there is partial image data (hereinafter referred to as “face image data”) including face portions of a plurality of users in image data captured by an imaging device such as a camera. The present invention relates to a face authentication database construction method, a face authentication apparatus, and a face authentication program that can automatically and efficiently register face image data of a user in a face authentication database.

  Conventionally, ID cards (Identity Cards) such as membership cards and employee ID cards have been issued to users who are subject to authentication, and the user identification information (hereinafter referred to as the ID card) recorded on the ID card when each user enters or exits. There is known a personal authentication system that performs personal authentication by reading a “user ID”) with a reader / writer. In addition, a personal authentication system that performs personal authentication using a user's fingerprint in order to cope with fraud such as forgery of the ID card is also known.

  However, when the above-described fingerprint authentication technology or the like is used, it is not preferable from the viewpoint of the convenience of the user because the user must be stopped and the fingerprint collected every time the user is authenticated. For this reason, recently, face authentication technology that can perform personal authentication without stopping a walking user has attracted attention. For example, in Patent Document 1, the feature amount of the user's face image data is registered as dictionary information in advance, and when the walking user approaches the camera, the user's face portion is captured by the camera. A technique is disclosed in which face image data is acquired, and the user's face authentication is performed by comparing the feature amount of the acquired user's face image data with the contents of dictionary information.

  However, according to the conventional face authentication system represented by this Patent Document 1, since the user's face image data must be registered in the face authentication database in advance, it is a burden on the user himself and the face authentication database operator. Result. This is because it takes time for the user and the operator to register the face image data obtained by photographing the face portion in the face authentication database, and the operator needs to manually edit the face image data. In particular, since the human face has the property of changing over time, the above-mentioned burden is periodically generated.

  For this reason, in Patent Document 2, when a card reader and a camera are installed near the entrance and an ID card to which a user ID is assigned is read by the card reader, the face of the person reflected in the camera An entrance / exit management device configured to automatically register image data in a face authentication database in association with a user ID is disclosed. If this patent document 2 is used, if face image data of a user is registered in association with a user ID only for the first time, the user can pass through the entrance / exit without performing an ID card reading operation for the second and subsequent times. In addition, since the user's face image data is automatically acquired and the automatically acquired face image data is automatically registered in the face authentication database in association with the user ID, the burden on the user and the operator can be reduced. .

JP 2009-104599 A JP 2005-146709 A

  However, even if the above-mentioned patent document 2 is used, if a plurality of users walk with them and the face portions of the plurality of users are reflected in the image data captured by the camera, each user's face image is displayed. It becomes unclear which data should be associated with the user ID. As a result, there is a problem that face image data corresponding to the user ID cannot be correctly registered unless the user is guided to a position where only one user can be photographed by the camera (for example, before the reader / writer).

  Therefore, when face image data of multiple users exists in the image data captured by the camera, how to accurately and efficiently perform face authentication of the face image data of the user who is the authentication target Automatic registration in the database is an important issue. Such a problem is not only a case where a user's still image is captured by a camera, but also a problem that occurs when a moving image is captured by a video camera and a still image is cut out.

  The present invention has been made to solve the above-described problems of the prior art, and in the case where face image data of a plurality of users exists in image data captured by an imaging device, the person to be authenticated It is an object of the present invention to provide a face authentication database construction method, a face authentication apparatus, and a face authentication program that can automatically and efficiently register face image data of a user to be registered in a face authentication database.

  In order to solve the above-described problems and achieve the object, the present invention provides an image pickup disposed in the vicinity of an operating device that acquires user identification information that uniquely identifies the user in response to the operation of the user. A face authentication database construction method for registering face image data of a user imaged by a device in a face authentication database in association with user identification information acquired by the operation device, whether or not the operation device has been operated An operation detection step of detecting a face image data including a face portion of the user in the image data imaged by the imaging device, and a user image of the face image data detected by the user. A distance calculating step of calculating a position from the imaging device, and calculating a distance from the operating device to the user based on the calculation result; and an operation of the operating device is detected; A registration processing step of associating the face image data of the user with the minimum distance with the user identification information acquired by the operation unit and registering it in the face authentication database when the distance to the user is equal to or less than a predetermined distance; It is characterized by including.

  The present invention also provides user face image data imaged by an imaging device arranged in the vicinity of an operating device that obtains user identification information that uniquely identifies the user in response to a user operation. A face authentication database construction method for registering in the face authentication database in association with user identification information acquired by the operation device, an operation detection step for detecting whether or not the operation device is operated, and the imaging device A tracking processing step of tracking the user while detecting the face image data including the face portion of the user in the image data captured by the step, and associating and holding a series of face image data of the tracked user; A distance calculating step of calculating a position of the tracked user from the imaging device and calculating a distance from the operating device to the user based on the calculation result; and an operation of the operating device is detected And When the distance from the operation device to the user is equal to or less than a predetermined distance, the series of face image data having the face image data of the user at the minimum distance and the user identification information acquired by the operation device. And a registration processing step of registering in the face authentication database in association with each other.

  Further, the present invention is the above-described invention, wherein the registration processing step includes registered face image data in the face authentication database associated with the same user identification information as the face image data of the user at the minimum distance, The similarity with the series of face image data is calculated, and the face image data having a predetermined similarity or higher is registered in the face authentication database.

  Further, in the present invention according to the above invention, the registration processing step calculates a similarity between the face image data of the user at the minimum distance and a series of face image data other than the face image data of the user at the minimum distance. Then, the face image data having a predetermined similarity or higher in the series of face image data is registered together with the face image data of the user having the minimum distance.

  Further, the present invention provides the registered user in the above invention, when one of a plurality of users in the image data captured by the imaging device is registered by the registration processing step. A warning output step of outputting a warning as a suspicious person when it is determined that there is a user who is within a predetermined range and is not registered by the registration processing step. Features.

  The present invention also provides an operating device that acquires user identification information that uniquely identifies the user in response to a user operation, and an imaging device that captures an image of the user disposed in the vicinity of the operating device. And a face authentication database for registering the face image data of the user imaged by the imaging device in association with the user identification information acquired by the operating device, and using the face authentication database, the user A face authentication device that performs face authentication of the user, a face image detection unit that detects face image data of the user imaged by the imaging device, and the imaging of the user in the image data captured by the imaging device A position calculation unit that calculates a position from the device and calculates a distance from the operation device to the user based on the calculation result, and an operation on the operation device is performed, and from the operation device to the user Is less than the predetermined distance If it is, characterized by comprising a registration processing unit in association with user identification information acquired by the minimum distance of the user's facial image data and the operating device is registered in the face authentication database.

  The present invention also provides an operating device that acquires user identification information that uniquely identifies the user in response to a user operation, and an imaging device that captures an image of the user disposed in the vicinity of the operating device. And a face authentication database for registering the face image data of the user imaged by the imaging device in association with the user identification information acquired by the operating device, and using the face authentication database, the user A face authentication device that performs face authentication of the user, tracking the user while detecting the face image data of the user in the image data captured by the imaging device, and a series of faces of the tracked user A tracking processing unit for associating and holding image data, a distance for calculating the position of the tracked user from the imaging device, and calculating a distance from the operation device to the user based on the calculation result Calculation unit and operation of the controller When the distance from the operation device to the user is equal to or less than a predetermined distance, the series of face image data associated with the face image data of the user with the minimum distance and the use acquired by the operation device And a registration processing unit for registering in the face authentication database in association with person identification information.

  The present invention also provides an operating device that acquires user identification information that uniquely identifies the user in response to a user operation, and an imaging device that captures an image of the user disposed in the vicinity of the operating device. And a face authentication database for registering the face image data of the user imaged by the imaging device in association with the user identification information acquired by the operating device, and using the face authentication database, the user A face authentication program of a face authentication device for performing face authentication of a user, comprising: a face image detection procedure for detecting face image data of a user imaged by the imaging device; and use in the image data imaged by the imaging device A distance calculation procedure for calculating a position of the person from the imaging device and calculating a distance from the operation device to the user based on the calculation result, and an operation on the operation device is performed. The user And a registration processing procedure for registering the face image data of the user with the minimum distance and the user identification information acquired by the operation unit in association with each other in the face authentication database when the distance at is less than a predetermined distance It is made to perform.

  The present invention also provides an operating device that acquires user identification information that uniquely identifies the user in response to a user operation, and an imaging device that captures an image of the user disposed in the vicinity of the operating device. And a face authentication database for registering the face image data of the user imaged by the imaging device in association with the user identification information acquired by the operating device, and using the face authentication database, the user A face authentication program of a face authentication device for performing face authentication of the user, tracking the user while detecting the user's face image data in the image data captured by the imaging device, and the tracked user The tracking processing procedure for associating and holding a series of face image data and the position of the tracked user from the imaging device are calculated, and the distance from the operating device to the user based on the calculation result Calculate distance calculation hand When the operation device is operated and the distance from the operation device to the user is equal to or less than a predetermined distance, the series of face image data associated with the face image data of the user with the minimum distance and A registration processing procedure for registering in the face authentication database in association with user identification information acquired by the operating device is executed by a computer.

  According to the present invention, when the operation of the operation device is detected and the distance from the operation device to the user is equal to or less than the predetermined distance, the face image of the user with the minimum distance is associated with the user identification information and face authentication is performed. Since it is configured to be registered in the database, when face image data of a plurality of users exists in the image data picked up by the image pickup device, the face image data of the user to be authenticated is accurately and efficiently Can be automatically registered in the face authentication database.

  According to the present invention, the user is tracked while detecting the face image data of the user in the image data captured by the imaging device, and the series of face image data of the tracked user is associated and held. When the operation of the operation device is detected and the distance from the operation device to the user is equal to or less than the predetermined distance, the series of face image data including the face image data of the user with the minimum distance is associated with the user identification information. Therefore, various face image data close to the real environment can be registered, and the association between the user identification information and the user's face image can be automatically registered with higher accuracy. it can.

FIG. 1 is a schematic diagram illustrating a configuration of an entrance / exit management device according to the first embodiment. FIG. 2 is a diagram illustrating an example of a face image acquired by the face image detection unit. FIG. 3 is an explanatory diagram for explaining the distance relationship among the camera, the reference image, and the user's face image when the angle of view is constant. FIG. 4 is an explanatory diagram for explaining the distance relationship between the camera and the user's face image when the angles of view are different. FIG. 5 is an explanatory diagram illustrating a state in which the face is inclined in the imaging direction. FIG. 6 is an explanatory diagram for calculating the position of the face image of the user using the distance from the camera and the angle with respect to the camera. FIG. 7 is an explanatory diagram showing the distance between the operation device and the user's face image. FIG. 8 is a flowchart showing a face image registration process procedure in the face recognition database by the entrance / exit management device shown in FIG. FIG. 9 is a schematic diagram illustrating the configuration of the entrance / exit management device according to the second embodiment. FIG. 10 is a flowchart showing a tracking processing procedure for generating a series of face image data by the tracking processing unit shown in FIG. FIG. 11 is a diagram illustrating an example of a series of face image data acquired by the tracking process. FIG. 12 is a flowchart showing a face image registration process procedure in the face recognition database by the entrance / exit management device shown in FIG.

  Exemplary embodiments of a face authentication database construction method, a face authentication apparatus, and a face authentication program according to the present invention will be described below in detail with reference to the accompanying drawings. In the following first and second embodiments, a case where the face authentication database construction method, the face authentication device, and the face authentication program according to the present invention are applied to an entrance / exit management device will be described.

  First, the features and configuration of the entrance / exit management device 1 according to the first embodiment will be described. FIG. 1 is a diagram illustrating the configuration of the entrance / exit management device 1 according to the first embodiment. An entrance / exit management device 1 shown in FIG. 1 is a device for managing entrance / exit of a user using face authentication, and is used for uniquely identifying each user in advance as in the prior art. The ID card 2a in which the person ID is recorded is issued. However, when the ID card 2a is issued, the face image data of the user is not registered. Note that personal information such as the name and affiliation of the user can be registered at the time of issuing the ID card 2a, but the description thereof is omitted here.

  Then, if the user causes the controller 2 to read the ID card 2a to enter the room for the first time, face image data including the user's face portion is cut out from the image data captured by the camera 3, and this is extracted. It is registered in the face authentication database 5 in association with the user ID of the ID card 2a. In this way, in this entrance / exit management device 1, instead of acquiring the user's face image data in a good environment in advance, only the ID card 2a is issued first, and the user's face image data is It is supposed to be acquired in a real environment. This is because the face authentication accuracy is further improved by registering the face image data of the user imaged in the actual environment in the face authentication database 5.

  Here, even if the entrance / exit management device 1 according to the first embodiment includes the face portions of a plurality of users in the image data captured by the camera 3 at the time of initial registration, The feature is that the most suitable face image data is extracted from the face image data, and the extracted face image data is associated with the user ID and registered in the face authentication database 5.

  Specifically, a distance threshold value Dth with the operation device 2 as a reference position is set in advance, and the face image data of the user located within the threshold value Dth is associated with the user ID to perform face authentication. Register in database 5. For example, when the face image data of the users M1 and M2 are included in the image data captured by the camera 3 shown in FIG. 1, the distance DX1 from the operation device 2 to the face portion of the user M1 and the operation device 2 The distance DX2 from the user to the face portion of the user M2 is calculated, and the face image data of the user M1 that becomes the distance DX1 within the threshold value Dth is registered in the face authentication database 5. Thereby, according to the entrance / exit management apparatus 1 according to the first embodiment, accurate face image data is obtained even when a plurality of user's face portions are reflected in the image data captured by the camera 3. Can be efficiently registered in the face authentication database 5.

  When the registration of the first face image data for the user is completed and the user performs the second and subsequent entry / exit, new face image data at that time is acquired and stored in the face authentication database 5. be registered. This is because the user's face part has the property of being easily changed with aging.

  As shown in FIG. 1, the entrance / exit management device 1 according to the first embodiment includes an operation device 2, a camera 3, a control unit 4, a face authentication database 5, and a gate 10. In addition, the control unit 4 includes an operation detection unit 11, a face image detection unit 12, a distance calculation unit 13, a registration processing unit 14, a similarity calculation unit 15, a face authentication processing unit 16, and an alarm output unit 17. And have. Although FIG. 1 illustrates the configuration of the entrance side of the entrance / exit management device 1, the face authentication database 5 can be commonly used on the exit side.

  The operation device 2 is a card reader that acquires the user ID of the ID card 2a held by the users M1 and M2. Specifically, when the ID card 2a is a contactless card, the user ID recorded on the ID card 2a is received by short-range wireless communication, and the ID card 2a is a magnetic recording card. Reads the user ID magnetically recorded on the ID card 2a. Note that the operation device 2 may be a numeric keypad or the like that allows the user himself / herself to input the user ID, as long as the user ID can be acquired.

  The camera 3 is an imaging device including a CCD (Charge Coupled Device) element that captures a user and outputs image data. The camera 3 is set in advance to an angle of view that can capture the face portion of the user who operates the operation device 2 and the face portion of the user on the way to the operation device 2.

  The control unit 4 is a computer that controls the entire entrance / exit management device 1, and is connected to the operation device 2, the camera 3, the gate 10, and the face authentication database 5. The control unit 4 performs at least a process of automatically registering the face image data of the user who operated the operation device 2 in the face authentication database 5 in association with the user ID. In addition, the control unit 4 performs face authentication processing of the user using the face image data registered in the face authentication database 5, and performs user entrance / exit management based on the authentication result. Specifically, the control unit 4 performs control to open the gate 10 when the user can be authenticated and maintain the closed state of the gate 10 when the user cannot be authenticated. In addition, since the well-known technique can be used for the opening / closing control of the gate 10, detailed description thereof is omitted here.

  The face authentication database 5 is a database that stores a user ID and one or a plurality of face image data in association with each other. In the face authentication database 5, the face image data of the user is not registered when the ID card 2a is given to the user, and the face image data of the user is registered when the ID card 2a is used for the first time. For this reason, it is desirable that the worker who issued the ID card 2a is present at the time of the first use of the user. The control unit 4 serves as a database management device (DBMS) for the face authentication database 5.

  The operation detection unit 11 is a processing unit that detects whether or not the operation device 2 has been operated by the user. For example, when the ID card 2a is a non-contact type card, it is detected that the user has operated the ID card 2a with the operation device 2 and receives the user ID from the ID card 2a. When the ID card 2a is a magnetic recording type card, it is detected that the user has brought the ID card 2a into contact with the operation device 2, and the user ID is read from the ID card 2a.

  The face image detection unit 12 is a processing unit that cuts out rectangular face image data including a human face portion present in image data captured by the camera 3. FIG. 2 is a diagram illustrating an example of face image data detected by the face image detection unit 12. As shown in the figure, here, the user M1 and the user M2 are included in the image data 20 captured by the camera 3, and the face image data 21 of the user M1 and the face image data of the user M2 are included. The situation where 22 is cut out is shown. Here, when the face image data of the user is cut out, a template showing the outline of a general face of a person is prepared, and this template is shifted to the image data 20 and matching is performed. Image data 21 and 22 can be detected. In addition to such template matching, various known face detection algorithms (image processing techniques) can be used.

  The distance calculation unit 13 is a processing unit that calculates the position of the user on the two-dimensional image data captured by the camera 3 and calculates the distance from the calculated position to the position of the operation device 2. The distance calculation unit 13 calculates a distance DX1 from the operation device 2 to the user M1 and a distance DX2 from the operation device 2 to the user M2 shown in FIG. The position of the user on the image data is obtained in a vector manner by the distance from the camera 3 and the angle from the optical axis of the camera 3. A detailed description of distance calculation by the distance calculation unit 13 will be described later.

  The registration processing unit 14 is a processing unit that registers user face image data in the face authentication database 5 in association with the user ID. As already described, in the first embodiment, when the ID card 2a is issued to the user, the user's face is not imaged in a different environment, but the user first uses the ID card 2a in the real environment. The user's face is imaged by the camera 3 at the time of entering the room using the and is registered in the face authentication database 5. Further, when the user enters or leaves the room for the second time or later, new face image data is additionally registered in the face authentication database 5 in association with the user ID. Specifically, when an operation using the user's ID card 2a is detected by the operation device 2 and the distance from the operation device 2 to the user is calculated, the distance is equal to or less than the threshold value Dth. The face image data of the user at the minimum distance and the user ID acquired by the operation device 2 are associated with each other and registered in the face authentication database 5.

  The similarity calculation unit 15 is a processing unit that calculates the similarity between face image data. For example, the cross-correlation coefficient between certain face image data and other face image data can be calculated, and the calculated cross-correlation coefficient can be used as the similarity. When such a cross-correlation coefficient is used as the similarity, the similarity increases when both face image data includes the face portion of the same person, and when both face image data includes a face portion of a different person, the similarity degree Becomes lower. The similarity calculation unit 15 can use a known similarity calculation technique other than the cross-correlation coefficient.

  The face authentication processing unit 16 is a processing unit that authenticates whether or not the user is a registered user registered in the face authentication database 5 by using the similarity calculated by the similarity calculation unit 15. is there. Specifically, the similarity calculation unit 15 is instructed to calculate the similarity between the face image data of the user detected by the face image detection unit 12 and the face image data registered in the face authentication database 5, and this Based on the similarity calculated by the similarity calculation unit 15, it is authenticated whether or not the user is a registered user registered in the face authentication database 5. This is because the user who has already been registered in the face authentication database 5 needs to control the opening of the gate 10 even if the ID card 2a is not operated.

  When there is only one face image data registered in the face authentication database 5, the user registers in the face authentication database 5 when the similarity between the face image data is equal to or greater than a predetermined threshold. If there are a plurality of face image data registered in the face authentication database 5, the user's face image data detected by the face image detection unit 12 and the face authentication database 5 are registered. If, for example, the top three similarities are extracted from the similarities of the plurality of face image data, and the average value of the extracted top three similarities is equal to or greater than a predetermined threshold value, the user recognizes the face authentication database. 5 can be determined to have been registered. For this determination process, an existing well-known technique can be used.

  The alarm output unit 17 is a processing unit that outputs an alarm when the face authentication processing unit 16 determines that the user has not been registered in the face authentication database 5. As the alarm output, in addition to the actual voice output, it is also possible to notify the occurrence of “abnormality” to a predetermined guard's mobile terminal. Note that the closed state of the gate 10 is maintained while the alarm is output.

Next, the concept of distance calculation by the distance calculation unit 13 will be described in detail with reference to FIGS. As shown in FIG. 3, the distance L from the camera 3 to the user's position is F as the number of pixels forming the width of the face portion on the face image data, and Ls as a preset reference distance, When the number of pixels of the width of the reference face portion set corresponding to this distance Ls is Fs,
L = (Ls × Fs) / F
It can be calculated using the following formula. The calculation formula is such that the size of the face portion in the image data captured by the camera 3 changes according to the distance from the operation device 2 to the user, and the user's position is closer to the operation device 2 and becomes larger. This is based on the fact that the position of the user becomes smaller the further away from the operation device 2. Here, the case where the number of pixels of the width of the face portion is used is shown, but the number of pixels from the head of the face to the chin, the number of pixels from the eyebrows to the mouth, or the width from the eyes to the chin It is also possible to use a predetermined width such as the number of pixels.

FIG. 3 shows the case where the angle of view αs of the reference image data and the image data of the distance calculation target is constant, but the distance calculation target is different from the angle of view α of the reference image data. When the image data is captured, it is necessary to correct the change in the angle of view. FIG. 4 is an explanatory diagram for explaining the relationship between the camera 3 and the distance to the image data when the angles of view are different. The distance L from the camera 3 to the user shown in FIG.
L = (Ls × tan (αs / 2) × Fs) / (tan (α / 2) × F)
It can be calculated using the following formula.

As shown in FIG. 5, when the face is tilted up and down or left and right, it is necessary to correct the tilt error. When the width of the face portion of the face image facing the front is F pixels, the width of the tilted face portion is F × cos β, where β is the tilt in the imaging direction of the camera 3. Therefore, the distance L is
L = (Ls × tan (αs / 2) × Fs) / (tan (α / 2) × F × cosβ)
It can be calculated using the following formula.

On the other hand, as shown in FIG. 6, the angle θ from the optical axis of the camera 3 to the user is defined as αh for the horizontal field angle of the camera 3 and N for the total number of pixels in the horizontal direction of the camera 3. If the horizontal coordinate of X is X,
θ = tan ⁻¹ ((2 × X / N) −1) × tan (αh / 2))
It can be calculated using the following formula.

  Therefore, the position (coordinates) P of the user with respect to the camera 3 can be calculated by calculating the distance L and the angle θ shown in FIG. When the user's position P can be calculated in this way, for example, as shown in FIG. 7, the position P1 of the user M1 with respect to the position P10 of the camera 3, and the position P2 of the user M2 with respect to the position P10 of the camera 3 Can be calculated. On the other hand, since the position P0 of the operating device 2 with respect to the position P10 of the camera 3 is known, the distance DX1 between the operating device 2 and the user M1 and the distance DX2 between the operating device 2 and the user M2 are calculated. be able to.

  Next, the registration processing procedure of face image data in the face authentication database 5 by the registration processing unit 14 shown in FIG. 1 will be described. FIG. 8 is a flowchart showing a registration processing procedure of face image data in the face authentication database 5 by the registration processing unit 14 shown in FIG. Here, a case where the user registers face image data for the first time using the ID card 2a is shown.

  As shown in the figure, the registration processing unit 14 first acquires the position P0 and the threshold value Dth of the operation device 2 with respect to the camera 3 (step S101). Thereafter, the image data captured by the camera 3 is acquired (step S102), and it is determined whether or not the operation device 2 has been operated by the user (step S103).

  Here, when the operating device 2 is operated by the user (step S103, Yes), it is determined whether or not a face portion is detected in the acquired image data (step S104). On the other hand, when the operating device 2 is not operated by the user (No at Step S103), the process proceeds to Step S102, and the next image data is acquired. Even when a face portion is not detected in the image data (No in step S104), the process proceeds to step S102 to acquire the next image data.

  If a face portion is detected in the image data (step S104, Yes), all face image data is transmitted to the distance calculation unit 13 to calculate the distance DX between the user and the operation device 2. (Step S105). Thereafter, it is determined whether or not the calculated distance DX is equal to or smaller than the threshold value Dth and the distance DX is the minimum distance among the acquired distances DX (step S106). When the distance DX is equal to or less than the threshold value Dth and the condition of the minimum distance is not satisfied (No at Step S106), the process proceeds to Step S102, and the next image data is acquired.

  On the other hand, when the distance DX is equal to or smaller than the threshold value Dth and the minimum distance (step S106, Yes), the user ID acquired from the operation device 2 is associated with the face image data. Then, it is registered in the face authentication database 5 (step S107). Thereafter, the process proceeds to step S102, and the next image data is acquired.

  As described above, in the first embodiment, when the operation of the operating device 2 by the user is detected and the distance DX from the operating device 2 to the user is equal to or less than the threshold value Dth, the user with the minimum distance is used. Since the face image data and the user ID acquired by the operation device 2 are associated with each other and automatically registered in the face authentication database 5, a plurality of face portions are reflected in the image data captured by the camera 3. Even if it is, the face image data of the user corresponding to the user ID can be automatically registered in the face authentication database 5 accurately and efficiently. Thus, by automatically registering the user's face image data, the burden on the user and the operator who constructs the face authentication database 5 is reduced. In addition, since a highly accurate face authentication database is constructed, the accuracy of the face authentication process itself is improved.

  Incidentally, in the first embodiment, since only the face image data of the user who has moved before the operation device 2 is registered in the face authentication database 5, only the face image data at the time when the user operates the operation device 2 is used. Is registered in the face authentication database 5. As a result, there is a problem in that the authentication processing result is lowered if the user's proper front face image data is not captured when the user operates the controller 2. In particular, the camera 3 and the operation device 2 are not always integrated like an interphone, and the orientation of the operation device 2 may be different from the orientation of the camera 3 due to construction problems. Is facing sideways with respect to the camera 3, the user turns sideways when the user operates the operation device 2, and the front face at the moment when the user operates the operation device 2 is displayed. It becomes difficult to obtain with the camera 3. In addition, when performing the face authentication process, it is desirable to register a plurality of face image data and improve the authentication accuracy. Therefore, even if the user operates the operation device 2, the face image data of the front face is stored. Even if it can be acquired, it is desirable to perform batch registration including other face image data.

  Therefore, as shown in FIG. 9, the entrance / exit management apparatus 101 according to the second embodiment tracks the user while detecting the face portion of the user in the image data captured by the camera 3, and uses the tracked usage. A tracking processing unit 18 is further provided for holding a series of face image data including a person's face portion in association with each other. The distance calculation unit 13 calculates the position of the tracked user from the camera 3 and calculates the distance from the controller 2 to the user based on the calculation result. When the user operates the operation device 2 and the distance DX from the operation device 2 to the user is equal to or less than the threshold value Dth, the registration processing unit 14 stores the face image data of the user at the minimum distance. A series of face image data including the image data is registered in the face authentication database 5 in association with the user ID read by the operation device 2. As a result, face images of various orientations and sizes, and a series of face image data suitable for lighting in the real environment are automatically registered in the face authentication database 5 to improve face authentication accuracy.

  First, with reference to the flowchart shown in FIG. 10, a user tracking processing procedure by the tracking processing unit 18 will be described. The tracking processor 18 first acquires image data captured by the camera 3 (step S201). Thereafter, it is determined whether or not a face portion is detected in the image data (step S202). When the face portion is not detected (No at Step S202), the process proceeds to Step S201, and new image data is acquired.

  On the other hand, when the face portion is detected (step S202, Yes), the face image data is held (step S203). Thereafter, it is determined whether or not the tracking condition is satisfied (step S204). This tracking condition is that the user can move within the image data within the acquisition time interval of each image data. For example, if the average moving speed of the user is 10 m / sec and the time for the camera 3 to process one frame is 30 frames / sec, the moving range of the user per frame is 10 (m / sec) ÷ 30 (sheets / second) = 0.33 m. The range within 0.33 m is a movable range.

  If the tracking condition, that is, if the user in the next image data is within the movable range (step S204, Yes), the face image data that satisfies this tracking condition is included in the current image including the previous face image data. The process is held in association with the face image data being tracked so far (step S205), and then the process proceeds to step S204, where it is repeatedly determined whether or not the tracking condition is satisfied at the position where the next face image data is detected. Hold face image data. On the other hand, if the tracking condition is not satisfied (step S204, No), the entire face image data being tracked is deleted (step S206), and the process is terminated.

  Therefore, in the tracking process, as long as the tracking condition is satisfied, a plurality of face image data including the face portion of the same user is held as a series of related face image data regardless of the state of the face portion. For example, as shown in FIG. 11, face image data 31 to 34 satisfying the tracking condition are held as a series of face image data 30 regardless of the state of the face portion.

  Then, in the registration processing unit 14, the distance DX between the user M1 of the detected face image data and the operation device 2 is determined by the face image registration processing procedure shown in the flowchart of FIG. It is determined whether the distance is equal to or smaller than the threshold value Dth (step S306). Note that the processes from step S301 to S306 are the same as those in steps S101 to S106. If the distance DX is equal to or less than the threshold value Dth and the minimum distance (step S306, Yes), the user ID read by the controller 2 and the face image data that satisfies the determination condition of step S306. Are registered in the face authentication database 5 in association with each other (step S307). Thereafter, the process proceeds to step S302, and the above series of processing is repeated.

  When the person A is walking just behind the user M1, the face image of the person A may be mixed in the tracking result of the user M1. In this case, when the user M1 operates the operation device 2, the face image data of the person A is mistakenly associated with the user ID of the user M1, and this may cause erroneous authentication and degradation of identification performance. Therefore, it is necessary to take measures not to register the face image data mixed in the tracking result in the face authentication database 5 by mistake.

  As a first countermeasure, the similarity between the face image data in the tracking result and all face image data of the user M1 stored in the face authentication database 5 is calculated, and the user is calculated from the similarity. Only the face image data determined as M1 may be registered in the face authentication database 5 in association with the user ID of the user M1.

  As a second countermeasure, on the contrary, the similarity between the face image data in the tracking result and all the face image data of the user M1 stored in the face authentication database 5 is calculated. Registration processing in the face authentication database 5 is not performed for face image data determined as a person other than the user M1 from the user M1, and the other face image data is associated with the user ID of the user M1 in the face authentication database 5. You should register.

  As a third countermeasure, first, as in the first embodiment, out of all face image data in the tracking result, face image data located at a distance closest to the operation device 2 is determined as the user M1. Furthermore, the face image data similar to the face image data determined as the user M1 among the other face image data in the tracking result is determined as the user M1. By performing such a two-step process, the face image data determined as the user M1 may be registered in the face authentication database 5 in association with the user ID of the user M1.

  As described above, in the second embodiment, when the operation device 2 is operated and the distance DX from the operation device 2 to the user is equal to or less than the threshold value Dth, the face image data of the user at the minimum distance is obtained. Are registered in the face authentication database 5 in association with the user ID read by the operation device 2 and, as described above, various orientations during walking, The face image data of a size can be registered in the face authentication database 5 at once. As a result, it is possible to efficiently and automatically register face image data suitable for actual environment lighting and flow lines to be used. In other words, it is possible to automatically construct a face authentication database corresponding to the actual environment, rather than a database constructed by normalizing the size of a face, estimating illumination fluctuations, and estimating the face orientation by a conventional system, The burden on the user and the construction worker can be reduced, and the face authentication accuracy can be improved.

  In addition, in the said Example 1 and 2, although the case where this invention was applied to the entrance / exit management system was shown, this invention is not limited to this, It can apply to various systems, such as an attendance management system. . Further, by using a card reader as the operation device 2, face authentication can be introduced into entrance / exit management and attendance management that have been realized only by using an IC card such as an employee ID card. Since the face image data for face authentication is automatically registered, the user can unlock the electric lock and stamp the time card without having to hold the ID card. Furthermore, the image at the time of unlocking (at the time of engraving) and the user ID can be registered in association with each other. As a result, the check by the administrator can be performed more strictly, and fraud can be prevented. In addition, the use of the IC card and the face verification process can be double-checked. By performing this double check, a system with higher security than usual can be realized.

  Moreover, in the said Example 1 and 2, although the case where the operation device 2 was used as the card reader was shown, this invention is not limited to this. The operation device 2 only needs to be able to exchange ID information, and can be applied even if it is not an IC card such as an employee ID card. For example, a web camera can be attached to a display of a personal computer (PC), and face image data can be registered so as to associate a user name, password, and face image data when logging in. Also, face image data can be registered by associating face image data with an unlocking password for an auto-lock installed at the entrance of the apartment.

  Furthermore, in the first embodiment, the distance calculation unit 13 performs the position calculation process using the image data and registers only the user who is closest to the operation device 2 in the face authentication database 5. By using a person's position and the authentication result, it is possible to detect the sharing of multiple persons. For example, when an unauthenticated person is approaching from the back of a registered user, the user M1 is notified as a companion of a suspicious person, and the companion prevention such as not unlocking the doorway is performed. You can also. In this case, the warning output unit 17 outputs a warning that the suspicious person is present.

  The face authentication database construction method, the face authentication device, and the face authentication program according to the present invention are suitable for an entrance / exit monitoring system, an attendance management system, and the like, and in particular, authenticate from image data in which face image data of a plurality of users exists. It is suitable for the case where face image data of a target user is accurately and efficiently extracted and automatically registered in the face authentication database.

DESCRIPTION OF SYMBOLS 1,101 Entrance / exit management apparatus 2 Controller 2a ID card 3 Camera 4 Control part 5 Face authentication database 10 Gate 11 Operation detection part 12 Face image detection part 13 Distance calculation part 14 Registration process part 15 Similarity degree calculation part 16 Face authentication process Unit 17 Alarm output unit 18 Tracking processing unit 20 Image data 21, 22 Face image data M1, M2 User Dth threshold value DX1, DX2 Distance

Claims (9)

Acquire user face image data captured by an imaging device arranged in the vicinity of an operation device that acquires user identification information that uniquely identifies the user in response to a user operation, using the operation device. A face authentication database construction method for registering in the face authentication database in association with the user identification information,
An operation detecting step for detecting whether or not the operating device has been operated;
A face image data detection step of detecting face image data including a user's face portion in the image data imaged by the imaging device;
Calculating a position of the user from which the face image data is detected from the imaging device, and calculating a distance from the operation device to the user based on the calculation result;
When the operation of the operation device is detected, and the distance from the operation device to the user is equal to or less than a predetermined distance, the face image data of the user at the minimum distance and the user identification information acquired by the operation device, And a registration processing step of registering in the face authentication database in association with each other. Acquire user face image data captured by an imaging device arranged in the vicinity of an operation device that acquires user identification information that uniquely identifies the user in response to a user operation, using the operation device. A face authentication database construction method for registering in the face authentication database in association with the user identification information,
An operation detecting step for detecting whether or not the operating device has been operated;
Tracking for tracking the user while detecting the face image data including the face portion of the user in the image data captured by the imaging device, and associating and holding the series of face image data of the tracked user Processing steps;
Calculating a position of the user being tracked from the imaging device, and calculating a distance from the operation device to the user based on the calculation result; and
When the operation of the operation device is detected and the distance from the operation device to the user is equal to or less than a predetermined distance, the series of face image data having the face image data of the user at the minimum distance and the operation device And a registration processing step of registering the acquired user identification information in the face authentication database in association with the acquired user identification information.   In the registration processing step, the similarity between the registered face image data in the face authentication database associated with the same user identification information as the face image data of the user at the minimum distance and the series of face image data The face authentication database construction method according to claim 2, wherein face image data having a predetermined similarity or higher in the series of face image data is registered in the face authentication database.   The registration processing step calculates a similarity between the face image data of the user at the minimum distance and a series of face image data other than the face image data of the user at the minimum distance, and among the series of face image data, The face authentication database construction method according to claim 2, wherein face image data having a predetermined similarity or higher is registered together with the face image data of the user at the minimum distance.   A user whose distance from the registered user is within a predetermined range when one of a plurality of users in the image data captured by the imaging device is registered by the registration processing step. 5. The method according to claim 1, further comprising an alarm output step of outputting an alarm as a suspicious person if it is determined that there is a user who is not registered by the registration processing step. The face authentication database construction method as described in any one. An operating device that acquires user identification information that uniquely identifies the user in response to an operation of the user, an imaging device that images the user disposed in the vicinity of the operating device, and the imaging device A face authentication database that associates and registers imaged user face image data and user identification information acquired by the operation unit, and performs face authentication of the user using the face authentication database An authentication device,
A face image detection unit that detects face image data of a user imaged by the imaging device;
A distance calculation unit that calculates a position of the user from the imaging device in the image data captured by the imaging device, and calculates a distance from the operation device to the user based on the calculation result;
When the operation device is operated, and the distance from the operation device to the user is equal to or less than a predetermined distance, the face image data of the user at the minimum distance and the user identification information acquired by the operation device; And a registration processing unit for registering in the face authentication database in association with each other. An operating device that acquires user identification information that uniquely identifies the user in response to an operation of the user, an imaging device that images the user disposed in the vicinity of the operating device, and the imaging device A face authentication database that associates and registers imaged user face image data and user identification information acquired by the operation unit, and performs face authentication of the user using the face authentication database An authentication device,
A tracking processing unit for tracking the user while detecting face image data of the user in the image data captured by the imaging device, and associating and holding a series of face image data of the tracked user;
Calculating a position of the user being tracked from the imaging device, and calculating a distance from the operation device to the user based on the calculation result;
When the operation device is operated and the distance from the operation device to the user is equal to or less than a predetermined distance, the series of face image data associated with the face image data of the user with the minimum distance and the operation A face authentication apparatus comprising: a registration processing unit that associates with user identification information acquired by a device and registers the information in the face authentication database. An operating device that acquires user identification information that uniquely identifies the user in response to an operation of the user, an imaging device that images the user disposed in the vicinity of the operating device, and the imaging device A face authentication database that associates and registers imaged user face image data and user identification information acquired by the operation unit, and performs face authentication of the user using the face authentication database An authentication device face authentication program,
A face image detection procedure for detecting face image data of a user imaged by the imaging device;
A distance calculation procedure for calculating a position of the user in the image data captured by the imaging device from the imaging device, and calculating a distance from the operation device to the user based on the calculation result;
When the operation device is operated, and the distance from the operation device to the user is equal to or less than a predetermined distance, the face image data of the user at the minimum distance and the user identification information acquired by the operation device; And a registration processing procedure for registering in the face authentication database in association with each other. An operating device that acquires user identification information that uniquely identifies the user in response to an operation of the user, an imaging device that images the user disposed in the vicinity of the operating device, and the imaging device A face authentication database that associates and registers imaged user face image data and user identification information acquired by the operation unit, and performs face authentication of the user using the face authentication database An authentication device face authentication program,
A tracking processing procedure for tracking the user while detecting the user's face image data in the image data captured by the imaging device, and associating and holding a series of face image data of the tracked user;
Calculating a position of the user being tracked from the imaging device, and calculating a distance from the operation device to the user based on the calculation result;
When the operation device is operated and the distance from the operation device to the user is equal to or less than a predetermined distance, the series of face image data associated with the face image data of the user with the minimum distance and the operation A face authentication program that causes a computer to execute a registration processing procedure for registering in the face authentication database in association with user identification information acquired by a device.

Images