JP2010086263A - Biometrics matching device, face matching device and biometrics matching method - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide a biometrics matching device which registers biometrics information of a lot of registrants without degrading matching speed and matching precision. <P>SOLUTION: The biometrics matching device stores biometrics information of registrants in the dictionary D, associating the biometrics information with feature information other than biometrics information that can be captured at a time of authentication. When biometrics information of an authentication subject is detected, the device extracts feature information from a captured image of the authentication subject other than biometrics information, selects biometrics information associated with the feature information which matches the extracted feature information from the dictionary D, and compares the selected biometrics information with the biometrics information of the subject to be authenticated. <P>COPYRIGHT: (C)2010,JPO&INPIT

Description

  The present invention relates to a biometric matching device, a face matching device, and a biometric matching method that perform person authentication by biometric matching using biometric information unique to each person such as a face image.

  Conventionally, a biometric matching device such as a face matching device performs person authentication by a matching technique (so-called biometric matching) using the biometric information of each person. In particular, a facial image as biometric information has a strong feature that it changes while maintaining the uniqueness of each person as compared to biometric information such as fingerprints, vein patterns, or irises. For this reason, in the face collation apparatus, the number of registrants to be collated is often limited in order to increase the efficiency of the degree of coincidence of face collation based on statistical knowledge. In such a face collation apparatus with a limited number of registrants to be collated, if there are more than a predetermined number of registrants, as an operational solution, categorize the groups so that the number of registrants is less than the predetermined number In addition, there is a method of installing a plurality of face collation devices so that each face image device has a registered number of 300 or less. It is often difficult to install a plurality of face matching devices in terms of construction or cost.

Further, as a technical solution for increasing the maximum number of registrants to be collated, it is conceivable to upgrade the face collation processing unit. For example, it is conceivable to increase the data amount of the face image to be photographed and the data amount of the face image to be registered, or to improve the image quality of the face image such as higher resolution. However, in reality, when the processing unit is advanced, the apparatus becomes expensive and the processing time becomes unsuitable for practical use. That is, it is not practical from the viewpoint of practicality to upgrade the processing unit in order to increase the maximum number of registrants to be verified.
JP 2003-141542 A

  An object of one aspect of the present invention is to provide a biometric matching device, a face matching device, and a biometric matching method that can register biometric information of a large number of registrants without impairing the matching speed or matching accuracy.

  A biometric matching device as one aspect of the present invention includes a first acquisition unit that acquires information including biometric information, and a second that acquires information including feature information other than the biometric information for grouping persons. Acquisition means, storage means for storing biometric information of a plurality of registrants in association with feature information other than the biological information included in information acquired by the second acquisition means, and acquisition by the first acquisition means The biometric information detecting means for detecting the biometric information of the person to be authenticated from the information obtained, and when the biometric information of the authentication target person is detected by the biometric information detecting means, the authentication target person from the information acquired by the second acquiring means The feature detection means for detecting feature information other than the biological information, and the registrants stored in the storage means are narrowed down based on the feature information other than the biological information detected by the feature detection means Includes means exits, and a biometric matching means for matching the biometric information of the registered person stored in the elected said memory means by said selecting means and detected biometric information by the biometric information detecting means.

  A biometric matching device as one aspect of the present invention includes a first acquisition unit that acquires information including biometric information, and a second that acquires information including feature information other than the biometric information for grouping persons. Acquisition means, storage means for storing biometric information of a plurality of registrants in association with feature information other than the biological information included in information acquired by the second acquisition means, and acquisition by the first acquisition means The biometric information detecting means for detecting the biometric information of the person to be authenticated from the obtained information, the biometric information of the authentication target detected by the biometric information detecting means and the biometric information of the registrant stored in the storage means are collated. When there are a plurality of biometric information of the registrant who is determined to be the same person as the biometric information of the person to be authenticated by the biometric verification means and the biometric verification means, the information acquired by the second acquisition means A feature detection unit that detects feature information other than the biometric information of the authentication target person, and feature information other than the biometric information of the authentication target person detected by the feature detection unit and the same person as the authentication target person And determining means for determining the result of the biometric matching process by comparing the determined registrant with feature information other than the biometric information.

  According to another aspect of the present invention, there is provided a face collation apparatus that obtains an image including feature information other than the face image information for grouping persons, and a first acquisition unit that acquires an image including face image information. Two acquisition means, storage means for storing face image information of a plurality of registrants in association with feature information other than the face image information included in an image acquired by the second acquisition means, and the first A face detecting unit for detecting face image information of the person to be authenticated from the image acquired by the acquiring unit; and when the face image information of the person to be authenticated is detected by the face detecting unit, from the image acquired by the second acquiring unit A feature detection unit that detects feature information other than the face image information of the authentication target person, and a registrant stored in the storage unit based on the feature information other than the face image information detected by the feature detection unit Included It has a selecting means, and a face recognition unit that collates the registrant's face image information stored in the elected said memory means by said selecting means and detected face image information by the face detection means.

  In the biometric matching method of the present invention, biometric information of a plurality of registrants is stored in a storage means in association with feature information other than the biometric information for grouping each registrant, and first information including biometric information is included. When the biometric information is detected from the acquired first information, and the biometric information of the person to be authenticated is detected by this detection, the second information including the characteristic information other than the biometric information of the authentication target person is obtained. And the feature information other than the biometric information of the person to be authenticated is detected from the obtained second information, and stored in the storage means based on the feature information other than the biometric information detected by the detection. The registered registrant is narrowed down, and the authentication subject's biometric information is collated with the registrant's biometric information stored in the narrowed-down storage means.

  According to one aspect of the present invention, it is possible to provide a biometric matching device, a face matching device, and a biometric matching method that can register biometric information of a large number of registrants without impairing the matching speed or matching accuracy.

The best mode for carrying out the present invention will be described below with reference to the drawings.
The embodiment described below can be applied to a biometric matching apparatus that performs personal authentication by matching biometric information of a person (biometric matching). In particular, as will be described later, the present embodiment can be applied to a biometric matching apparatus that can acquire biometric information used for biometric matching and feature information other than the biometric information. However, in the following description, as an embodiment, a face matching device that performs personal authentication or traffic control by matching (face matching) of a human face image (face feature information obtained from a moving image of a face) as biometric information. explain. That is, in the embodiments described later, the face image, face collation or face collation device can be realized by replacing it with biometric information, biometric collation or biometric collation device or the like.

1A and 1B are diagrams showing an example of an external configuration of a face matching device 1 as a biometric matching device according to an embodiment of the present invention. FIG. 1A is a front view showing an external configuration example of the face matching device 1. FIG.1 (b) is a side view which shows the external appearance structural example of the face collation apparatus 1 shown to Fig.1 (a).
As shown in FIG. 1A, various units as a user interface are provided on the front of the face matching device 1. In the example shown in FIG. 1A, the front of the face matching device 1 includes a first camera 11, a second camera 12, a display unit 13, an operation unit 14, an audio unit 15, and the like.

  The first camera 11 is constituted by a digital video camera or the like. The first camera 11 captures an image of a specific area (imaging range) according to the installation position. The first camera 11 shoots an image including at least a person's face existing in the shooting range at the time of collation processing using a face image or registration of a face image. For example, the first camera 11 captures a human face image as a moving image. The first camera 11 may always take an image, or may take an image in accordance with a specific control signal such as person detection. The first camera 11 may be color or monochrome.

  The second camera 12 has a shooting range substantially similar to that of the first camera 11. The second camera 12 is a camera that photographs a person photographed by the first camera 11. The second camera 12 captures an image including a feature other than a human face (which may include a face image). For example, the second camera 12 shoots an item worn by each person such as clothes (uniforms, work clothes, etc.), a hat, a name tag, or an armband as features other than the face. The features other than the face photographed by the second camera 12 do not need to uniquely identify each person, but group all registrants so that the number of persons to be collated is a predetermined number or less. I just need it. The second camera 12 may capture a moving image, but in the present embodiment, the second camera 12 will be described as capturing a still image.

  The first camera 11 and the second camera 12 are installed so that one person can shoot at the same time or almost the same time. In particular, the first camera 11 and the second camera 12 are arranged so that the person whose face is photographed by the first camera 11 and the person who is photographed by the second camera 12 cannot be interchanged. This is a configuration for establishing the same identity between the person whose face is photographed by the first camera 11 and the person whose face is photographed by the second camera 12. That is, the first camera 11 and the second camera 12 may have any form that can determine that the person photographed by the first camera 11 and the person photographed by the second camera 12 are the same person.

  The display unit 13 is composed of, for example, a liquid crystal display device. The display unit 13 displays a face image being photographed by the first camera 11, a guidance display, a result of face matching, or the like. The operation unit 14 includes a group of hard keys such as numeric keys or function keys. The operation unit 14 is for the user to input an ID number or a password as personal identification information, for example. The voice unit 15 includes a speaker that outputs voice such as voice guidance and a microphone that inputs voice uttered by the user.

  Further, as shown in FIG. 1B, an interface unit 16 is provided on the side surface or the back surface of the face matching device 1. The interface unit 16 includes various interfaces. For example, the interface unit 16 is provided with a network interface or an interface for outputting a traffic control signal to a traffic control system described later.

The face matching device 1 includes a control unit and a storage device. The control unit is connected to the units 11-16. For example, the control unit is connected to various external devices via the interface unit 16. The control unit has a function of controlling each part and performing various processes described later. For example, the control unit includes a CPU and various memories, and the CPU realizes various functions by executing a control program stored in the memory.
Further, the face matching device 1 may be provided with a connection terminal 17 whose contact is short-circuited when the face matching process is successful. For example, the connection terminal 17 may be used when there is a device that performs some operation in conjunction with the result of the face matching process of the face matching device 1.

Next, the configuration of a face matching system (traffic control system) including the face matching device 1 will be described.
FIG. 2 is a diagram schematically showing a configuration example of a face matching system (traffic control system) including the face matching device 1.
As shown in FIG. 2, the face matching system includes a face matching device 1, a network 2, a traffic control device 3, and the like. As described above, each part of the face matching device 1 is controlled by a built-in control unit. The control unit of the face matching device 1 is connected to the network 2 and the traffic control device 3 through various interfaces.

Connected to the network 2 are a monitoring and management PC (monitoring terminal) 2A and a data server 2B for holding data. That is, the face matching device 1 can communicate with the monitoring terminal 2A or the data server 2B via the network 2.
The traffic control device 3 controls the traffic of a person according to the result of face matching by the face matching device 1. In the example shown in FIG. 2, the traffic control device 3 is configured by a door or the like provided with an electric lock 3 </ b> B controlled by an electric lock control panel 3 </ b> A. Moreover, when the said traffic control apparatus 3 is comprised with the electric lock 3B installed in the door, the said control unit 21 is connected to 3 A of electric lock control panels which control the electric lock 3B. The traffic control device 3 may be configured by an automatic door or the like. When the traffic control device 3 is composed of an automatic door, the control unit of the face matching device 1 is connected to a door opening / closing mechanism (not shown) that opens and closes the automatic door via an interface.

Next, the configuration of the control system of the face matching device 1 will be described.
FIG. 3 is a block diagram schematically showing a configuration example of a control system (control unit) of the face collation apparatus 1.
As shown in FIG. 3, the face matching device 1 includes a control unit 21, a hard disk drive (HDD) 22 as a storage device, a power supply unit 23, and the like in addition to the above-described configuration.
The control unit 21 includes a CPU 30, a ROM (Read Only Memory) 31, a RAM (Random Access Memory) 32, a non-volatile memory 33, a first image input unit 34, a second image input unit 35, a display control unit 36, and an audio control. Unit 37, key input control unit 38, communication control unit 39, output control unit 40, and the like. The control unit 21 is configured as, for example, a processing boat in which each part is mounted on a substrate.

  The CPU 30 controls the entire control unit 21. The ROM 31 is a nonvolatile memory. The ROM 31 functions as a program memory that stores control programs or control data. The RAM 32 is a volatile memory. The RAM 32 functions as a working memory that temporarily stores work data and the like. The nonvolatile memory is a rewritable nonvolatile memory such as an EEPROM or a flash ROM. The nonvolatile memory functions as a rewritable nonvolatile memory that stores application programs or system setting information. The CPU 30 realizes various processing functions by executing control programs stored in the ROM 31, the nonvolatile memory 33, or the HDD 22. The processing functions realized by the CPU 30 will be described in detail later.

  The first image input unit 34 is an interface for inputting an image taken by the first camera 11. In the present embodiment, it is assumed that the first camera 11 captures a human face image or the like as a moving image. Therefore, the first image input unit 34 is composed of an image interface capable of inputting a moving image. The second image input unit 35 is an interface for inputting an image taken by the second camera 12. In the present embodiment, it is assumed that the second camera 12 captures a still image. Therefore, the second image input unit 35 is composed of an image interface for inputting a still image.

  The display control unit 36 is connected to the display unit 13. The display control unit 36 controls the display content of the display unit 13. The voice control unit 37 is connected to the voice unit 15. The sound control unit 37 controls sound generated by a speaker as the sound unit 15. In addition, the voice control unit 37 performs control to input voice with a microphone as the voice unit 15. The key input control unit 38 is connected to the operation unit 14. The key input control unit 38 is an interface that takes in information input by each key as the operation unit 14 into the control unit 21.

  The communication control unit 39 is connected to a network communication connector provided in the interface unit 16. The communication control unit 39 controls data communication via the network 2. The output control unit 40 is connected to an authentication result output connector provided in the interface unit 16. The output control unit 40 outputs a control signal based on the result of face matching. In the system as shown in FIG. 2, the output control unit 40 is connected to an electric lock control panel 3 </ b> A as the traffic control device 3 through the interface unit 16. That is, the output control unit 40 outputs a control signal based on the result when face matching is successful.

  The hard disk drive (HDD) 22 is a device that stores various data. The HDD 22 has a dictionary D composed of dictionary data for each registrant. The dictionary D is associated with each registrant's dictionary data such as the registrant's face image, facial feature amount, feature information other than facial feature information (still image feature) information, personal identification information, and a password. Is remembered. Further, as will be described later, the dictionary D is stored in the HDD 22 in a grouped state based on still image feature information. The power supply unit 23 supplies power to each unit such as the control unit 21 and the HDD 22.

Next, the configuration of the face matching processing function in the face matching device 1 will be described.
FIG. 4 is a block diagram illustrating a configuration example of the face matching processing function in the face matching device 1.
As described above, the face collation apparatus 1 realizes various processing functions by the CPU 30 executing various control programs. The face matching process is also realized by the CPU 30 executing a face matching program. FIG. 4 is a block diagram illustrating various functions for performing face matching processing. For this reason, each part shown in FIG. 4 is also a function implement | achieved when the said CPU30 runs a program.

As shown in FIG. 4, the face collation processing function in the face collation apparatus 1 includes a first image acquisition unit (moving image acquisition unit) 51, a face detection unit 52, a face feature extraction unit 53, a face collation unit 54, and a second image. An acquisition unit (still image acquisition unit) 55, a feature detection unit 56, a dictionary selection unit 57, and the like are included.
The first image acquisition unit 51 is an interface for acquiring an image photographed by the first camera 11. In the present embodiment, it is assumed that the first camera 11 captures a moving image composed of a plurality of frames of images that are continuous at a predetermined interval. In this case, the first image acquisition unit 51 includes an image input interface that acquires a moving image. That is, the first image acquisition unit 51 acquires image data including the face of a person (photographed person) photographed by the first camera 11 as a moving image. The first image acquisition unit 51 outputs a moving image captured by the first camera 11 to the face detection unit 52.

The face detection unit 52 performs a process of detecting a candidate area (face area) where a human face is present from an image of each frame as a moving image acquired by the first image acquisition unit 51. The face detection unit 52 outputs information indicating the detected face area to the face feature extraction unit 53. Further, when a human face is detected, the face detection unit 52 notifies the second image acquisition unit 55 or the second camera 12 of information indicating that a human face has been detected. .
In the face detection unit 52, various methods can be applied as a method for detecting a face region from the supplied image. For example, the face detection unit 52 can apply a method of searching a pattern having a high similarity with a dictionary pattern for detection created from a face learning pattern in advance as a face area from an input image.

  The face feature extraction unit 53 extracts face feature information from the face area image detected by the face detection unit 52. That is, the face feature extraction unit 53 extracts face feature information from the face image of the person photographed by the first camera 11. For example, the face feature extraction unit 53 cuts out the face image of each frame into a certain size and shape with reference to the face feature points, and uses the shade information as a feature amount (face feature information) in each frame. To do. The face feature extraction unit 53 accumulates face feature information (feature amount of each frame) calculated from the face image of each frame, and calculates face feature information of the person (photographed person) from the set.

  Here, the gray value in an image area of m pixels × n pixels cut out from each frame is used as a feature amount as it is. This gray value is m × n-dimensional information. It is assumed that a gray value composed of m × n-dimensional information is a feature vector of each frame. Then, a plurality of feature vectors are obtained from the face image of each frame. In the present embodiment, a partial space obtained from these feature vectors is used as facial feature information of the person. The subspace is calculated by obtaining a correlation matrix (or covariance matrix) of feature vectors obtained from each frame and obtaining an orthonormal vector (eigenvector) by KL expansion. Here, in the subspace, k eigenvectors corresponding to eigenvalues are selected in descending order of eigenvalues and expressed using the eigenvector set.

  The face feature extraction unit 53 obtains the correlation matrix Cd from the feature vector and diagonalizes it with the correlation matrix Cd = ΦdΛdΦdT to obtain the eigenvector matrix Φ. This partial space is used as face feature information for collating face images. The registrant's facial feature information may be acquired from the registrant in advance and registered as a dictionary. Further, the partial space itself may be used as face feature information for identification.

  The face collation unit 54 performs collation processing between the face image of the photographed person and the face image of the registrant registered in the dictionary D. Here, the face collation unit 54 calculates the similarity between the face feature information (partial space) of the photographed person extracted by the face feature extraction unit 53 and the registrant face feature information registered in the dictionary D. . Further, the face collation unit 54 determines whether the subject and the registrant are the same person based on whether the calculated similarity is equal to or greater than a predetermined threshold. However, the face collation unit 54 includes the face of the registrant's dictionary data narrowed down by a dictionary selection unit 57 (to be described later) among the dictionary data registered in the dictionary D and the face feature information (subspace) of the subject. A verification process is performed.

  Various methods can be applied to the face matching process in the face matching unit 54. For example, the mutual subspace method is used for the face matching process in the face matching unit 54. In the mutual subspace method, a partial space (dictionary subspace) as registrant face feature information registered in the dictionary D and a subspace (input subspace) as face feature information of a subject as a recognition target person ) Is calculated.

  The second image acquisition unit 55 is an interface for acquiring an image taken by the second camera 12. In the present embodiment, it is assumed that the second camera 12 captures a still image of the person when the face detection unit 52 detects the face of the person. Accordingly, the second image acquisition unit 55 includes an image input interface (still image acquisition unit) that acquires a still image captured by the second camera 12 when the face detection unit 52 detects a human face. The That is, the second image acquisition unit 55 acquires a still image in which a person (photographed person) whose face is captured by the first camera 11 is captured by the second camera 12. The second image acquisition unit 55 outputs a still image captured by the second camera 12 to the feature detection unit 56.

  The feature detection unit 56 performs processing for detecting feature (feature image feature) information other than human face feature information from the still image acquired by the second image acquisition unit 55. The feature information of the still image is information for classifying the registrant. The feature information of the still image is information that can be detected from the still image, is different from the face feature information extracted by the face feature extraction unit 53, and is always constant information (the same information as at the time of registration). It will be. Further, as the feature information of the still image, a plurality of registrants may be the same information as long as it is equal to or less than a predetermined number of people. That is, the feature information of the still image may be information that can be classified into a group of a predetermined number or less by all registered users.

  For example, the feature information of a still image includes the description content of the name tag (such as “department”, “title” or “location (floor)”) described in the name tag, and the characteristics of the name tag itself (name tag color, shape or Size, etc.), features of the items attached to the name tag (color, shape or length of the strap attached to the name tag), armband, collar emblem, type of hat (thickness of line attached to the hat, line) Color or number of lines), medals, emblems, logos attached to clothes (work clothes), and the like. For example, when the “department” detected from the name tag is used as the feature information of the still image, the feature detection unit 56 detects the image area of the name tag from the still image acquired by the second image acquisition unit 55, and the name tag The process of recognizing “department” from the image is performed.

  The dictionary selection unit 57 performs a process of narrowing down dictionary data based on still image feature information detected by the feature detection unit 56. Accordingly, dictionary data of each registrant is registered in the dictionary D in association with the feature information of the still image. For example, when “department” detected from a name tag is used as still image feature information, the dictionary selection unit 57 performs the above operation based on information indicating “department” detected from the still image by the feature detection unit 56. Dictionary data used for face matching by the face matching unit 54 is selected. As a result, the face matching unit 54 can perform face matching processing on the face feature information of the subject and the face feature information of the registrant narrowed down by the “department”.

Next, the configuration of the dictionary D will be described.
FIG. 5 is a diagram showing a configuration example of the dictionary D. As shown in FIG.
In the example shown in FIG. 5, the dictionary D includes, for each registrant, face video dictionary data (registrant's registered face data) used for face matching and a designated ID number (registrant's identification information). Data is stored. Such dictionary data of each registrant is stored in a state of being classified for each feature of the still image. In the example illustrated in FIG. 5, “memory mapping data”, a plurality of “still image feature data”, and a plurality of “registrant dictionary data” are stored in association with addresses.

  Each “still image feature data” is data for grouping registrants. In the present embodiment, it is assumed that still image feature data can be detected from a still image captured by the second camera 12. For example, when “department” detected from a name tag or armband given to the chest is used as still image feature data, information indicating “department” is stored in each “still image feature data”. In the example shown in FIG. 5, “S01” is stored as the first “still image feature data” at the address “M-002”, and the second “still image of the still image” is stored at the address “M-303”. “S02” is stored as “feature data”.

  Each “registrant's dictionary data” is registered in the dictionary D in association with any “still image feature data”. In other words, in the dictionary D, each “registrant's dictionary data” is registered as a registrant's dictionary data group using the corresponding “still image feature data” as a key. That is, each “registrant's dictionary data” is stored in a block formed for each “still image feature data”. Each block is configured such that the number of “registrant dictionary data” is equal to or less than a predetermined number. The reason for this will be described later in detail.

  For example, in the example shown in FIG. 5, the addresses “M-002” to “M-302” form the first face dictionary block. In the first face dictionary block, each “registrant's dictionary data” at addresses “M-003 to M-302” (dictionary data for a maximum of 300 people) is “feature of still image” at address “M-002”. It is registered in association with “Data”. Similarly, addresses “M-303” to “M-603” form a second face dictionary block. In the second face dictionary block, each “registrant's dictionary data” at addresses “M-304 to M-603” is registered in association with “still image feature data” at address “M-303”. Yes.

  The “memory mapping data” is data obtained by collecting the header addresses of the respective blocks. In the example shown in FIG. 5, “feature data of still image” is stored at the head of each block. That is, the address “M-002” is the start address of the first face dictionary block, and the address “M-303” is the start address of the second face dictionary block. Therefore, in the example illustrated in FIG. 5, the “memory mapping data” stores the address “M-002”, the address “M-303”, and the like as the header address of each block. That is, by reading information from the header address of each block stored in “memory mapping data”, it is possible to read “characteristic data of still image” of each block.

Next, two processing examples will be described as an embodiment assuming the actual operation of the face matching device 1.
First, a first processing example in the face matching device 1 will be described.
In the face matching device 1, it is assumed that the number of registrants to be matched in the face matching process for one person is set from the viewpoint of processing speed and matching accuracy. That is, the face collation apparatus 1 is assumed to guarantee a predetermined collation speed and collation accuracy in the face collation process for a predetermined number or less of registrant dictionary data. For example, if the face matching device 1 is a face matching process for 300 or fewer registrants, it is assumed that the processing speed and the matching accuracy can be used in actual operation. In such a face collation apparatus 1, in an actual face collation process, it is necessary to narrow down to 300 or less registrants (persons to be collated) to be actually collated.

  As described above, the face collation apparatus 1 has the second camera 12 for capturing a still image in addition to the first camera 11 for capturing a face image. In the face collation apparatus 1, the number of persons to be collated is narrowed down to a predetermined number or less by “still image feature data” detected from still images captured by the second camera 12. As a result, the face collation apparatus 1 performs face collation processing for a predetermined number or less of registrants. That is, in the first processing example, face matching processing is performed after narrowing down the number of registrants to be subjected to face matching to a predetermined number or less based on feature data of still images.

FIG. 6 is a flowchart for explaining the processing procedure of the first processing example in the face collation apparatus 1. FIGS. 7A to 7D are diagrams schematically showing various processing states in the processing procedure of the first processing example.
First, it is assumed that a person who wants to receive authentication stands in front of the face matching device 1. In this case, the first camera 11 captures an image including the face of a person in a standing posture as shown in FIG. 7A (step S11). In the collation processing function realized by the control unit 21 of the face collation apparatus 1, the face detection unit 52 performs a process of detecting a human face from the image photographed by the first camera 11 (step S12). . For example, when an image as shown in FIG. 7A is taken by the first camera 11, the face detection unit 52 detects a face region as shown by a dotted line in FIG. 7B.

  When the face of the person is detected by the face detection unit 52 (step S12, YES), the second camera 12 captures the person as a still image as shown in FIG. 7C (step S21). . In the collation processing function realized by the control unit 21 of the face collation apparatus 1, features other than the person's face (still image of the person) are captured by the feature detection unit 56 on the still image captured by the second camera 12. Processing for detecting feature data) is performed (step S22).

  For example, the second camera 12 captures a still image as shown in FIG. In this case, the feature detection unit 56 detects an image region indicating the feature of a still image as indicated by a dotted line in FIG. In the example shown in FIG. 7C, the feature detection unit 56 detects a name tag or an armband attached near the chest as information indicating feature data of a still image. For example, when “department” is used as feature data of a still image, the feature detection unit 56 detects “department” as feature data from an image of a name tag or an armband in a still image taken by the second camera 12.

  When the feature data of the still image is detected by the feature detection unit 56, the dictionary selection unit 57 performs a process of narrowing down registrants to be collated in the face matching process based on the detected feature data of the still image. Performed (steps S23 to S26). That is, the dictionary selection unit 57 collates the “still image feature data” of each block stored in the dictionary D with the feature data detected from the still image by the feature detection unit 56 (step S23). When a block of “still image feature data” that matches the feature data detected by the feature detection unit 56 is found by this matching process (YES in step S24), the dictionary selection unit 57 sets the header address of the block. It is acquired (step S25) and notified to the face collation unit 54 (step S26).

  On the other hand, the first camera 11 shoots an image including a face as a moving image in parallel with the processing of steps S21 to S26 (step S31). For such a moving image (images of continuous frames at a predetermined interval), feature analysis is performed by the face detection unit 52 and the face feature extraction unit 53 (step S32). That is, the moving image captured by the first camera 11 is acquired by the first image acquisition unit 51. As shown in FIG. 7D, the face detection unit 52 detects a face area from each frame image acquired by the first image acquisition unit 51. The face feature extraction unit 53 extracts face feature information from the face area image detected by the face detection unit 52. Thus, the face feature extraction unit 53 extracts face feature information as a moving image based on the plurality of face images detected by the face detection unit 52 from the image of each frame. The face feature information extracted by the face feature extraction unit 53 is supplied to the face matching unit 54 as face feature information (input face feature information) of the person to be authenticated.

  The face matching unit 54 is supplied with the input face feature information from the face feature extracting unit 53 and stores a dictionary area of the registrant to be verified from the dictionary selection unit 57 (that is, a still image). Information indicating the dictionary data narrowed down by the feature data) is supplied. Thereby, the face matching unit 54 performs face matching processing for matching the input face feature information with the dictionary face feature information in each narrowed-down dictionary data (steps S33 to S36).

  That is, the face collation unit 54 first sets the header address supplied from the dictionary selection unit 57 (step S33). This is a process of setting the head position of the face dictionary block to be collated in the dictionary D. In order to read the dictionary data for one person from the dictionary D, the face matching unit 54 sets the address of the next data based on the set address (step S34). For example, the address is updated to “+1” in the dictionary having the configuration shown in FIG.

  Thus, when a new address in the dictionary D is set, the face matching unit 54 reads the dictionary face feature information of one registrant from the set address in the dictionary D. When the dictionary face feature information is read from the dictionary D, the face collation unit 54 collates (face collation) the input face feature information supplied from the face feature extraction unit 53 and the dictionary face feature information (step S35). ). Here, the similarity between input face feature information and dictionary face feature information is calculated as face matching.

  When the face matching is completed, the face matching unit 54 determines whether or not matching has been completed for all dictionary data of the face dictionary block specified by the header address. That is, the face collation unit 54 determines whether or not there is dictionary data of a registrant to be face collated next (step S36). If it is determined that there is dictionary data of the registrant to be face-matched next (NO in step S36), the face-matching unit 54 returns to step S34 and determines the next dictionary data in the dictionary D. Perform face matching.

  If it is determined that there is no registrant's dictionary data to be face-matched (step S36, YES), the face-matching unit 54 outputs a comprehensive face-matching result (step S37). The process ends. This face matching device is assumed to be a system connected to a traffic control device. For this reason, the face collation unit 54 opens the electric lock 3B on the lock control panel 3A of the traffic control device 3 when there is dictionary data in which the similarity obtained by the face collation exceeds a predetermined threshold. If there is no dictionary data that outputs a control signal to lock and the similarity obtained by face matching is equal to or greater than a predetermined threshold, an electric lock is applied to the lock control panel 3A of the traffic control device 3. A control signal for closing is output.

  However, the face matching unit 54 may notify information indicating which registrant matches the person to be authenticated as a result of the face matching process. For example, when there is only one dictionary data in which the similarity obtained by face matching is equal to or greater than a predetermined threshold, the face matching unit 54 indicates that the person to be authenticated is the registrant of the dictionary data (registrant Is output as a processing result. Further, when there is no dictionary data in which the similarity obtained by face matching is equal to or greater than a predetermined threshold, the face matching unit 54 identifies the person to be authenticated as the registrant registered in the dictionary. The fact that it was not done is output as a processing result. Further, when there are a plurality of dictionary data in which the similarity obtained by face matching is equal to or greater than a predetermined threshold, the face matching unit 54 determines that the person to be authenticated is a registrant who has the maximum similarity. Output as a result. When there are a plurality of dictionary data whose similarity obtained by face matching is equal to or higher than a predetermined threshold, the face matching unit 54 determines whether the person to be authenticated has a similarity equal to or higher than a predetermined threshold. A processing result may be output to indicate that it is either.

  A biometric matching device (for example, a face matching device) to which the first processing example as described above is applied registers biometric information of a registrant in a dictionary in association with feature information other than biometric information that can be photographed at the time of authentication. In addition, when biometric information (for example, a face image) of the authentication target person is detected, feature information different from the biometric information (for example, an object worn by the person) is detected from an image obtained by photographing the authentication target person. Feature information) is detected, biometric information associated with feature information that matches the detected feature information is selected from the dictionary, and the selected biometric information and biometric information of the person to be authenticated are biometrically compared.

  That is, according to the biometric matching device to which the first processing example as described above is applied, even when dictionary data of registrants exceeding the limit number is registered, registration of matching targets in one matching process is performed. The number of persons can be limited to a predetermined limit number or less, and as a result, biometric matching with registrants exceeding the limit number set as the processing capability of the apparatus can be performed. In addition, since the number of registered registrants can be limited to a predetermined limit or less, the processing speed of biometric matching and the accuracy of biometric matching can be improved.

Next, a second processing example in the face matching device 1 will be described.
In the second processing example, the face collation for all registrants is not performed, but the number of registrants to be collated is not limited, and whether or not the result is correct is determined based on still image feature data. It is. For example, in the second processing example, the similarity between the input face feature information and the dictionary face feature information of each registrant is calculated as face matching processing. Among these similarities, when there are a plurality of similarities that are equal to or greater than the predetermined threshold, that is, when a plurality of registrants that are likely to be authenticated are detected, the characteristics of the still image detected from the still image of the authentication target It is possible to select a registrant with matching data as a verification result.

  FIG. 8 is a flowchart for explaining the processing procedure of the second processing example in the face collating apparatus 1.

  First, it is assumed that a person who wants to receive authentication stands in front of the face matching device 1. In this case, the first camera 11 captures an image including the face of a person in a standing posture (step S41). In the collation processing function realized by the control unit 21 of the face collation apparatus 1, the face detection unit 52 performs a process of detecting a human face from an image photographed by the first camera 11 (step S42). .

  When the face of the person is detected by the face detection unit 52 (YES in step S42), the second camera 12 photographs the person (the person to be authenticated photographed by the first camera 11) as a still image. (Step S51). In the collation processing function realized by the control unit 21 of the face collation apparatus 1, features other than the person's face (still image of the person) are captured by the feature detection unit 56 on the still image captured by the second camera 12. Processing for detecting feature data) is performed (step S52). When the feature data of the still image is detected by the feature detection unit 56, the dictionary selection unit 57 holds the detected feature data of the still image in the memory such as the RAM 32 as the feature data of the person to be authenticated (step S53). ).

  On the other hand, the first camera 11 shoots an image including a face as a moving image in parallel with the processing of steps S51 to S53 (step S61). For such a moving image (images of continuous frames at a predetermined interval), feature analysis is performed by the face detection unit 52 and the face feature extraction unit 53 (step S62). That is, the moving image captured by the first camera 11 is acquired by the first image acquisition unit 51. The face detection unit 52 detects a face region from each frame image acquired by the first image acquisition unit 51. The face feature extraction unit 53 extracts face feature information from the face area image detected by the face detection unit 52. Thus, the face feature extraction unit 53 extracts face feature information as a moving image based on the plurality of face images detected by the face detection unit 52 from the image of each frame. The face feature information extracted by the face feature extraction unit 53 is supplied to the face matching unit 54 as face feature information (input face feature information) of the person to be authenticated.

  The face matching unit 54 performs face matching processing for matching the input face feature information supplied from the face feature extracting unit 53 with the dictionary face feature information in the dictionary data of all registrants registered in the dictionary D. (Steps S63 to S66). The face collation unit 54 first sets the header address of the dictionary D (step S63). Furthermore, in order to read one dictionary data from the dictionary D, the face collation unit 54 updates the address of the next data with the set address as a reference (step S64). For example, the address is updated to “+1” in the dictionary D having the configuration shown in FIG. Thus, when a new address in the dictionary D is set, the face matching unit 54 reads the dictionary face feature information of one registrant from the set address in the dictionary D. When the dictionary face feature information is read from the dictionary D, the face collation unit 54 collates (face collation) the input face feature information supplied from the face feature extraction unit 53 with the dictionary face feature information (step S65). ).

  Here, the similarity between input face feature information and dictionary face feature information is calculated as face matching. It should be noted that all the similarities obtained by the face matching may be stored in a memory such as the RAM 32 in association with each dictionary data. Here, however, the dictionary data whose similarity is equal to or greater than a predetermined threshold value. Is stored in the RAM 32.

  When the face collation in step S65 is completed, the face collation unit 54 determines whether collation for all dictionary data registered in the dictionary D is completed. That is, the face matching unit 54 determines whether there is dictionary data to be face-matched next (step S66). If it is determined that there is dictionary data of the registrant to be face-matched next (step S66, NO), the face-matching unit 54 returns to step S64 and continues until the dictionary data to be face-checked disappears. Steps S64 to S66 are repeatedly executed.

  If it is determined by the above determination that there is no registrant dictionary data to be face-matched (step S66, YES), the face matching unit 54 performs a comprehensive face-matching result determination process (step S67). In this determination processing, the face collation unit 54 matches the feature data of the still image associated with each dictionary data having a similarity equal to or greater than a predetermined threshold and the feature data obtained from the still image of the person to be authenticated. It is determined whether or not to do. The face matching unit 54 outputs the determination result as a result of the final face matching process (step S66).

  For example, as a result of the determination process, among dictionary data having a similarity equal to or greater than a predetermined threshold, a dictionary associated with feature data of a still image that matches feature data obtained from the still image of the person to be authenticated When the data exists, the face matching unit 54 determines that the person to be authenticated has been identified as the registrant of the dictionary data.

If there is no dictionary data associated with the feature data of the still image that matches the feature data obtained from the still image of the person to be authenticated among the dictionary data whose similarity is equal to or greater than the predetermined threshold, The face collation unit 54 outputs a processing result indicating that the person to be authenticated has not been identified as a registrant registered in the dictionary.
However, when there are a plurality of dictionary data in which the similarity of matching still image feature data is equal to or greater than a predetermined threshold, the face matching unit 54 indicates that the person to be authenticated is the registrant who has the maximum similarity. The determination result may be used, or a determination result may be output indicating that the person to be authenticated is any of the registrants having a similarity equal to or higher than a predetermined threshold.

A biometric matching device (for example, a face matching device) to which the second processing example as described above is applied is characterized by characteristic information (for example, a person wearing it) other than biometric information (for example, a face image) that can be photographed during authentication. The biometric information of the registrant is registered in the dictionary in association with the feature information detected from the object, and the biometric information of the authentication target person and the biometric information of the authentication target person are registered in the biometric matching process. When multiple biometric information of registrants that meet the criteria for matching with biometric information is obtained,
By comparing the feature information associated with the biometric information of the registrant and the feature information detected from the image obtained by photographing the authentication target, biometric verification between the authentication target and the registrant Determine the result.

  That is, according to the biometric matching device to which the second processing example as described above is applied, even if there are a plurality of registrants that satisfy the matching match criteria in the biometric matching, other than biometric information that can be photographed at the time of authentication This feature information makes it possible to determine the selection or selection of a registrant that matches the person to be authenticated. Further, even if the determination on the biometric matching result is not performed based on the feature information other than the biometric information, the information is added to the result of the biometric matching process by adding the feature information other than the biometric information to the person to be authenticated. Can be used as the number of evidences to identify

Next, the registration process in the face collation apparatus 1 will be described.
The face matching device 1 can capture a face image as a moving image and a person as a still image. With such a configuration, it is possible not only to narrow down the dictionary data as described above, but also to ensure security in the traffic control system even in the absence of an administrator. For example, when registering dictionary data, the registration operation can be performed in a state where sufficient security is ensured even if the administrator is absent.

  That is, in the face collation apparatus 1, the display unit 13, the voice unit 15, the operation unit 14, and the like can provide the registrant with voice guidance or screen guidance for an operation procedure for dictionary registration. is there. That is, in the face collation apparatus 1, it is technically possible to register face feature information as dictionary data by an operation of a registrant himself even when an administrator is absent. However, when the registration is performed in the absence of the administrator, for example, the following cases are assumed in addition to those that are normally registered.

(1) The registered number is wrong ... Operation mistake (2) The person who planned to register has not come ... Plan failure (3) Narismashi ... Unauthorized When these situations occur, there is no administrator Even so, the face matching device 1 can ensure security.

FIGS. 9A to 9D are diagrams schematically illustrating a plurality of registrant states. FIG. 10 shows a recording medium S showing the results of registration processing by a plurality of registrants as shown in FIGS.
For example, it is assumed that four persons as shown in FIGS. 9A to 9D perform a registration operation. In this case, the face collation apparatus 1 uses the first image acquisition unit 51, the face detection unit 52, and the face feature extraction unit 53 to obtain dictionary face feature information from a moving image of a person's face photographed by the first camera 11. Generate. At this time, the face collation apparatus 1 captures a still image of the person being captured by the first camera 11 using the second camera 12. In the face collation apparatus 1, feature information, that is, feature data of a still image is detected from the still image captured by the second camera 12 by the second image acquisition unit 55 and the feature detection unit 56.

  In the example shown in FIGS. 9A to 9D, the employee number is detected from the name tag or armband attached to each person's chest as the feature data of the still image. In the example shown in FIG. 9A, the employee number “001047” is detected as the still image feature data. In the example shown in FIG. 9B, the employee number “001048” is detected as the still image feature data. In the example shown in FIG. 9C, the employee number “001049” is detected as the feature data of the still image. In the example shown in FIG. 9D, the employee number “001050” is used as the feature data of the still image. Has been detected.

  The still image feature data is registered in the HDD 22 as the dictionary D in association with dictionary data for each registrant including face feature information created from a moving image. For the examples shown in FIGS. 9A to 9D, for example, as the grouping (still image feature data) in the dictionary D, the upper three digits of the employee number (“001 *”) are used. It is possible to use “001” in “**”) or a four-digit number (“0010” in “0010 **”). In this case, a face dictionary block having the upper three-digit number (“001”) or the four-digit number (“0010”) in the employee number is formed in the dictionary D.

  In the registration process as described above, the face collation apparatus 1 stores still image feature data together with a registered person's face image as information indicating registration contents. Further, as shown in FIG. 10, the face collation apparatus 1 records the registered person's face image and still image feature data on a recording medium using a printer (not shown) or the like. In the example shown in FIG. 10, as the list information of the persons registered during a specific period, the registered face images, still image feature data (number shooting), and the numbers actually registered in the dictionary D are associated with each other. Is printed on the recording medium S. The registered person's face image may be, for example, a face image detected from a still image, or may be a face image detected from a frame image constituting a moving image. Further, the information as shown in FIG. 10 may be displayed on the display unit 13 or a display of a PC connected to the network 2. Further, the registration data that the administrator determines to be illegal registration by the confirmation by the recording medium S or the display content as described above is deleted from the dictionary D.

  According to the above-described form, the administrator can easily confirm the registered content by using the recording medium S on which the list of the feature data of the face image and the still image is recorded after the registration process is completed. That is, in the face collation apparatus, registration processing can be executed even when the administrator is absent, and the administrator can easily confirm the contents of such registration processing every predetermined period. As a result, it is possible to properly manage the contents of registration processing performed in the absence of an administrator, and to secure security. For example, even if an attempt is made to register with an illegally shot image, considering that facial feature information obtained from moving images, still image feature data, and facial images remain as clear evidence, respectively, Strong deterrence.

  In the registration processing in the face collation apparatus 1 described above, the description has been made assuming that the feature data of the still image is an employee card, a name tag, an armband, or the like, but is not limited thereto. For example, as feature data of still images, when taking documents such as registration applications with photos or handwritten number slips handed over in advance. Also, a method in which the boss stands side by side with the registrant and confirms the boss's face can be realized as an operation mode.

FIG. 1A is a front view showing an external configuration example of the face matching device. FIG. 1B is a side view showing an example of the external configuration of the face matching device. FIG. 2 is a diagram schematically illustrating a configuration example of a face matching system (traffic control system) including a face matching device. FIG. 3 is a block diagram schematically illustrating a configuration example of a control system (control unit) of the face matching device. FIG. 4 is a block diagram illustrating a configuration example of the face matching processing function in the face matching device. FIG. 5 is a diagram illustrating a configuration example of a dictionary. FIG. 6 is a flowchart for explaining the processing procedure of the first processing example in the face collation apparatus. FIGS. 7A to 7D are diagrams schematically illustrating various processing states in the processing procedure of the first processing example. FIG. 8 is a flowchart for explaining the processing procedure of the second processing example in the face collation apparatus. FIGS. 9A to 9D are diagrams schematically illustrating a plurality of registrant states. FIG. 10 is a recording medium showing the result of registration processing by a plurality of registrants.

Explanation of symbols

  DESCRIPTION OF SYMBOLS 1 ... Face collation apparatus, 3 ... Traffic control apparatus, 11 ... 1st camera, 12 ... 2nd camera, 13 ... Display part, 14 ... Operation part, 15 ... Audio | voice part, 16 ... Interface part, 21 ... Control unit, 22 DESCRIPTION OF SYMBOLS ... Hard disk drive (HDD), D ... Dictionary, 23 ... Power supply part, 30 ... CPU, 31 ... ROM, 32 ... RAM, 33 ... Non-volatile memory, 34 ... 1st image input part, 35 ... 2nd image input part, 36 ... Display control unit, 37 ... Audio control unit, 38 ... Key input control unit, 39 ... Communication control unit, 40 ... Output control unit, 51 ... First image acquisition unit, 52 ... Face detection unit, 53 ... Face feature extraction 54, a face collation unit, 55, an image acquisition unit, 56, a feature detection unit, and 57, a dictionary selection unit.

Claims (7)

First acquisition means for acquiring information including biological information;
Second acquisition means for acquiring information including feature information other than the biological information for grouping persons;
Storage means for storing biometric information of a plurality of registrants in association with feature information other than the biometric information included in the information acquired by the second acquisition means;
Biometric information detection means for detecting biometric information of the person to be authenticated from the information acquired by the first acquisition means;
A feature detection unit for detecting feature information other than the biometric information of the authentication target person from the information acquired by the second acquisition unit when the biometric information of the authentication target person is detected by the biometric information detection unit;
A selection means for narrowing down a registrant stored in the storage means based on feature information other than the biological information detected by the feature detection means;
Biometric collating means for collating biometric information detected by the biometric information detecting means and biometric information of a registrant stored in the storage means selected by the selecting means;
A biometric collation apparatus characterized by comprising: First acquisition means for acquiring information including biological information;
Second acquisition means for acquiring information including feature information other than the biological information for grouping persons;
Storage means for storing biometric information of a plurality of registrants in association with feature information other than the biometric information included in the information acquired by the second acquisition means;
Biometric information detection means for detecting biometric information of the person to be authenticated from the information acquired by the first acquisition means;
Biometric matching means for matching the biometric information of the person to be authenticated detected by the biometric information detecting means with the biometric information of the registrant stored in the storage means;
When there are a plurality of biometric information of the registrant who is determined to be the same person as the biometric information of the person to be authenticated by the biometric matching means, the biometric information of the person to be authenticated is obtained from the information acquired by the second acquisition means. Feature detection means for detecting feature information other than
By comparing the feature information other than the biometric information of the person to be authenticated detected by the feature detection means with the feature information other than the biometric information of the registrant determined to be the same person as the person to be authenticated. A determination means for determining a result of the matching process;
A biometric collation apparatus characterized by comprising: First acquisition means for acquiring an image including face image information;
Second acquisition means for acquiring an image including feature information other than the face image information for grouping persons;
Storage means for storing face image information of a plurality of registrants in association with feature information other than the face image information included in an image acquired by the second acquisition means;
Face detection means for detecting face image information of the person to be authenticated from the image acquired by the first acquisition means;
A feature detection unit for detecting feature information other than the authentication target person's face image information from the image acquired by the second acquisition unit when the face detection unit detects the authentication target person's face image information;
A selection means for narrowing down a registrant stored in the storage means based on feature information other than the face image information detected by the feature detection means;
Face collation means for collating face image information detected by the face detection means with face image information of a registrant stored in the storage means selected by the selection means;
A face matching device characterized by comprising: First acquisition means for acquiring an image including face image information;
Second acquisition means for acquiring an image including feature information other than the face image information for grouping persons;
Storage means for storing face image information of a plurality of registrants in association with feature information other than the face image information included in an image acquired by the second acquisition means;
Face detection means for detecting face image information of the person to be authenticated from the image acquired by the first acquisition means;
Face collation means for collating the face image information of the person to be authenticated detected by the face detection means with the face image information of the registrant stored in the storage means;
When there are a plurality of face image information of a registrant who is determined to be the same person as the face image information of the person to be authenticated by the face matching means, the image of the person to be authenticated is obtained from the image obtained by the second obtaining means. Feature detection means for detecting feature information other than face image information;
Comparing feature information other than the face image information of the person to be authenticated detected by the feature detection means with feature information other than the face image information of the registrant determined to be the same person as the person to be authenticated. Determining means for determining the result of the face matching process by:
A face matching device characterized by comprising: The first acquisition means acquires a moving image including a face image,
The face detection means detects the face image of the person to be authenticated from each frame image of the moving image acquired by the first acquisition means,
Further, by integrating the features of the face image detected from each frame of the moving image acquired by the face acquisition means by the face detection means, face feature information as face image information used for face matching by the face matching means Having facial feature extraction means for extracting
The face collation apparatus according to claim 3 or 4, characterized in that The biometric information of a plurality of registrants is stored in storage means in association with feature information other than the biometric information for grouping each registrant,
Obtaining first information including biological information;
The biological information is detected from the acquired first information,
When the biometric information of the person to be authenticated is detected by this detection, second information including feature information other than the biometric information of the person to be authenticated is acquired,
The feature information other than the biometric information of the person to be authenticated is detected from the acquired second information,
Narrow down registrants stored in the storage means based on feature information other than biological information detected by this detection,
Collating the biometric information of the person to be authenticated with the biometric information of the registrant stored in the narrowed-down storage means;
A biometric matching method characterized by comprising: The biometric information of a plurality of registrants is stored in storage means in association with feature information other than the biometric information for grouping each registrant,
Obtaining first information including biological information;
The biometric information of the person to be authenticated is detected from the acquired first information,
The biometric information of the detected person to be authenticated is collated with the biometric information of the registrant stored in the storage means,
When there are a plurality of biometric information of the registrant who is determined to be the same person as the biometric information of the person to be authenticated by this collation, second information including feature information other than the biometric information of the person to be authenticated is acquired. And
From the acquired second information, feature information other than the biometric information of the person to be authenticated is detected,
The result of the biometric matching process is performed by comparing the detected feature information other than the biometric information of the authentication target person with the feature information other than the biometric information of the registrant determined to be the same person as the authentication target person. Determine
A biometric matching method characterized by the above.

Images