JP2024090999A - Information processing device, information processing method, and computer program - Google Patents

Abstract

[Problem] To provide an information processing device, information processing method, and computer program for stably correcting a score based on one piece of biometric information and another piece of biometric information even when there is a bias or difference in the attributes of the biometric information. [Solution] In a biometric authentication system having a biometric authentication device, an input device, a video output device, a biometric authentication device, and a biometric information sensor and an entrance/exit management device communicably connected to the biometric authentication device via a network, the biometric authentication device includes a calculation unit 206 that obtains a correction coefficient based on at least the feature amount of the first biometric information, the feature amount of the second biometric information, and the attribute of the first biometric information, a correction unit 207 that obtains a score based on the feature amount of the first biometric information and the feature amount of the second biometric information and corrects the obtained score using the correction coefficient, and an authentication unit 208 that performs authentication based on the corrected corrected score. [Selected Figure] Figure 2

Description

The present invention relates to authentication technology.

In one-to-one biometric authentication, two pieces of biometric information are compared, a matching score is calculated based on the degree of similarity, and it is determined whether the people corresponding to the two pieces of biometric information are the same person. In one-to-N biometric authentication, one piece of given biometric information is compared with the biometric information of N people registered in advance, and it is determined whether the person corresponding to the one piece of biometric information matches any of the N people registered, or whether it matches none of the people registered. To perform biometric authentication with high accuracy, it is required to output a high matching score for the feature pair of the person himself/herself and a low matching score for the feature pair of other people, and further, that these scores fall within a certain range and have little variation. However, in general, in biometric authentication such as face recognition, the matching score is easily affected by the situation when the biometric information is acquired (lighting conditions, face direction, etc.).

In response to this problem, a method has been disclosed for improving accuracy by correcting the matching score. For example, in Patent Document 1 and Non-Patent Document 1, in addition to the registered person, the features of multiple unregistered people are stored as reference features. During matching, matching is also performed with this group of reference features, and the matching score with the registered person is corrected using the distribution and density of these matching scores. This is said to reduce variation in the matching score and prevent rejection of the person in question and acceptance of a false person.

Patent No. 06962458

Liu, Jiaheng, et al. "Inter-class discrepancy alignment for face recognition." arXiv preprint arXiv:2103.01559 (2021).

However, prior art techniques do not always work well in cases where the attributes of the matching person are significantly different from those of the reference person, or where registered people are biased toward certain attributes in one-to-N authentication. This invention provides technology for stably correcting scores based on one piece of biometric information and another piece of biometric information, even when there is a bias or difference in the attributes of the biometric information.

One aspect of the present invention is characterized by comprising a first acquisition means for acquiring first biometric information and second biometric information, a second acquisition means for acquiring a correction coefficient based at least on a feature of the first biometric information, a feature of the second biometric information, and an attribute of the first biometric information, a correction means for calculating a score based on the feature of the first biometric information and the feature of the second biometric information and correcting the calculated score using the correction coefficient, and an authentication means for performing authentication based on the corrected score corrected by the correction means.

The configuration of the present invention makes it possible to stably correct the score based on one piece of biometric information and the other piece of biometric information, even when there is a bias or difference in the attributes of the biometric information.

FIG. 1 is a block diagram showing an example of the configuration of a biometric authentication system. FIG. 2 is a block diagram showing an example of the functional configuration of a biometric authentication device 101. 4 is a flowchart of the operation of the biometric authentication device 101. 4A and 4B are schematic diagrams illustrating the processes of (Equation 2) and (Equation 3), respectively. FIG. 2 is a block diagram showing an example of the functional configuration of a biometric authentication device 101. 11 is a flowchart of a process performed by the biometric authentication device 101 to generate a registration correction coefficient _CE and register it in the storage unit 501. 10 is a flowchart of a process performed by the biometric authentication device 101 to determine whether a person whose biometric information has been obtained by the biometric sensor 105 corresponds to any one of N persons permitted to pass through a gate. 11 is a flowchart of a process performed by the biometric authentication device 101 to generate a registration correction coefficient _CE and register it in the storage unit 501.

The following embodiments are described in detail with reference to the attached drawings. Note that the following embodiments do not limit the invention according to the claims. Although the embodiments describe multiple features, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined in any manner. Furthermore, in the attached drawings, the same reference numbers are used for the same or similar configurations, and duplicate explanations are omitted.

[First embodiment]
In this embodiment, a biometric authentication system is described that determines whether a target person who is about to pass through a gate is a person who is permitted to pass through the gate, and opens the gate if the target person is a person who is permitted to pass through the gate. In the biometric authentication system, the target person who is about to pass through the gate first holds an ID card on which his/her information is recorded over an ID card reader and faces the camera. The ID card reader reads the information recorded on the ID card, and the camera captures the target person. Then, the biometric authentication system determines whether the target person is a person who is permitted to pass through the gate based on a face image that is registered in advance in association with the information read from the ID card by the ID card reader and an image of the target person captured by the camera. Then, in the biometric authentication system, if the target person is a person who is permitted to pass through the gate, the gate is opened (opened), and if the target person is not a person who is permitted to pass through the gate, the gate is not opened (closed).

First, a configuration example of a biometric authentication system according to this embodiment will be described with reference to the block diagram of FIG. 1. As shown in FIG. 1, the biometric authentication system according to this embodiment has a biometric authentication device 101, an input device 102, a video output device 103, a biometric information sensor 105 that is communicably connected to the biometric authentication device 101 via a network 104, and an entrance/exit management device 106.

First, the biometric authentication device 101 will be described. The biometric authentication device 101 is a computer device such as a PC (personal computer), a smartphone, or a tablet terminal device.

The CPU (Central Processing Unit) 101a executes various processes using computer programs and data stored in the RAM 101c. As a result, the CPU 101a controls the operation of the entire biometric authentication device 101, and executes or controls various processes described as processes performed by the biometric authentication device 101.

ROM (Read Only Memory) 101b stores setting data for the biometric authentication device 101, computer programs and data related to starting up the biometric authentication device 101, computer programs and data related to the basic operations of the biometric authentication device 101, etc.

RAM (Random Access Memory) 101c has an area for storing computer programs and data loaded from ROM 101b or external storage device 101d. RAM 101c also has an area for storing biometric information acquired from biometric sensor 105 via communication I/F 101g. RAM 101c also has a work area used by CPU 101a when executing various processes. In this way, RAM 101c can provide various areas as needed.

The external storage device 101d is a large-capacity information storage device such as a hard disk drive. The external storage device 101d stores an operating system (OS), computer programs and data for causing the CPU 101a to execute or control various processes described as processes performed by the biometric authentication device 101, and the like.

Computer programs and data stored in the external storage device 101d are loaded into the RAM 101c as appropriate under the control of the CPU 101a, and are processed by the CPU 101a.

The external storage device 101d may include optical disks such as flexible disks (FD) and compact disks (CD) that are removable from the biometric authentication device 101, magnetic cards, optical cards, IC cards, memory cards, etc.

The input I/F 101e is connected to an input device 102. The output I/F 101f is connected to a video output device 103. The communication I/F 101g is an interface for data communication with external devices (the entrance/exit management device 106 and the biometric information sensor 105) via the network 104.

The CPU 101a, ROM 101b, RAM 101c, external storage device 101d, input I/F 101e, output I/F 101f, and communication I/F 101g are all connected to the system bus 101h.

Next, the input device 102 will be described. The input device 102 is a user interface such as a keyboard, mouse, or touch panel, and can be operated by a user to input various instructions and information to the CPU 101a.

Next, the video output device 103 will be described. The video output device 103 has an LCD screen or a touch panel screen, and displays the results of processing by the CPU 101a as images, text, etc. Note that the video output device 103 may also be a projection device such as a projector that projects images and text.

Next, the biometric sensor 105 will be described. The biometric sensor 105 is an imaging device such as a network camera that is set up to capture an image of the vicinity of the gate, and outputs the captured image obtained by capturing an image of the vicinity of the gate as biometric information. The biometric sensor 105 may be an imaging device that captures a moving image of the vicinity of the gate and outputs an image of each frame in the moving image as a captured image. The biometric sensor 105 may also be an imaging device that captures a still image of the vicinity of the gate periodically or irregularly, and outputs the still image as a captured image.

Next, the entrance/exit management device 106 will be described. The entrance/exit management device 106 is a device that includes the gate described above, a control device that controls the opening and closing of the gate, and the ID card reading device described above.

The block diagram of FIG. 2 shows an example of the functional configuration of the biometric authentication device 101. In this embodiment, a case will be described in which each functional unit (excluding the storage unit 204) shown in FIG. 2 is implemented by a computer program. In the following, the functional units (excluding the storage unit 204) shown in FIG. 2 may be described as the subject of processing, but in reality, the functions of the functional units are realized by the CPU 101a executing a computer program corresponding to the functional units. However, one or more of the functional units shown in FIG. 2 may be implemented by hardware.

The operation of the biometric authentication device 101 will be described with reference to the flowchart in FIG. 3. A person who is about to pass through the gate holds an ID card on which his or her information is recorded over an ID card reader and faces the biometric information sensor 105. At this time, the ID card reader reads the information recorded on the ID card (information related to the person) and transmits the read information to the biometric authentication device 101 via the network 104. The biometric information sensor 105 also transmits a captured image of the person to the biometric authentication device 101 via the network 104.

In step S301, the acquisition unit 201 receives information transmitted from the ID card reader and acquires a facial image of the target person registered in advance in the external storage device 101d in association with the information as biometric information i _A. On the other hand, the acquisition unit 201 acquires a captured image of the target person transmitted from the biometric information sensor 105 as biometric information i _B.

The acquisition unit 201 performs normalization processing on the biometric information _iB . In this embodiment, normalization processing is performed by performing face organ point detection on the captured image that is the biometric information _iB , cutting out a face area from the captured image based on information obtained by the organ point detection, and re-setting the image in the cut-out face area as the biometric information _iB . The normalization processing for extracting an image of the face area from the biometric information _iB and re-setting the image as the biometric information _iB is not limited to the above processing, and may be, for example, a processing for performing face frame detection on the captured image as the biometric information _iB , cutting out a face area based on information obtained by the face frame detection, and re-setting the image in the cut-out face area as the biometric information _iB . Furthermore, when the captured image of a person acquired from the biometric information sensor 105 is an image in which the face is large and facing forward, the normalization processing is not essential. Since the biometric information _iA is assumed to be an image in which a person's face is shown facing forward, normalization processing does not need to be performed on the biometric information _iA , but normalization processing may be performed on the biometric information _iA depending on the situation.

In step S302, the extraction unit 202 extracts a feature f _A from the biometric information i _A and extracts a feature f _B from the biometric information i _B (f _A , f _B ∈ ^{R D} ). In this embodiment, the extraction unit 202 uses a neural network as a machine learning model for extracting features (face features in this embodiment) from an image, and extracts features from an image that is biometric information. However, various methods can be applied to extract features from an image that is biometric information, and are not limited to a specific method. For example, a linear dimensionality reduction method such as principal component analysis (PCA) or linear discriminant analysis (LDA), a nonlinear dimensionality reduction method such as kernel PCA, or a clustering method such as k-means may be applied, and any machine learning method that can project any multidimensional data onto (a certain point on) another multidimensional space may be used. In this embodiment, the same machine learning model is used to acquire the features f _A and f _B , and each of these is a D-dimensional vector. However, the features _fA and _fB may be vectors of the same dimension or different dimensions obtained using different machine learning models. When the features _fA and _fB have different dimensions, it is necessary to select a method corresponding to them for calculating the first reference score _s1 described later. In addition, the machine learning model used by the extraction unit 202 is assumed to have been trained in advance so that the person himself/herself is extracted as a close feature and others are extracted as a distant feature in the feature space.

In step S303, the calculation unit 203 calculates a first matching score _s1 by calculating the following (Equation 1) using the feature amount _fA and the feature amount _fB extracted by the extraction unit 202. In this embodiment, _s1 ∈R is represented as a scalar.

Here, <x, y> represents the inner product of x and y, and ∥x∥ represents the length of x (L2 norm). As shown in (Equation 1), in this embodiment, the feature values f _A and f _B are normalized and the inner product is calculated, that is, the cosine similarity is calculated as the first matching score s _1. However, the method of calculating the first reference score s ₁ is not limited to the method shown in (Equation 1), and may be a distance-based index such as Euclidean distance or Manhattan distance, or a similarity-based index such as Pearson's correlation coefficient, and may be any measure that can express the similarity between two points in a multidimensional space. In addition, when feature values f _A and f _B of different dimensions are obtained in step S302, Jaccard coefficient, Earth Mover's Distance, etc. may be used as a measure of similarity. This embodiment is not limited to these.

Here, the storage unit 204 stores a feature set corresponding to each attribute of biometric information that may be input to the biometric authentication device 101 as a reference feature set. The storage unit 204 may be implemented using the RAM 101c or the external storage device 101d. Here, the "attributes of biometric information" refers to a tendency of bias regarding information including the race, sex, age, face direction, facial expression, lighting conditions, and presence or absence of attachments (masks, sunglasses, etc.) of the face (person) captured in an image including the face of a person. In this embodiment, the biometric information i _A has an attribute that it tends to be captured with a front-facing, expressionless face, no attachments, and ideal lighting conditions without highlight blowout, black blowout, backlight, etc. On the other hand, the biometric information i _B has an attribute that it tends to be captured with a front-facing, expressionless face, but may be wearing a mask, and the lighting conditions are not as ideal as those of the face image associated with the information recorded on the ID card.

Each reference feature group held by the holding unit 204 is generated by the following procedure. First, face images that match a target attribute, which is one of the attributes of biometric information that can be input to the biometric authentication device 101, are collected. For example, if a face image such as biometric information _iA is input, similar imaging conditions are reproduced and the faces of multiple people are captured under the imaging conditions to collect the images. Alternatively, face images with similar tendencies are extracted from face images collected on the Internet to collect the images. Next, features are extracted from each of the collected face images using the same method as the extraction unit 202, and the extracted feature group is set as a reference feature group corresponding to the target attribute. The reference feature group held by the holding unit 204 may be generated by the biometric authentication device 101 or may be generated by a device other than the biometric authentication device 101. The holding unit 204 may be included in the biometric authentication device 101, or may be an external device capable of communicating with the biometric authentication device 101. In this embodiment, the storage unit 204 stores at least a reference feature set having the same attribute as the attribute of the biometric information _iA and a reference feature set having the same attribute as the attribute of the biometric information _iB .

In step S304, the selection unit 205 acquires (selects) a reference feature group Φ _A corresponding to the attribute of biometric information i _A (corresponding to feature f _A ) from among the reference feature groups stored for each attribute in the storage unit 204. In addition, the selection unit 205 acquires (selects) a reference feature group Φ B corresponding to the attribute of biometric information i _B (corresponding to feature f _B ) from among the reference feature groups stored for _{each attribute} in the storage unit 204.

In step S305, the calculation unit 206 calculates the following (Formula 2) using the feature amount f _A and the reference feature amount group Φ _B to obtain a correction coefficient c _A. The calculation unit 206 also calculates the following (Formula 3) using the feature amount f _B and the reference feature amount group Φ _A to obtain a correction coefficient c _B.

The correction coefficients c _A and c _B are values that indicate the degree of separation from each comparison target, and are obtained from the sparse-dense relationship in the feature space. These values have the property that they are small for attributes that have been well learned in the machine learning model used by the extraction unit 202, and conversely, they are large for attributes that have not been well learned. This property is used to suppress the variation in the first matching score s ₁ during score correction, which will be described later. In this embodiment, the feature f _A and the reference feature group Φ _B are used to calculate the correction coefficient c _A , and the feature f _B and the reference feature group Φ _A are used to calculate the correction coefficient c _B. This is because it is possible to grasp how the feature of the same attribute as the comparison target is distributed in the feature space.

Here, k _A is a parameter indicating the number of reference features used in the calculation of c _A , φ _Bj ∈ Φ _B is k _A features in Φ _B that are closest to f _A , k _B is a parameter indicating the number of reference features used in the calculation of c _B , and φ _Aj ∈ Φ _A is k _B features in Φ _A that are closest to f _B. Here, Figs. 4(a) and (b) are diagrams that respectively show the process of (Formula 2) and (Formula 3). In Fig. 4, the mesh sphere represents the feature space, the circular points represent f _A , the triangular points represent f _B , the diamond-shaped points represent Φ _B , and the pentagonal points represent Φ _A. In Figs. 4(a) and 4(b), the reference features connected by the arrows represent φ _Bj ∈ Φ _B when k _A = 3, and φ _Aj ∈ Φ _A when k _B = 3, respectively. As shown in (Formula 2) and (Formula 3), c _A and c _B are the sum of the cosine similarities between f _A and φ _Bj ∈ Φ _B and between f _B and φ _Aj ∈ Φ _A , which are exponentially non-negated. Therefore, if the length of the arrow in FIG. 4 is assumed to be the distance between two points in the feature space, the larger/smaller the sum of the lengths of the arrows is, that is, the less similar/more similar f _A /f _B is to the feature of the same attribute as the comparison target, the smaller/larger c _{A and c B are. Note that c A and c B are calculated} using only nearby features here because if features other than the nearby ones are used, the degree of similarity will not differ regardless of which reference feature group is used. In addition, the correction coefficients are calculated for both c _A and c _B because the ability to deal with outliers and the like is weakened when only one of them is used. Although c _A and c _B are calculated by the above procedure, this is merely an example, and c _A and c _B may be any values that satisfy the above-mentioned properties. The present embodiment is not limited to this.

In step S306, the correction unit 207 corrects the first matching score _s1 calculated by the calculation unit 203 in step S303 using the correction coefficients _cA and _cB calculated by the calculation unit 206 in step S305 in accordance with the following (Equation 4), to calculate a second matching score _s2 as a corrected score. In this embodiment, _s2 ∈R is represented as a scalar.

Here, α and β are parameters for filling the scale difference between c _A and c _B , respectively. In this embodiment, it is assumed that there is no scale difference, and each is set to 1. This scale difference occurs due to the difference in the number of features constituting Φ _A and Φ _B used when calculating c _A and c _B , or the difference in parameters k _A and k _B. For example, if the number of features constituting Φ _A and Φ _B is the same, k _A is 5, and k _B is 10, α can be set to 1 and β can be set to 2. The method of setting α and β is not limited to this, and for example, they may be determined by a function that inputs at least one of parameters k _A and k _B. In addition, in (Equation 4), the exponent of s ₁ is taken and made non-negative, and then correction is performed, but this is merely an example, and any method that can make s ₁ non-negative may be used. For example, a method such as the following (Equation 5) may be used, in which correction is performed after subtracting -1, _which is the lower limit value of s 1, which is the cosine similarity, i.e., adding 1.

In step S307, the authentication unit 208 performs authentication of the target person based on the second matching score _s2 obtained by the correction unit 207 in step S306. If the second matching score _s2 is equal to or greater than the threshold, the authentication unit 208 outputs "authentication of the target person is successful" as the authentication result, and if the second matching score _s2 is less than the threshold, the authentication unit 208 outputs "authentication of the target person is unsuccessful" as the authentication result. Note that the authentication process of the target person based on the second matching score _s2 is not limited to the above process. For example, the authentication result may be the output of an authentication judgment function that inputs the second matching score _s2 . The authentication unit 208 transmits the authentication result to the entrance/exit management device 106 via the communication I/F 101g. If the authentication result received from the authentication unit 208 is "authentication of the target person is successful", the control device in the entrance/exit management device 106 controls the gate to an open state, and if the authentication result received from the authentication unit 208 is "authentication of the target person is unsuccessful", the control device controls the gate to a closed state.

In this way, according to this embodiment, the matching score is corrected taking into account the bias of the attributes of the two pieces of biometric information. Specifically, two types of reference feature groups, Φ _A and Φ _B , are prepared, and the correction coefficient is calculated by mutually referring to each of them. This makes it possible to suppress the variation in the matching score more than the conventional method and improve the authentication accuracy.

Second Embodiment
In the following embodiments including this embodiment, differences from the first embodiment will be described, and unless otherwise specified below, it is assumed that the embodiments are the same as the first embodiment. In the biometric authentication system according to this embodiment, face images of N (N is an integer equal to or greater than 1) people who are permitted to pass through the gate are registered in advance. Then, the target person who is about to pass through the gate turns his/her face toward the camera. Then, in the biometric authentication system, based on the image of the target person captured by the camera and the face images of the N people who are permitted to pass through the gate, it is determined which person who is permitted to pass through the gate is the target person. Then, if the target person is any of the people permitted to pass through the gate, information related to the target person is output and the gate is opened, and if the target person is not any of the people permitted to pass through the gate, information related to the target person is not output and the gate is not opened.

The block diagram of FIG. 5 shows an example of the functional configuration of the biometric authentication device 101 according to this embodiment. In FIG. 5, the functional units having the same functions as those shown in FIG. 2 are given the same reference numerals, and the description of these functional units will be omitted or simplified.

In the biometric authentication system according to this embodiment, the registration correction coefficient C _E is generated in advance and registered in the storage unit 501. The storage unit 501 may be implemented using the RAM 101c or the external storage device 101d. The process performed by the biometric authentication device 101 to generate the registration correction coefficient C _E and register it in the storage unit 501 will be described with reference to the flowchart in Fig. 6. The process according to the flowchart in Fig. 6 is started, for example, in response to an administrator of the biometric authentication system operating the input device 102 to input a registration instruction.

The processing of steps S601 to S605 (registration processing) is repeated N times (the number of people permitted to pass through the gate). The following describes the lth (l=1 to N)th registration processing.

In step S602, the acquisition unit 201 acquires registered biometric information iEl from a "registered biometric information group I _E = {i _E1 , i _E2 , ..., i _EN } including the registered biometric information (face image) of each of N persons permitted to pass through the gate" that is preregistered in _the external storage device 101d. The registered biometric information _iEl is the registered biometric information (face image) of the l-th person among the N persons permitted to pass through the gate. Here, the attributes of the registered biometric information i _E1 , i _E2 , ..., i _EN are all assumed to be the same as the attributes of the biometric information _iA .

In step S603, the extraction unit 202 extracts the registered feature f _EL from the registered biometric information i _El in the same manner as in the first embodiment. In step S604, the extraction unit 202 registers the registered feature f _EL extracted in step S603 in the storage unit 204. By performing the processes in steps S601 to S605 for l=1 to N, a group of registered features F _E ={f _E1 , f _E2 , ..., f _EN } is registered in the storage unit 204.

In step S606, the acquisition unit 201 acquires biometric information (face image) from the biometric information sensor 105 as matching biometric information _iS , in the same manner as in the first embodiment. Here, the attributes of the matching biometric information _iS are assumed to be the same as the attributes of the biometric information _iB .
Then, the extraction unit 202 extracts a matching feature _fS from the matching biometric information _iS in the same manner as in the first embodiment. Then, the selection unit 205 acquires (selects) a reference feature group _ΦS corresponding to an attribute of the matching feature _fS from the storage unit 204 in the same manner as in the first embodiment.

In this embodiment, since the attributes of the biometric information input during matching are known in advance, a reference feature set having the same attributes is acquired as Φ _S. However, this is merely an example, and if the biometric information input during matching is unknown at this point, a method may be adopted in which a plurality of registered correction coefficient sets C _E , which will be described later, are calculated from a plurality of reference feature sets Φ _S having different attributes, and the C _E used during score correction is switched in step S705, which will be described later.

In step S607, the calculation unit 206 performs a calculation according to the following (Equation 6) using the registered feature set F _E acquired by the selection unit 205 from the storage unit 204 and the reference feature set Φ _S acquired (selected) in step S606, to acquire a registered correction coefficient set C _E = {c _E1 , c _E2 , ..., c _EN }.

Here, k _E is a parameter that determines the number of reference features used to calculate c _El , and φ _Slj ∈Φ _S represents the k _E features in Φ _S that are closest to f _El .

In step S608, the calculation unit 206 registers in the holding unit 501 the set of registered correction coefficients C _E ={c _E1 , c _E2 , . . . , c _EN } calculated in step S607.

Next, the process performed by the biometric authentication device 101 to determine whether the person whose biometric information has been acquired by the biometric information sensor 105 corresponds to any one of the N people permitted to pass through the gate will be described with reference to the flowchart in FIG. 7.

In step S701, the acquisition unit 201 acquires matching biometric information _iS of a target person output from the biometric information sensor 105. In step S702, the extraction unit 202 extracts a matching feature _fS from the matching biometric information _iS in the same manner as in the first embodiment.

In step S703, the calculation unit 203 calculates a first matching score group S1 = { _{s11, s12, ..., s1N} by performing a calculation according to the following (Equation 7) using the registered feature group F1E held by the holding} unit 204 and the matching feature _fS extracted in step S702.

In step S704, the calculation unit 206 calculates a matching correction coefficient cS by performing a calculation according to the following (Equation 8) using the registered feature group _FE held by the holding unit 204 and the matching feature _fS extracted in step _S702 .

Here, k _S is a parameter that determines the number of reference features used in calculating c _S , and f _Ej ∈F _E represents the k _S features in F _E that are closest to f _S.

In step S705, the correction unit 207 corrects the first matching score group _S1 using the registered correction coefficient group C _E and the matching correction coefficient c _S stored in the storage unit 501 according to the following (Equation 9) to calculate the second matching score group _S2 = { _s21 , _s22 , ..., _s2N }.

When a plurality of registered correction coefficient groups C _E are calculated in step S607, a registered correction coefficient group C _E corresponding to the attribute of the matching feature f _S may be selected from the plurality of registered correction coefficient groups C E in step S705.

In step S706, the authentication unit 208 compares each of the scores s ₂₁ , s ₂₂ , . . . , s _2N included in the second match score group S ₂ with a threshold value.

If the result of this magnitude comparison indicates that one of the scores _s21 , _s22 , ..., _s2N is equal to or greater than the threshold value, the authentication unit 208 determines that the target person is "the person corresponding to that one score" and displays information (image and/or text) representing "the person corresponding to that one score" on the video output device 103. Furthermore, the authentication unit 208 transmits an instruction to the control device included in the entrance/exit management device 106 to open the gate.

Furthermore, if the result of this magnitude comparison indicates that two or more of the scores _s21 , _s22 , ..., _s2N are equal to or greater than the threshold value, the authentication unit 208 determines that the target person is "the person corresponding to the maximum score of the two or more scores" and causes information (image and/or text) representing "the person corresponding to the maximum score of the two or more scores" to be displayed on the video output device 103. Furthermore, the authentication unit 208 transmits an instruction to the control device included in the entrance/exit management device 106 to open the gate.

If two or more of the scores _s21 , _s22 , ..., _s2N are equal to or higher than the threshold value, the authentication unit 208 may display information (images and/or characters) representing "persons corresponding to each of the two or more scores" on the video output device 103. In this case, the authentication unit 208 transmits an instruction to the control device included in the entrance/exit management device 106 to open the gate.

Furthermore, if the result of this magnitude comparison shows that none of the scores _s21 , _s22 , ..., _s2N is equal to or greater than the threshold value, the authentication unit 208 determines that the target person is "not a person permitted to pass through the gate" and displays information to that effect (image and/or text) on the video output device 103. Furthermore, the authentication unit 208 transmits an instruction to the control device included in the entrance/exit management device 106 to close the gate.

In this way, according to this embodiment, the registration side corrects the matching score taking into account the bias in the attributes of the matching side, and the matching side corrects the matching score taking into account the bias in the attributes of the registration side. Specifically, in the first embodiment, the reference feature sets were used to calculate the two correction coefficients, but in this embodiment, the registration feature set that is also the comparison target for calculating the first matching score set is used to calculate the matching correction coefficient. This makes it possible to suppress the variation in the matching score more than in the first embodiment and improve authentication accuracy.

[Third embodiment]
In the second embodiment, reference feature groups corresponding to various attributes are stored in advance in the storage unit 204, and a registered correction coefficient group C _E is generated based on a reference feature group Φ _S corresponding to a matching feature f _S among the reference feature groups stored in the storage unit 204 and a registered feature group F _E.

In contrast, in this embodiment, the registered correction coefficient group C _E is estimated from the registered feature group F _E. This eliminates the need to previously store a reference feature group for each attribute in the storage unit 204, thereby reducing memory costs.

In the biometric authentication system according to this embodiment, a registration correction coefficient C _E is generated in advance and registered in the storage unit 501. The process performed by the biometric authentication device 101 to generate the registration correction coefficient C _E and register it in the storage unit 501 will be described with reference to the flowchart in Fig. 8. The process according to the flowchart in Fig. 8 is started, for example, when an administrator of the biometric authentication system operates the input device 102 to input a registration instruction. Among the process steps shown in Fig. 8, the same process steps as those shown in Fig. 6 are assigned the same step numbers, and a description of these process steps will be omitted.

In step S801, the calculation unit 206 estimates the registered correction coefficient group C _E from the registered feature group F _E acquired by the selection unit 205 from the storage unit 204. In this embodiment, the calculation unit 206 estimates the registered correction coefficient group C _E from the registered feature group F _E using a correction coefficient estimator trained using a method similar to the learning method described in Non-Patent Document 1. That is, the calculation unit 206 prepares a new module (hereinafter, correction coefficient estimation head) that estimates the registered correction coefficient corresponding to the attribute of the biometric information by inputting the output (i.e., feature) of the neural network used in the extraction unit 202, and performs learning so as to minimize the loss function shown in the following (Equation 10). This makes it possible to obtain the registered correction coefficient without using the reference feature group during registration.

Here, i is any biometric information used for learning, M(x) is the output (i.e., feature) of the neural network when x is input to the neural network, H(x) is the output (i.e., correction coefficient) of the correction coefficient estimation head when x is input to the correction coefficient estimation head, _kL is a parameter that determines the number of reference features to be used, and _φLj ∈ _ΦL is the _kL feature values nearest to M(i) in the reference feature group _ΦL used for learning. Note that _ΦL is a feature group extracted by using the trained neural network for feature extraction from a part or all of the learning data used when training this neural network. Alternatively, when the attribute of the biometric information input during matching is known in advance as in this embodiment, a feature group extracted from a collection of data of the attribute may be used as _ΦL .

In step S801, the calculation unit 206 uses the correction coefficient estimation head obtained in this manner to estimate a registered correction coefficient group C _E corresponding to the attributes of the registered biometric information group I _E from the registered feature amount group F _E. Specifically, each element of the registered correction coefficient group C _E is given by the following (Equation 11).

However, this is merely one example, and the correction coefficient estimation head may be trained using the output of the intermediate layer of the neural network as input, or a machine learning model may be prepared that directly estimates the correction coefficient from biometric information. The correction coefficient estimation head may also be trained online each time biometric information is input in the registration process or matching process. Alternatively, multiple correction coefficient estimation heads may be trained using different methods, and all of the estimation results obtained may be stored, and one of these may be selected and used when correcting the score. This embodiment is not limited to this.

In this way, according to this embodiment, not only can the same correction effect as the second embodiment be obtained, but the calculation of the registered correction coefficient group can be simplified, thereby reducing the amount of calculation and memory usage and improving the authentication speed.

[Fourth embodiment]
In the above embodiments, the k nearest neighbors are selected from the reference feature group or the registered feature group in (Formula 2), (Formula 3), (Formula 6), and (Formula 8), and the correction coefficients are calculated using these, but other methods may be used. For example, all data of the reference feature group or the registered feature group may be used instead of the k nearest neighbors, or sample data of these may be used.

In addition, in (Equation 2), (Equation 3), (Equation 6), and (Equation 8), the exponents of the cosine similarity are taken, made non-negative, and summed up to be used as the correction coefficient, but other methods may be used. For example, instead of the sum, a statistical quantity such as the average, weighted average, or standard deviation may be used.

In addition, in each of the above embodiments, a reference feature set that matches the attribute of the comparison target is used, but in reality, there are cases where the attribute of the input biometric information cannot be known in advance. In such a case, the selection unit 205 can use an "attribute estimator that estimates attributes using features as input" to estimate attributes corresponding to the features of the biometric information, and use the estimated attributes when calculating the correction coefficient. Here, the attribute estimator takes features as input, but biometric information or intermediate output of a feature extraction model may also be used as input. In addition, instead of determining the reference feature set to be used when calculating the correction coefficient based on attributes, the average value of all features of the reference feature set may be calculated in advance as a representative feature, and the input feature and the representative feature may be compared at the time of selection, and the reference feature set with the closest distance in the feature space may be selected. Here, the average value is used as an example of the representative feature, but a weighted average or a randomly selected feature may also be used as the representative feature.

In addition, in each of the above embodiments, the storage unit 204 stores the reference feature set as the "information set for calculating the correction coefficient," but biometric information may be stored instead as the "information set for calculating the correction coefficient."

In addition, in each of the above embodiments, the biometric information is described as an image, but the biometric information may be any information that better indicates the characteristics of a living body. For example, the biometric information may be any one of face data, fingerprint data, palm print data, vein data, voice print data, iris data, etc., or a combination of these.

In the first embodiment, information related to the target person is obtained by reading an ID card, but it may be obtained by other methods. For example, the entrance/exit management device 106 may be provided with a reading device that reads information such as the target person's voice or fingerprint, and biometric information corresponding to the information read by the reading device may be obtained.

In addition, in each of the above embodiments, the biometric authentication device 101, the entrance/exit management device 106, and the biometric information sensor 105 are each separate devices, but for example, the biometric authentication device 101 may be integrated with the entrance/exit management device 106. Also, the biometric authentication device 101 may be integrated with the biometric information sensor 105. In this way, the biometric authentication device 101 may be integrated with any of the devices included in the biometric authentication system.

In addition, in each of the above embodiments, a system has been described that authenticates whether or not a target person is a person permitted to pass through a gate, and controls the opening and closing of the gate depending on the result of the authentication. However, the operation of the biometric authentication device 101 described in each of the above embodiments may be applied to any system that requires authentication of a target person. For example, the biometric authentication device 101 may be applied to a system that authenticates whether or not a target person is a "person permitted to use a device," and controls to allow operation of the device if the target person is a "person permitted to use a device."

The biometric authentication device 101 may be implemented as a single computer device or multiple computer devices. In the latter case, the multiple computer devices work together to perform the processes described as the processes performed by the biometric authentication device 101.

In addition, the numerical values, processing timing, processing order, processing subject, data (information) acquisition method/send destination/send source/storage location, etc. used in each of the above embodiments are given as examples to provide a concrete explanation, and are not intended to be limited to these examples.

Furthermore, any or all of the embodiments described above may be used in appropriate combination.Furthermore, any or all of the embodiments described above may be used selectively.

Other Embodiments
The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that implements one or more of the functions.

The invention of this specification includes the following information processing device, information processing method, and computer program.
(Item 1)
A first acquisition means for acquiring first biometric information and second biometric information;
a second acquisition means for acquiring a correction coefficient based on at least a feature amount of the first biometric information, a feature amount of the second biometric information, and an attribute of the first biometric information;
a correction means for calculating a score based on the feature amount of the first biometric information and the feature amount of the second biometric information, and correcting the calculated score using the correction coefficient;
and an authentication unit that performs authentication based on the corrected score corrected by the correction unit.
(Item 2)
The first acquisition means is
acquiring a captured image obtained by capturing an image of a person as the first biometric information;
2. The information processing apparatus according to item 1, further comprising: acquiring, as the second biometric information, a face image of the person that is registered in advance in association with information about the person.
(Item 3)
The second acquisition means is
determining a first correction coefficient based on the feature amount of the first biometric information and a first reference feature amount group corresponding to the attribute of the second biometric information among a reference feature amount group previously collected for each of a plurality of attributes;
3. The information processing apparatus according to item 1 or 2, characterized in that a second correction coefficient is calculated based on the feature of the second biometric information and a second reference feature group corresponding to the attribute of the first biometric information from among a reference feature group previously collected for each of a plurality of attributes.
(Item 4)
The first acquisition means is
acquiring a captured image obtained by capturing an image of a person as the first biometric information;
2. The information processing apparatus according to item 1, further comprising: acquiring a plurality of pre-registered face images as the second biometric information.
(Item 5)
The second acquisition means is
determining a first correction coefficient based on a first reference feature group corresponding to an attribute of the first biometric information from among a reference feature group previously collected for each of a plurality of attributes and the feature amount of the second biometric information;
5. The information processing apparatus according to item 1 or 4, further comprising: determining a second correction coefficient based on a feature amount of the first biometric information and a feature amount of the second biometric information.
(Item 6)
The second acquisition means is
determining a first correction coefficient corresponding to an attribute of the first biometric information based on a feature amount of the second biometric information;
2. The information processing apparatus according to item 1, further comprising: a second correction coefficient calculated based on a feature amount of the first biometric information and a feature amount of the second biometric information.
(Item 7)
7. The information processing apparatus according to any one of items 1 to 6, wherein the second acquisition means estimates an attribute corresponding to the first biometric information from a feature amount of the first biometric information.
(Item 8)
7. The information processing device according to any one of items 3, 5, and 6, wherein the correction means corrects the score using the first correction coefficient and the second correction coefficient.
(Item 9)
9. The information processing device according to item 8, wherein the correction means corrects the score using a parameter for filling a scale difference between the first correction coefficient and the second correction coefficient.
(Item 10)
10. The information processing device according to any one of items 1 to 9, wherein the attributes are tendencies of bias regarding information including race, sex, age, face direction, facial expression, lighting conditions, and the presence or absence of clothing.
(Item 11)
moreover,
11. The information processing apparatus according to any one of items 1 to 10, further comprising a transmitting means for transmitting a result of authentication by the authenticating means to a control device that controls opening and closing of a gate.
(Item 12)
An information processing method performed by an information processing device,
a first acquisition step in which a first acquisition means of the information processing device acquires first biometric information and second biometric information;
a second acquisition step in which a second acquisition means of the information processing device acquires a correction coefficient based on at least a feature amount of the first biometric information, a feature amount of the second biometric information, and an attribute of the first biometric information;
a correction step in which a correction means of the information processing device calculates a score based on the feature amount of the first biometric information and the feature amount of the second biometric information, and corrects the calculated score using the correction coefficient;
an authentication step in which an authentication means of the information processing device performs authentication based on the corrected score corrected in the correction step.
(Item 13)
12. A computer program for causing a computer to function as each of the means of the information processing device according to any one of items 1 to 11.

The invention is not limited to the above-described embodiment, and various modifications and variations are possible without departing from the spirit and scope of the invention. Therefore, the following claims are appended to disclose the scope of the invention.

201: Acquisition unit 202: Extraction unit 203: Calculation unit 204: Storage unit 205: Selection unit 206: Calculation unit 207: Correction unit 208: Authentication unit

Claims (13)

A first acquisition means for acquiring first biometric information and second biometric information;
a second acquisition means for acquiring a correction coefficient based on at least a feature amount of the first biometric information, a feature amount of the second biometric information, and an attribute of the first biometric information;
a correction means for calculating a score based on the feature amount of the first biometric information and the feature amount of the second biometric information, and correcting the calculated score using the correction coefficient;
and an authentication unit that performs authentication based on the corrected score corrected by the correction unit. The first acquisition means is
acquiring a captured image obtained by capturing an image of a person as the first biometric information;
The information processing apparatus according to claim 1 , further comprising: acquiring, as the second biometric information, a face image of the person that is registered in advance in association with the information of the person. The second acquisition means is
determining a first correction coefficient based on the feature amount of the first biometric information and a first reference feature amount group corresponding to the attribute of the second biometric information among a reference feature amount group previously collected for each of a plurality of attributes;
The information processing device according to claim 1, further comprising: a processor configured to calculate a second correction coefficient based on the feature of the second biometric information and a second reference feature group corresponding to the attribute of the first biometric information from among a group of reference features previously collected for each of a plurality of attributes. The first acquisition means is
acquiring a captured image obtained by capturing an image of a person as the first biometric information;
The information processing apparatus according to claim 1 , further comprising: acquiring a plurality of pre-registered face images as the second biometric information. The second acquisition means is
determining a first correction coefficient based on a first reference feature group corresponding to an attribute of the first biometric information from among a reference feature group previously collected for each of a plurality of attributes and the feature amount of the second biometric information;
The information processing apparatus according to claim 1 , further comprising: a second correction coefficient calculated based on a feature amount of the first biometric information and a feature amount of the second biometric information. The second acquisition means is
determining a first correction coefficient corresponding to an attribute of the first biometric information based on a feature amount of the second biometric information;
The information processing apparatus according to claim 1 , further comprising: a second correction coefficient calculated based on a feature amount of the first biometric information and a feature amount of the second biometric information. The information processing device according to claim 1, characterized in that the second acquisition means estimates attributes corresponding to the first biometric information from the feature amount of the first biometric information. The information processing device according to claim 3, characterized in that the correction means corrects the score using the first correction coefficient and the second correction coefficient. The information processing device according to claim 8, characterized in that the correction means corrects the score using a parameter for filling the scale difference between the first correction coefficient and the second correction coefficient. The information processing device according to claim 1, characterized in that the attributes are bias tendencies related to information including race, sex, age, face direction, facial expression, lighting conditions, and the presence or absence of clothing. moreover,
2. The information processing apparatus according to claim 1, further comprising a transmitting means for transmitting a result of authentication by said authenticating means to a control device that controls opening and closing of a gate. An information processing method performed by an information processing device,
a first acquisition step in which a first acquisition means of the information processing device acquires first biometric information and second biometric information;
a second acquisition step in which a second acquisition means of the information processing device acquires a correction coefficient based on at least a feature amount of the first biometric information, a feature amount of the second biometric information, and an attribute of the first biometric information;
a correction step in which a correction means of the information processing device calculates a score based on the feature amount of the first biometric information and the feature amount of the second biometric information, and corrects the calculated score using the correction coefficient;
an authentication step in which an authentication means of the information processing device performs authentication based on the corrected score corrected in the correction step. A computer program for causing a computer to function as each of the means of an information processing device according to any one of claims 1 to 11.

Images