JP4553526B2 - Personal verification device - Google Patents

Description

[0001]
[Field of the Invention]
The present invention relates to a collation apparatus that performs personal authentication based on human physical features such as facial images, fingerprints, voices, irises, and DNAs, and more particularly to a personal collation apparatus that performs personal verification by weighting a plurality of feature amounts.
[0002]
[Prior art]
Conventionally, a personal verification device has been proposed in which a plurality of feature amounts are extracted from a face image, a fingerprint image, an audio signal, and the like, and collated with a personal feature registered in advance.
For example, in the face image matching device, feature amounts are extracted for each of a plurality of features such as face contours, part arrangements such as eyes and mouths, and shapes of eyes, and the degree of similarity with each feature amount of a registered face image is calculated. To do. Each similarity is weighted with a contribution based on general knowledge obtained in advance through experiments and research, and whether or not it is the person is verified. General knowledge is that the shape and contour of the eyes are highly personal, and the nose is not very personal.
[0003]
[Problems to be solved by the invention]
However, in the conventional personal verification device, since weighting is performed on each feature amount based on general knowledge, weighting considering personality for each registrant cannot be performed. In general, weighting for “eye shape and the like” becomes large, but “part arrangement” may show the individuality most depending on the person. Even in such a case, the conventional personal verification device only increases the weight of “eye shape etc.” and cannot increase the weight of “part arrangement”. There was a problem of lacking.
Since such individual correspondence cannot be achieved, there is a problem that the rejection rate and the other person's acceptance rate increase, and the reliability of the personal verification device is impaired.
Accordingly, an object of the present invention is to realize a personal verification device capable of weighting features in accordance with the personality appearing on the characteristics of the collator, and to improve the reliability of the personal verification device. .
[0004]
[Means for Solving the Problems]
In order to achieve such an object, the present invention is a personal verification device that extracts a plurality of features indicating physical features and takes into account the contribution to personal verification for each feature, Input means for inputting, feature information extracting means for extracting feature values for each of the plurality of features from each of the reference information including physical features serving as a reference and the registered information including physical features of the registrant; Similarity calculating means for calculating the similarity between the reference information and the registered information using the feature amount extracted by the feature amount extracting means; and a contribution degree calculating means for calculating a contribution degree for matching each feature from the similarity degree; And a personal verification device characterized by comprising a determination means for performing personal verification using the degree of similarity weighted by the contribution.
According to this configuration, the degree of contribution of each feature to identity determination is determined based on the similarity between arbitrary reference information and registration information, and thus it is possible to weight each feature that reflects the individuality of each registrant. .
[0005]
In a preferred aspect of the present invention, the contribution calculation means corrects the reference contribution obtained in advance according to the similarity. Thereby, it is possible to prevent weighting that deviates extremely from the standard contribution obtained from general knowledge.
[0006]
In a preferred aspect of the present invention, the contribution calculation means increases the contribution as the similarity is lower. As a result, since feature values that are greatly different from the reference information and the registered information are given greater weight, the personality of the registered information can be reflected in the verification.
[0007]
In a preferred aspect of the present invention, the reference information is generated by averaging information of a plurality of people. Thereby, since it becomes possible to extract the individuality of the registrant with respect to general physical characteristics, it is possible to make the personal verification device versatile.
[0008]
In a preferred aspect of the present invention, a face image is used as a physical feature. As a result, it is possible to use a plurality of facial contours, part arrangements, and the like from a facial image showing one physical feature.
[0009]
In a preferred aspect of the present invention, a face image and an audio signal are used as physical features. Thereby, the feature obtained from the face image and the feature obtained from the audio signal can be used.
[0010]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, as an embodiment of the present invention, a mode in which the present invention is applied to the face image matching device 1 will be described.
FIG. 2 is an overall block diagram of the face image matching device 1. The face image collation apparatus 1 is composed of an imaging unit 110 that captures a collator's face composed of a lens, a CCD element, etc., a numeric keypad 120 that is an operation unit, and a ROM / RAM. A storage unit 130 for storing, a display unit 140 composed of an LCD or LED and presenting a collation result to the operator, and an output for outputting a lock / unlock signal to the electric lock 160 of the doorway to be managed And a control unit 100 that is configured by a CPU and the like and that controls the entire face image matching device 1. Note that the image input unit 111 and the operation input unit 121 are A / D converters that respectively A / D convert operations from the imaging unit 110 or the numeric keypad 120 and input them to the control unit 100.
The control unit 100 executes a registration process 101, a verification process 102, an electric lock control process 103, and the like.
The storage unit 130 stores a reference contribution 131, a reference face image 132, a registered face image 133, a contribution 134, a program 135, and the like.
The output unit 150 is connected to an electric lock 160 installed at a doorway to be managed.
[0011]
Next, each process in the control unit 100 of the face image matching device 1 will be described.
First, the generation process of the reference face image 132, which is one of the registration processes 101, will be described with reference to FIG. The registration process 101 is a process when the administrator sets the registration mode with the numeric keypad 120. The registration mode is shifted when the administrator inputs an ID code or the like from the numeric keypad 120 and is authenticated as a regular administrator.
[0012]
Here, the reference face image 132 is a face image serving as a reference for extracting a feature amount having a strong individuality from the face image of the registrant. If the face image collation device is versatile, it is preferable to use an average Japanese face image. If the registrant is a male, an average facial image of a male may be used. In addition, it is possible to consider age, race, and the like. The reference face image 132 may be a face image that is arbitrarily fixed and may be selected according to the collator.
[0013]
In the present embodiment, the reference face image 132 that is integrated after the feature amount is averaged for each of the features of the plurality of face images shown in FIG. In the averaging process, first, a feature amount is extracted from each of a plurality of arbitrarily selected face images for each part adopted as a feature such as contour, left eye, right eye, nose, and mouth. Note that the part extraction processing uses a general method such as extracting a contour based on a difference from a background image or extracting an area having a strong horizontal edge as an eye. Next, after performing a matching process such as a size for each part, a luminance average of each pixel is obtained for each part, thereby averaging each part in a plurality of face images. Then, the averaged parts are integrated to obtain one reference face image 132. The reference face image 132 generated in this way is a face image suitable for a reference with less individuality because the individuality of a plurality of face images is averaged. The reference face image 132 obtained in this way is stored in the storage unit 130 in advance. The reference face image 132 subjected to the averaging process is used. However, the present invention is not limited to this, and any face image may be used as the reference face image 132.
[0014]
Next, the registration process 101 performed by the control unit 100 will be described with reference to FIG.
First, the reference face image 132 and the registered face image 133 are read from the storage unit 130.
Here, the registered face image 133 is a face image of a person permitted to enter and exit by the face image matching device. The registered face image 133 is set to the registration mode by the administrator, and then the registrant's face is photographed by the imaging unit 110, A / D converted by the image input unit 111, and stored as a digital image in the storage unit 130. Yes. The registered face image 133 is stored together with an ID code for each registrant.
Feature amounts are extracted from the reference face image 132 and the registered face image 133 for each feature of “contour”, “part location”, “left eye”, “right eye”, “nose”, and “mouth” to the feature amount extraction unit 2. To extract.
That is, in the extraction of “right eye” and “left eye”, first, an edge is extracted by a Sobel operator or the like from an area extracted as a difference between a background image in which a person is not captured and a reference face image or the like. Of these, regions that have strong horizontal edges and that have a strong matching degree as a result of the shift matching using the standard “right eye” or “left eye” template image are determined as candidate regions. Out of the candidate areas extracted by the above processing, an area that is substantially in the left and right direction and located in the upper half of the head area with respect to the vertical center line of the face is extracted as an eye area.
The “nose” is extracted as an inverted T-shaped region in a predetermined region below the midpoint of the line connecting the “right eye” and “left eye”.
“Mouth” is extracted as a region having a strong horizontal edge in a preset region below “nose”. When a color camera is used, it is possible to combine the characteristics that the red component is relatively stronger than the surrounding area.
The “contour” is a feature indicating the shape of the head region, and is determined by combining the region including the head and neck extracted as the difference between the input face image and the background image and the shape of the jaw. The shape of the jaw is extracted as a boundary with a shadow formed on the neck along the shape of the jaw, for example, in a parabolic region centered on the “mouth” of the input face image. The “contour” is extracted by extracting the edge of the region located above the jaw shape from the region extracted as the difference from the background image.
“Partial arrangement” includes the horizontal distance between the centroid coordinates of the “right eye” and “left eye” areas, the vertical distance between the centroid coordinates of the centroid coordinates and the centroid coordinates of the “nose” area, the midpoint of the centroid coordinates and the “mouth” The vertical distance between the centroid coordinates of the “region”, the vertical distance between the centroid coordinates of the “nose” region and the “mouth” region, the centroid coordinates of the “nose” region and the straight line connecting the centroid coordinates of the “right eye” or “left eye” region And the angle formed by the straight line connecting the barycentric coordinates of the “right eye” or “left eye” area and the barycentric coordinates of the “mouth” area.
[0015]
Next, the similarity between each feature amount extracted from the reference face image 132 extracted by the feature amount extraction unit 2 and each feature amount extracted from the registered face image 133 is sent to the similarity calculation unit 3 for each feature. To calculate.
The similarity of each feature will be specifically described.
The similarity between “contours” is calculated as a difference area surrounded by edges between “contours” extracted from the reference face image 132 and the registered face image 133, that is, a gap area. As a result, a difference in characteristics related to the contour shape (for example, “the jaw is sharp”, “the heel is stretched”, etc.) is expressed.
The similarity of “part placement” refers to the distance and angle between the barycentric coordinates of each part indicating characteristics related to the relative position of each part (such as “wide eye spacing” or “parts gathered in the center of the face”). Calculate based on
The similarity of “right eye”, “left eye”, “nose”, and “mouth” is between edges indicating the shape (large eyes / thin / hanging / upward eyes, wide mouth / narrow / thick / thin) Are calculated based on the gap area, edge strength difference, and edge gradient difference. Alternatively, it is also possible to obtain the similarity as a correlation value obtained by shifting and matching the region image. For example, when a color camera is used, a grayscale image, a hue image, a saturation image, or the like indicating a color (brightness, color saturation, etc.) can be used.
Thus, since the similarity calculated for each feature amount is a different scale such as a distance or an angle, it is necessary to match with the same scale that can compare these magnitude relationships.
First, the magnitude relationship of the calculated values is unified. That is, processing such as calculating a reciprocal from a distance or angle value or subtracting from a predetermined value is performed so as to follow a similarity property of “similar as the value increases”.
Next, the origin of the calculated value is unified. Similarities are obtained in advance between the reference face image 132 and all the face images used to calculate the reference face image 132, and the features extracted from the reference face image 132 and the registered face image 133 using these average values. Normalization is performed so that the average value becomes 1 by dividing the similarity between the quantities.
Then, the variance of the calculated values is unified. The normalized value is corrected using the variance value of the similarity obtained between the reference face image 132 and all the face images used to calculate the reference face image 132. As a result, the variance of the calculated similarity is kept within a certain range.
[0016]
Next, a method for calculating the contribution 134 executed by the contribution calculation unit 4 will be described.
First, the reference contribution 131 for each feature is read from the storage unit 130. Here, the reference contribution 131 is a degree to which each feature contributes to the personal verification, and is a degree obtained from experimental or research findings. For example, a matching experiment was performed individually on four parts of “right eye”, “left eye”, “nose”, and “mouth”, and weights were experimentally examined based on the obtained matching performance. It is known that collation accuracy is improved by increasing the weight in the order of “mouth” and “nose”.
In addition, according to the presentation by Inoue et al., As a result of consideration from a psychological aspect, it was found that "eye size" and "face contour below eyes" are important features for identification ( Chemical Journal Vol.34 No.3, p249-p254).
An example of the “reference contribution 131” obtained in consideration of these results is shown in FIG.
As the degree of similarity is larger, the registered face image 133 is more similar to the reference face image 132. Therefore, in order to reduce the contribution to similar features, the reference contribution 131 (B) is divided by the similarity (S) (this value is defined as the pre-correction contribution (W ′)). When the similarity (S) is as close as possible to 0 or 0, the contribution is infinite, and the other features are ignored. In this way, a lower limit is set for the similarity (S) so that the contribution 134 does not deviate abnormally. For example, when the lower limit of the similarity (S) is set to “0.4”, “0.4” is used when the similarity (S) is less than this. In FIG. 4, the actually calculated similarity (S) of “contour” is “0.3”, but this is replaced with the predetermined lower limit value “0.4”. Set the upper limit in the same way.
The contribution 134 (W) is obtained by dividing the pre-correction contribution (W ′) by the sum of the pre-correction contribution (W ′) obtained for all the feature quantities (“1.146” in FIG. 4). Is calculated as a normalized value.
FIG. 4 shows an example of the calculation result. In the case of “contour”, the similarity is “0.4”, the reference contribution (B) 131 is “0.15”, and the sum of the contributions W ′ before correction is “1.146”, so the contribution 134 is “0.327”. However, in order to set the sum of the calculated contributions to “1”, it is adjusted to “0.326” by adjusting the contribution of the “contour” which is the maximum contribution. Since the “contour” has individuality as compared with the reference face image 132, it can be seen that the “contour” is higher than the reference contribution 131 “0.15”, which is the contribution used conventionally. In the case of “nose”, the similarity is “1.4” and the reference contribution 131 is “0.10”, so if the same calculation as described above is performed, the contribution 134 becomes “0.062”. It can be seen that the “nose” is lower in personality than the reference face image 132, and is therefore lower than the reference contribution 131 “0.10”.
Even if the reference contribution 131 that is the contribution of each feature obtained from the knowledge is not obtained, the above contribution calculation method is used as it is by setting the reference contribution 131 to “1”. Can do.
Each calculated contribution 134 is stored in the storage unit 130 in association with the registered face image 133 (storage process 5).
It should be noted that the registered face image 133 of the present embodiment is that of a person who has the strongest personality in the contour and has little personality in the nose.
In this way, the person to be verified who is permitted to enter and exit the managed entrance door is registered. In addition, when registering a plurality of people, the same processing is repeated.
[0017]
Next, referring to FIG. 5, the collation processing 102 of the personal collation apparatus will be described.
First, the collator inputs his / her ID code from the numeric keypad 120 according to the guidance of the display unit 140 and photographs the face by the imaging unit 110. At the time of photographing a face, for example, an indication of the binocular position and the face being photographed are displayed on the display unit 140, so that the collator adjusts to match that position. When the position is adjusted, the numeric keypad 120 is operated to determine the face image.
When an input is received from the numeric keypad 120 for determining the face image, a collation process 102 shown in FIG. 5 is executed.
Based on the ID code input previously, the registered face image 133 of the collator is read from the storage unit 130. The feature amount extraction unit 2 extracts feature amounts of “contour”, “part location”, “left eye”, “right eye”, “nose”, and “mouth” from the registered face image 133 and the input face image. The extracted feature amount is input to the similarity calculation means 3.
The similarity calculation unit 3 calculates the similarity between each feature quantity extracted from the reference face image 132 and each feature quantity extracted from the registered face image 133 for each feature quantity. The feature amount extraction unit 2 and the similarity calculation unit 3 perform the same processing as the feature amount extraction processing unit 2 and the similarity calculation processing unit 3 described with reference to FIG.
Then, a weighting process 6 is performed for each calculated similarity. That is, the similarity for determination of each feature amount is calculated by multiplying the corresponding degree of similarity by the contribution degree 134 stored in the storage unit 130. Furthermore, the total similarity that reflects the individuality is calculated by obtaining the sum of the similarities for determination obtained for all the feature quantities. In this way, by assigning a weight according to individuality to each feature amount, it is possible to place importance on feature amounts that contain a lot of individuality compared to feature amounts that have poor individuality. As a result, it is possible to perform collation with emphasis on characteristics different from others, that is, individuality.
Based on the total similarity, it is determined in the comprehensive determination 7 whether or not the user is the person himself / herself. That is, when the total similarity exceeds a predetermined value, it is determined that the collator is the registrant. On the other hand, if it is less than the predetermined value, it is determined that the person is not the person.
As a result of the determination, if the user is confirmed, the display unit 140 displays “Please enter” and sends an unlock signal from the output unit 150 to the electric lock 160. When the unlock signal is received, the electric lock 160 is unlocked. The electric lock control is executed as an electric lock control process 103.
On the other hand, if it is determined that the user is not the person himself / herself, only that fact is displayed on the display unit 140 and the electric lock 160 is not unlocked.
As a result, if the collator is the person himself / herself, the person can pass through the entrance / exit door and can reject the passage of persons other than the person himself / herself.
[0018]
In the present embodiment, the personal collation device using a face image has been described. However, the present invention is not limited to this, and the present invention can be similarly applied to a fingerprint image and an audio signal. In addition, when applying to a fingerprint image or an audio | voice signal, it cannot be overemphasized that the feature-value established as each authentication technique is selected suitably as a feature-value. For example, the case of an audio signal will be described. Here, a description will be given by taking as an example “speaking content fixed type” speaker verification in which the voice used for personal verification is fixed in advance for each collator. First, the input speech signal is subjected to frame analysis with a predetermined frame length and frame period, and a frame corresponding to a phoneme is extracted by a known recognition method such as HMM. The phoneme sequence obtained as a result corresponds to each feature in the face image such as eyes and nose. From the extracted phoneme sequence, for example, an LPC cepstrum coefficient corresponding to a spectrum envelope, which is an outline of the frequency intensity distribution, is calculated as a feature amount. When the phoneme series is extracted as a plurality of continuous frames, for example, an LPC cepstrum coefficient is calculated from a frame located at the center thereof, or a value obtained by averaging the LPC cepstrum coefficients calculated for each frame is used. For each extracted feature quantity, by performing the same process as in the case of the face image, it is possible to weight each phoneme series in consideration of individuality. For the processing of the similarity calculation means, the contribution calculation means, and the like, a method that has already been generalized as an audio signal authentication technique is used. Although there are various methods, the description is omitted because it is not the essence of the present invention.
[0019]
In this embodiment, a plurality of features extracted from a face image have been described. However, the contribution of each feature when combining various physical features such as a face image and an audio signal, or a face image and a fingerprint image. May be applied at any time. For example, a case where a face image and an audio signal are combined will be described. First. The reference face image and the reference voice information described above are stored in advance in the storage unit 130 as reference information serving as a reference for personality. The significance of the reference information is as described above. Next, the feature amount extraction unit 2 extracts the above-described feature amount from both the registered face image and the registered voice information. Specifically, it is “contour”, “part arrangement” or the like in the case of a face image, and “phonological sequence” in the case of a speech signal. As described above, a plurality of feature amounts included in each of the face image and the sound signal may be used. However, the face image information and the sound signal as a whole are regarded as one feature amount, and the two features of the face image and the sound signal are combined. It is good also as extracting quantity.
[0020]
【The invention's effect】
According to the present invention, by assigning a weight according to individuality to each feature, it is possible to perform collation with emphasis on features that contain a lot of individuality compared to features that have little individuality. As a result, it is possible to perform high-precision collation that emphasizes different characteristics from others, that is, individuality.
[Brief description of the drawings]
FIG. 1 is a diagram illustrating registration processing in a face image matching device. FIG. 2 is a block diagram of the face image matching device. FIG. 3 is a diagram illustrating generation of a reference face image. FIG. 5 is a diagram explaining collation processing in the face image collation apparatus.
1 ... Face image matching device
100: Control unit, 101: Registration process, 102: Verification process, 103: Electric lock control process
110: Imaging unit, 111 ... Image input unit, 120 ... Numeric keypad, 121 ... Operation input unit
130: storage unit, 131: reference contribution, 132: reference face image, 133: registered face image, 134: contribution, 135: program, etc. 136: collation face image
140 ... display unit, 150 ... output unit, 160 ... electric lock,
2 ... feature quantity extraction means, 3 ... similarity calculation means, 4 ... contribution calculation means, 5 ... storage processing,
6 ... Weighting processing, 7 ... Overall judgment, 8 ... Electric lock control, etc.

Claims (1)

A personal verification device that extracts a plurality of features from a human face image and considers the contribution to personal verification for each feature,
An input means for inputting the face image of the collator;
A feature amount extraction means for calculating a feature amount for each of the plurality of features from a reference face image and a registered face image generated by averaging a plurality of face images ;
Similarity calculation means for calculating the similarity between the reference face image and the registered face image for each feature using the feature quantity extracted by the feature quantity extraction means;
A contribution calculation means for dividing a predetermined reference contribution by the similarity and calculating a contribution to matching of each feature ;
A personal verification apparatus comprising: a determination unit that performs personal verification using the similarity weighted by the contribution.

Images