KR20190098672A - Method and apparatus of performing combined authentication - Google Patents

Abstract

According to one embodiment of the present invention, provided are a method and an apparatus for performing complex authentication. The method for performing complex authentication performs sole authentication based on a first modality among modalities, determines whether to perform complex authentication based on a combination of the modalities when the sole authentication fails, and performs the complex authentication in accordance with the determination on whether to perform the complex authentication.

Description

METHOD AND APPARATUS OF PERFORMING COMBINED AUTHENTICATION}

The embodiments below relate to a method and apparatus for performing complex authentication.

With the development of various mobile devices including smart phones, and user devices such as wearable devices, the importance of security authentication is increasing, and accordingly, interest in biometrics is increasing. Biometrics further enhance the security of user devices and allow more secure use of applications such as mobile payments. Biometrics has high authentication rates and widespread utility.

According to an embodiment, a method of performing complex authentication may include performing sole authentication based on a first modality among modalities; Determining whether to perform complex authentication based on a combination of the modalities when the single authentication fails; And performing the complex authentication according to the determination of whether to perform the above.

The determining of whether to perform the composite authentication may include determining whether a second condition for the composite authentication is satisfied, which is distinguished from the first condition for the single authentication. The second condition may be a condition different from the first condition.

The performing of the independent authentication may include determining whether the first characteristic of the first modality satisfies the first condition.

The determining of whether to perform the complex authentication may include determining whether a first characteristic of the first modality satisfies the second condition.

The modalities include the first modality and a second modality different from the first modality, and the determining of whether to perform the composite authentication is a first feature of the first modality, a second feature of the second modality Or determining whether the combination of the first feature and the second feature satisfies the second condition.

The performing of the complex authentication may include: generating a third feature by fusion of the first feature and the second feature according to a determination of performing the complex authentication; And performing the complex authentication based on the third feature.

The performing of the complex authentication may include determining whether the third feature satisfies a third condition for the complex authentication.

The second condition may be determined differently for each combination of the modalities.

The second condition may be determined based on a false acceptance rate (FAR) of the most secure modalities among the modalities.

The second condition may be determined based on a false positive rate of the modality having the highest convenience among the modalities.

The combination of modalities may be determined based on the security of the complex authentication or the convenience of the complex authentication.

The modalities may include any one or a combination of a face image, a fingerprint image, an iris image, a vein image, a palm image, a sign, a voice, a gait, a DNA structure of a user.

According to an embodiment, a method of performing complex authentication may include determining whether to perform complex authentication based on at least one of a first characteristic of a first modality and a second characteristic of a second modality; And performing composite authentication based on the first feature and the second feature, depending on whether the composite authentication is performed.

The performing of the composite authentication may include generating a third feature by combining the first feature and the second feature; And performing the complex authentication based on the third feature.

Determining whether the composite authentication is performed may include whether the first feature, the second feature, or a combination of the first feature and the second feature satisfies a second condition different from the first condition for the single authentication. Determining; And if the second condition is satisfied, determining to perform complex authentication by a combination of the first modality and the second modalities.

The second condition may be determined based on a first false recognition rate FAR of the first modality, a second false recognition rate of the second modality, or a combination of the first false recognition rate and the second false recognition rate.

The performing of the complex authentication may include determining whether the third feature satisfies a third condition for the complex authentication.

According to an embodiment, a method of performing complex authentication may include determining whether a first entry condition corresponding to a first combination of modalities is satisfied; Performing authentication by the first combination according to whether the first entry condition is satisfied; If authentication by the first combination fails, determining whether a second entry condition corresponding to the second combination of modalities is satisfied; And performing authentication by the second combination according to whether the second entry condition is satisfied.

The first entry condition and the second entry condition may be determined differently for each combination of the modalities.

The determining of whether the first entry condition is satisfied may include determining whether the first entry condition is satisfied based on whether the first combination of modalities satisfies a second condition different from the first condition for the single authentication. It may include.

The first combination of modalities includes a first modality and a second modality different from the first modality, and determining whether the first entry condition is satisfied comprises: a first characteristic of the first modality and the second modality Determining whether at least one of the second features of satisfies the second condition; And when the second condition is satisfied, determining that the first entry condition is satisfied.

According to an embodiment, a method of performing complex authentication may include performing sole authentication based on a first modality among modalities; And when the single authentication fails, performing complex authentication by a combination of the modalities.

The method for performing the complex authentication may further include performing an exclusive authentication based on a second modality among the modalities before performing the complex authentication when the single authentication fails.

The performing of the composite authentication may include generating a third feature by combining the first feature of the first modality and the second feature of the second modalities; And performing the complex authentication based on the third feature.

According to an embodiment of the present disclosure, a biometric authentication method for authenticating a user by using different first biomodality and second biomodality may include a first condition and a second biometric in which the user's biometric information reflects a characteristic of the first biomodality. Determining whether one of the second conditions in which the characteristic of the modality is reflected is satisfied; Determining whether the biometric information of the user satisfies a complex condition in which the feature of the first biomodality and the feature of the second modality are reflected; And if the biometric information of the user satisfies any one of the first condition and the second condition and satisfies the compound condition, determining that the compound authentication is successful.

The complex condition may be a condition in which a feature in which the feature of the first bio-modality and the feature of the second bio-modality are fused are reflected.

The complex condition may be a condition in which a score based on a feature of the first biomodality and a score based on a feature of the second biomodality are combined.

Determining whether the complex condition is satisfied comprises: generating a third feature by fusion of the feature of the first biomodality and the feature of the second biomodality; And determining whether the complex condition in which the third feature is reflected is satisfied.

Determining whether the complex condition is satisfied comprises: generating a third score by combining a first score based on the feature of the first biomodality and a second score based on the feature of the second biomodality; And determining whether the complex condition based on the third score is satisfied.

The first biomodality may be an iris modality and the second biomodality may be a face modality.

1 is a view for explaining a method of performing a composite authentication according to an embodiment.
2 is a flowchart illustrating a method of performing complex authentication according to an embodiment.
3A and 3B are diagrams for describing a first condition and a second condition, according to an embodiment.
4 is a block diagram of an apparatus for performing complex authentication according to an embodiment.
5 is a diagram for describing an operation of a classifier for performing complex authentication, according to an exemplary embodiment.
6A, 6B, and 7 through 11 are flowcharts illustrating a method of performing complex authentication according to embodiments.
12 illustrates a user interface in accordance with one embodiment.
13A and 13B illustrate a process of registering biometrics information according to an embodiment.
14 illustrates an operation of performing complex authentication by using correlation between different modalities according to an embodiment.
15 is a block diagram of an apparatus for performing complex authentication according to an embodiment.

The specific structural or functional descriptions disclosed herein are merely illustrative for the purpose of describing embodiments in accordance with technical concepts. The disclosed embodiments may be modified in various other forms and the scope of the present disclosure is not limited to the disclosed embodiments.

Terms such as first or second may be used to describe various components, but such terms should be interpreted only for the purpose of distinguishing one component from another component. For example, the first component may be referred to as a second component, and similarly, the second component may also be referred to as a first component.

When a component is referred to as being "connected" to another component, it should be understood that there may be a direct connection or connection to that other component, but there may be other components in between.

Singular expressions include plural expressions unless the context clearly indicates otherwise. As used herein, the terms "comprise" or "have" are intended to designate that the described feature, number, step, operation, component, part, or combination thereof exists, but includes one or more other features or numbers, It is to be understood that it does not exclude in advance the possibility of the presence or addition of steps, actions, components, parts or combinations thereof.

Unless defined otherwise, all terms used herein, including technical or scientific terms, have the same meaning as commonly understood by one of ordinary skill in the art. Terms such as those defined in the commonly used dictionaries should be construed as having meanings consistent with the meanings in the context of the related art, and are not construed in ideal or excessively formal meanings unless expressly defined herein. Do not.

Embodiments may be implemented in various forms of products, such as personal computers, laptop computers, tablet computers, smart phones, televisions, smart home appliances, intelligent cars, kiosks, wearable devices, and the like. For example, embodiments may be applied to user authentication in smart phones, mobile devices, smart home systems, intelligent cars, ATM devices, and the like. Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. Like reference numerals in the drawings denote like elements.

1 is a diagram for describing a method of performing complex authentication, according to an exemplary embodiment. Referring to FIG. 1, a situation in which the user 50 performs biometric recognition or authentication using the image sensor 110, the infrared sensor 120, or the fingerprint sensor 130 of the mobile device 100 is illustrated.

According to one embodiment, biometrics or authentication may be performed using a plurality of modalities based on various biometric information such as face, iris, fingerprint, and vein. 'Modality' refers to a user's own biometric information, such as, for example, a user's face, fingerprint, iris, vein, palm, sign, voice, gait, DNA structure, etc. Alternatively, the present invention may correspond to various aspects in which unique information for recognizing or authenticating a user is expressed.

In using various biometric information, the security level of biometric authentication may be different for each modality. For example, the iris modality may have a False Acceptance Rate (FAR) of ten million, and the fingerprint modality and facial modality may have a million percent false recognition rate. The 'perception rate' may correspond to a rate at which a person's biometric information may be misrecognized as his or her biometric information. The lower the recognition rate of the biometric information, the higher the security, that is, the higher the security level may correspond to the modality.

In the case of using a composite authentication that performs authentication by combining various modalities, security may be improved by utilizing the advantages of each modality. According to embodiments, authentication performance for various situations may be guaranteed by complex authentication through a combination of modalities.

For example, suppose a user is authenticated using an iris image and a face image. The iris image and the face image may be photographed through an infrared (IR) camera, a color camera, a monochrome camera, or a 3D camera, respectively. The 3D camera may be implemented in various ways, such as a ToF (Time-of-Flight) type camera or a Structured Light type camera, and the above-described methods are merely exemplary and the 3D camera according to the embodiment. Is not limited to the foregoing manners. According to an embodiment, the iris image may be photographed through an infrared camera, and the face image may be photographed through a color camera or a 3D camera.

Iris modality has higher security than face modality. Suppose that user A and user B are twins. In iris modality, user A and user B have different feature vectors at the other person level, but in face modality, user A and user B may have similar feature vectors at the same level. Thus, the twins are difficult to distinguish from each other based on face modality, but can be well distinguished from each other based on iris modality.

However, the performance of the iris modality may be degraded compared to the performance of the face modality depending on the shooting environment. The iris image may be, for example, photographed using an infrared sensor after light emission of an infrared LED (LED) included in a mobile device. The iris image may be deteriorated in quality, such as when there is strong outdoor light. When the quality of the image is deteriorated, the recognition performance is deteriorated. Therefore, when only the iris modality is used, the recognition performance may be deteriorated in a specific environment such as the presence of strong light in the outdoors.

Since the face image does not deteriorate the quality of the image even when there is strong outdoor light, the recognition performance of the face modality may not be degraded even when the recognition performance of the iris modality is degraded. However, unlike the iris image, the face image may be vulnerable to a change in the type of external lighting, a low illumination situation, or a change in pose of the face.

Therefore, the recognition performance can be maintained in various situations by using the iris image and the face image in combination.

Embodiments to be described below may consider not only security but also user convenience in performing complex authentication based on a combination of modalities. For example, embodiments may provide a technology for improving user convenience while maintaining security, or may provide a technology for improving security while maintaining user convenience.

Embodiments may preferentially perform a single authentication and then perform complex authentication.

For example, security should be considered as important in a payment application, and a scenario of performing single authentication and complex authentication may be set to improve user convenience while maintaining security. Embodiments perform a single authentication primarily using a highly secure modality (eg, iris modality), and then, when primary authentication fails, a plurality of modalities (eg, iris and face modality). Can be combined to perform complex authentication.

As another example, the convenience of the user should be considered as important in unlocking the smart phone, and a scenario of performing single authentication and complex authentication may be set to improve security while maintaining user convenience. Embodiments perform a single authentication primarily using a user-friendly modality (for example, face modality), and when the first authentication fails, a modality of a characteristic distinguished from the modality used for the first authentication (eg For example, iris modality) may be used to perform a second single authentication. Embodiments may perform complex authentication by combining a plurality of modalities (eg, face modality and iris modality) when the second authentication fails.

Embodiments provide a technology that satisfies both security and user convenience by determining an entry condition for determining whether to perform complex authentication before performing complex authentication. The entry condition may be predetermined according to a combination of modalities for complex authentication or the type of application to which complex authentication is applied.

The entry condition for determining whether to perform the composite authentication may be a condition regarding the modality of at least one of the modalities for the composite authentication. For example, the entry condition may include a first condition for face modality, a second condition for iris modality, or a combination of the first condition and the second condition. The entry condition may be a condition that is distinguished from the reference condition for determining whether the single authentication using the individual modality is successful.

According to an embodiment, the entry condition may be a condition different from the reference condition for determining whether the single authentication using the individual modality is successful. For example, if the entry condition includes a second condition regarding iris modality, the second condition may be a condition that requires a reduced false recognition rate compared to the false recognition rate of single authentication using iris modality.

For example, complex authentication may be applied to a payment application. In this case, while focusing on security, an entry condition for improving user convenience may be set. Embodiments determine the entry conditions for determining whether to perform complex authentication using a highly secure modality (eg, iris modality), and then combine a plurality of modalities (eg, iris modality and face modality). Composite authentication can be performed.

As another example, complex authentication may be applied to unlock the smartphone. In this case, the user's convenience may be emphasized, but an entry condition for improving security may be set. Embodiments determine the entry conditions for determining whether to perform complex authentication using a single modality (eg, iris modality, face modality or fingerprint modality), and then determine a plurality of modalities (eg, face modality, fingerprint modality). , And iris modality) may be combined to perform complex authentication.

As described in detail below, the entry condition may be set to use a combination of single modalities, instead of using only a single modality. In addition, when performing complex authentication, various methods of combining a plurality of modalities may be set.

The score of each modality may be determined in the form of, for example, a matching score or a distance score. The matching score may correspond to a score indicating a similarity between the registration frame and the input frame, that is, the similarity between the frames. Lower matching scores mean lower similarities between frames, and higher matching scores mean higher similarities between frames. The higher the matching score is, the more likely the mobile device 100 will accept the authentication of the user 50. In addition, the distance score may correspond to a score indicating a feature distance (eg, Euclidean distance) between the registration frame and the input frame. The low distance score means that the feature distance between frames in the feature vector space is close, and the high distance score means that the feature distance between the frames is far. The lower the distance score of the mobile device 100 is, the higher the probability of accepting the authentication of the user 50 is.

As described above, in contrast to user authentication using single modality, embodiments can improve user's device access convenience and ensure high security through complex authentication based on a combination of modalities. In addition, embodiments may improve user convenience by maintaining user recognition and authentication performance in various environments by using various modalities.

2 is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 2, an apparatus for performing complex authentication according to an embodiment (hereinafter, referred to as an “authentication apparatus”) performs sole authentication based on a first modality among modalities (210). The authentication apparatus may perform individual authentication of individual modalities and, in the case of failure of single authentication, may perform complex authentication. More specifically, the authentication apparatus may perform sole authentication by determining whether the first characteristic of the first modality satisfies the first condition. The first feature may correspond to, for example, an Euclidean distance or a similarity score determined based on feature vectors extracted from the first modality. Also. The first condition may, for example, correspond to a threshold distance for sole authentication or a threshold score for sole authentication. The first condition for exclusive authentication may be determined differently according to the type of modality. If sole authentication is accepted in step 210, the authentication device may determine that authentication is successful (240).

If the sole authentication fails in step 210, the authentication apparatus determines whether to perform the composite authentication by the combination of the modalities based on the second condition different from the first condition for the single authentication. It is determined whether the satisfaction of the (220). Here, 'composite authentication' means, for example, (fingerprint, iris), (fingerprint, iris), (fingerprint, face), (iris, face), (fingerprint, iris, face) among a plurality of modalities such as (fingerprint, iris, face). It can be understood to perform authentication by various combinations of modalities such as.

In operation 220, the authentication apparatus may determine whether to perform complex authentication on the combination of modalities by the combination. The authentication apparatus may determine whether to perform complex authentication by the combination of the modalities based on whether the combination of the modalities satisfies the second condition. In this case, the second condition may be determined differently for each combination of modalities.

The second condition may be determined based on, for example, a false positive rate of the most secure modality in the combination of modalities. For example, when the combination of modalities includes an iris and a face, the second condition may be determined based on one tenth of a million, which is a false recognition rate of the iris that is more secure than the face. The second condition may require a false recognition rate that is different from the one millionth false recognition rate. Alternatively, when the combination of modalities includes a face and a fingerprint, the second condition may be determined based on a millionth, which is a false recognition rate of the fingerprint having higher security than the face. The second condition may require a mitigated false recognition rate compared to one million false recognition rates.

Alternatively, the second condition may be determined based on, for example, the false recognition rate of the modality having the highest convenience in the combination of the modalities. For example, when the combination of modalities includes a face and a sign, the second condition may be determined based on a half thousand which is a false recognition rate of the face that is more convenient than the sign. The second condition may require a false recognition rate that is different from the one thousandth false recognition rate. The method for determining the first condition and the second condition by the authentication apparatus will be described in detail with reference to FIGS. 3A and 3B below.

The combination of modalities may include, for example, a first modality and a second modality that is different from the first modality. According to an embodiment, the combination of modalities may include three modalities, such as, for example, a first modality, a second modality, and a third modality, or may include more modalities. The number of modalities included in the combination of modalities may be determined based on, for example, the security of complex authentication or the convenience of complex authentication.

In operation 220, the authentication apparatus may determine whether at least one of the first characteristic of the first modality and the second characteristic of the second modality satisfies the second condition. If it satisfies the second condition, the authentication device may decide to perform complex authentication by a combination of modalities. If the second condition is not satisfied at step 220, the authentication device may determine that the authentication has failed (250).

The authentication apparatus performs complex authentication according to the determination of performing complex authentication in step 220 (230). In step 230, the authentication device may perform complex authentication by determining a combined condition of a score based on the first feature and a score based on the second feature. For example, the authentication apparatus may determine whether the condition for the score based on the first feature and the condition for the score based on the second feature are satisfied through a logical operation to satisfy the third condition.

Alternatively, the authentication apparatus may generate the third feature by fusion of the first feature and the second feature, and perform complex authentication based on the third feature. For example, the authentication apparatus may perform complex authentication based on whether the third feature satisfies the third condition for complex authentication. The third condition may be determined according to the false recognition rate that is targeted for the complex authentication.

In operation 230, the authentication apparatus may determine whether a combination of the first feature and the second feature, or the third feature satisfies the third condition, for example, using a pre-learned classifier. A method for the authentication apparatus to perform complex authentication based on the third feature will be described in detail with reference to FIG. 5 below.

3A and 3B are diagrams for describing a method of determining a first condition and a second condition, according to an exemplary embodiment. The first condition may, for example, correspond to a threshold distance for performing sole authentication, and the second condition may, for example, correspond to a threshold distance used for determining whether to perform complex authentication. The threshold distance may be determined by, for example, a verification rate (VR), a false recognition rate (FAR), a false reject rate (FRR), or a combination thereof.

Hereinafter, for convenience of description, a method of determining the first condition and the second condition using a false recognition rate (FAR) will be described, but the method of determining the first condition and the second condition is not necessarily limited thereto. It can be determined by various performance indicators. Further, although the performance indicators for determining the first condition and the second condition are the same, the target scores for determining the threshold corresponding to each of the first condition and the second condition may be different from each other. For example, in response to the first condition, a threshold that satisfies a predetermined false recognition rate FAR may be determined for a raw score. In response to the second condition, a threshold that satisfies a certain false recognition rate FAR may be determined for a score to which additional processing (eg, filtering) is applied to a raw score.

Referring to FIG. 3A, a histogram 310 may display a feature distance according to a first feature of a first modality among data corresponding to a user (same person). Also shown is a histogram 330 showing the feature distances by the first feature of the first modality between data corresponding to the user and data corresponding to others. Here, the feature distance may correspond to a Euclidean distance representing a degree of difference between the face image of the person and the face image of another person. The feature distance may have a smaller value as the similarity between the data to be compared increases and the larger the similarity between the data to be compared may be greater.

According to an embodiment, in addition to the feature distance, the authentication device may be configured to include, for example, the first device by various performance indicators such as normalized cross correlation (NCC), matching score, or similarity score between feature vectors. The condition and the second condition may be determined.

In the graph of FIG. 3A, the X axis may represent a feature distance due to the feature of the first modality, and the Y axis may represent the number of samples corresponding to the feature distance.

The authentication apparatus according to an embodiment may determine the first condition according to whether to emphasize security or convenience of an application. For example, when a user wants to perform a financial institution related application such as a bank or a securities business, the authentication device may determine the first condition so that strict authentication is performed by emphasizing security. Alternatively, when the user wants to unlock to perform a simple function of a mobile device such as a camera or a notepad, the authentication device may emphasize authentication for convenience, omit authentication, or set the first condition so that authentication with a changed standard is performed. You can decide.

For example, in the case of emphasizing security, the authentication device permits a false recognition in the iris modality of the entire area of the histogram 330 representing the characteristic distance by the first characteristic of the iris modality of another person (eg, The characteristic distance of the boundary line 350 for dividing the area of the lower tenth one million) may be set as the first condition. In this case, if the characteristic distance of the iris modality is not satisfied with the false recognition rate of one millionth, the single user authentication fails.

However, since the iris modality does not work well in the outdoor environment with strong UV light, in order to solve the inconvenience of the user, in one embodiment, even if the authentication by the iris modality fails, when the iris modality satisfies a predetermined condition, Complex authentication can be performed with facial modalities that work well in the environment. If the iris modality does not satisfy the characteristic distance threshold of the boundary line 350 for single authentication, but the iris modality satisfies the characteristic distance threshold of the other boundary line 370, the authentication apparatus performs complex authentication by a combination of iris modality and face modality. You can decide to do it. The fact that the iris modality satisfies the feature distance threshold may mean that the feature distance due to the iris modality is less than (or less than) the feature distance threshold.

In this case, the boundary line for the first condition and the boundary line for the second condition may be determined in different score histograms. For example, referring to FIG. 3B, the boundary line 350 for the first condition is set based on the original modality scores, eg, histogram 310 and histogram 330, and for the second condition. The boundary line 370 may be set based on a score that applies additional processing (eg, filtering) to the score, for example, the histogram 320 and the histogram 340.

In one embodiment, even if the iris modality fails to be authenticated alone, if the second condition is satisfied, the user's convenience may be improved while maintaining security by being composed of a combination of elements of complex authentication, that is, modalities. More specifically, when the single authentication fails, the rate of misrecognizing the other person as the user may be reduced by setting the entry condition (for example, the second condition) instead of performing complex authentication immediately. In other words, by performing complex authentication only when the entry condition (eg, the second condition) is passed, it is possible to maintain higher security than when the entry condition (eg, the second condition) is not used. have. In addition, compared to the case of using only the first condition, the rate at which the person is excluded from others is lowered, so that the user's convenience is improved.

4 is a block diagram of an apparatus for performing complex authentication according to an embodiment. Referring to FIG. 4, the authentication apparatus according to an embodiment may include a registration database (DB) 410, a matching unit 420, an entry condition determination unit 430, and an authentication unit 440. Operations of the matching unit 420, the entry condition determining unit 430, and the authenticating unit 440 may be performed by, for example, the processor 1510 of FIG. 15 to be described later.

The registration database 410 may include registration feature vectors for each modalities. The registration database 410 may be provided corresponding to each of the modalities, for example, or may be provided as one integrated database including all the modalities.

The matching unit 420 may call the registration feature vector of the modality corresponding to each of the input first modality input 1 and / or the second modality input 2 from the registration database 410 in real time.

The matching unit 420 extracts the feature vector of the first modality and / or the second modality, registers the feature vector of the first modality and / or the feature vector of the second modality and the registered feature vectors of the modalities stored in the registration database 410. These features can be matched to yield feature distance or similarity scores. The matching unit 420 does not extract the feature vectors of the first modality and the second modality at once, extracts the feature vector of the first modality according to the authentication state of the authenticator 440, and then, if necessary, Feature vectors can be extracted.

The matching unit 420 may transmit the feature distance or the similarity score according to the matching result to the entry condition determiner 430 and the authenticator 440.

For example, if the feature distance between the feature vector of the first modality and the registered feature vector, which is received from the matching unit 420, satisfies the first condition, the authenticator 440 determines that the independent authentication is successful, and then authenticates alone. You can output success. If the single authentication is successful, the authentication device may output the authentication success (accept). If the feature distance between the feature vector of the first modality and the registered feature vector does not satisfy the first condition, the authenticator 440 may determine that the single authentication has failed and output the single authentication failure. The authenticator 440 may transmit the authentication result to the entry condition determiner 430.

The entry condition determiner 430 may determine whether an entry condition for complex authentication is satisfied based on a matching result (for example, single authentication success or single authentication failure) received from the matching unit 420. . When a single authentication failure is transmitted from the matching unit 420, the entry condition determining unit 430 may determine whether an entry condition for complex authentication is satisfied.

The entry condition determiner 430 may determine whether to perform complex authentication based on a combination of modalities, for example, based on a second condition different from the first condition for the single authentication. For example, the matching unit 420 indicates that the feature distance between at least one of the first feature vector of the first modality and the second feature vector of the second modality and the registered feature vector of the modalities stored in the database satisfies the second condition. When the information is received, the entry condition determiner 430 may determine that an entry condition for complex authentication is satisfied. If it is determined that the entry condition for the composite authentication is satisfied, the entry condition determination unit 430 may request the authentication unit 440 to perform the composite authentication based on a combination of modalities. If it is determined that the entry condition for the complex authentication is not satisfied, the authentication device may output an authentication failure.

The authenticator 440 may perform complex authentication based on a condition in which a score based on the first characteristic and a score based on the second characteristic are combined. Alternatively, the authenticator 440 may generate a third feature by fusion of the first feature and the second feature, and perform complex authentication based on the third feature. The authenticator 440 may output a result of performing the complex authentication (for example, a compound authentication success or a compound authentication failure). The authentication device may output authentication success if the compound authentication succeeds, or output authentication failure if the compound authentication fails. The authenticator 440 may perform complex authentication by using a classifier 500 which will be described later with reference to FIG. 5. The process of the authentication unit 440 performing the complex authentication using the third feature will be described in detail with reference to FIG. 5 below.

5 is a diagram for describing an operation of a classifier for performing complex authentication, according to an exemplary embodiment. Referring to FIG. 5, a classifier 500 is shown that performs complex authentication by a third feature fusion of a first feature of a first modality and a second feature of a second modality.

For example, suppose that the 16D feature vectors of the iris image and the 3D feature vectors of the face image are input to the classifier 500.

The classifier 500 may generate a 19-dimensional feature vector by fusion of 16-dimensional feature vectors of iris modality and 3-dimensional feature vectors of face modality. In this case, the 16-dimensional feature vectors of the iris modality may be, for example, the Hamming distance of the iris, the bit count of the iris, the radius of the iris, the shape and color of the iris, and the retinal capillaries. It may be a feature vector corresponding to features such as morphemes. In addition, the three-dimensional feature vector of the face modality may be, for example, a feature vector corresponding to features such as an entire face and a partial face.

The classifier 500 may determine the acceptance or failure of the complex authentication by comparing the 19-dimensional feature vectors with a boundary line (or reference value) separated by the person or the person.

In the lower left of FIG. 5, a diagram illustrating the learning phase of the classifier 500 is shown. In the drawing showing the learning phase, the X axis represents the 16-dimensional feature space of the iris image, and the Y axis represents the three-dimensional feature space of the face image. For convenience of description, a three-dimensional feature space has been described as an example. However, according to an exemplary embodiment, the feature space may be extended to a feature space of 20 dimensions having individual axes for each dimension.

The points indicated in the learning phase may correspond to 19-dimensional vectors generated by fusion of 16-dimensional feature vectors and 3-dimensional feature vectors. The boundary line 510 may correspond to a parameter that the classifier 500 is trained to distinguish from accepting or failing complex authentication by 19-dimensional vectors. The classifier 500 may learn a parameter (or weight) for feature vectors using, for example, a Support Vector Machine (SVM). In this case, the parameter learned by the classifier 500 may be determined according to, for example, the level of the false recognition rate targeted by the authentication apparatus.

In the lower right of FIG. 5, there is shown a diagram illustrating the authentication phase of the classifier 500. In the authentication phase, the boundary line 530 may correspond to a parameter of the classifier 500. Line 530 may correspond to a simplified representation of a 19-dimensional feature vector in two dimensions. When the fused 19-dimensional feature vector is generated, the authentication apparatus may perform complex authentication on 19-dimensional inputs based on the boundary line 530. For example, the authentication apparatus may include, for example, a first input I ₁ , F ₁ located on the left side based on the boundary line 530 as the acceptance of authentication, or a second input I located on the right side based on the boundary line 530. ₂ , F ₂ ) may determine that authentication failed.

6A is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 6A, a process of performing complex authentication on iris data having a first modality and face data having a second modality is illustrated.

The authentication apparatus according to an embodiment may perform complex authentication through steps 610 and 650 of extracting feature information of the modality. In addition, the authentication apparatus may further perform step 630 of determining whether to perform complex authentication based on the security level. Steps 630 and 650 may be performed sequentially or non-sequentially, depending on the embodiment.

In operation 610, when the iris data and the face data are input, the authentication apparatus may extract a feature vector for each of the modalities. The authentication device may calculate an iris score or a face score by matching the extracted feature vectors with registration feature vectors for each modality stored in the registration database 605. In this case, the iris score or the face score may correspond to a feature distance or a similarity score between the feature vector extracted from each of the modalities and the registered feature vector.

In operation 650, the authentication apparatus may determine whether the iris score or the face score satisfies the single authentication condition. If the single authentication condition is satisfied, the authentication device may accept the authentication.

If the iris score or the face score does not satisfy the sole authentication condition, the authentication apparatus may determine whether the face score or the iris score satisfies the predetermined false recognition rate based on the security level in step 630. Specifically, whether the face score alone satisfies the predetermined false recognition rate, whether the iris score alone satisfies the predetermined false recognition rate, or whether both the face score and the iris score satisfies the predetermined false recognition rate is predetermined It can be used to determine whether the false recognition rate is satisfied.

The predetermined false recognition rate may be determined as, for example, a false recognition rate for authentication of facial data or a false recognition rate for authentication of iris data according to the security level. If (face score, iris score) does not satisfy a predetermined false recognition rate in step 630, the authentication device may perform new authentication process shown in FIG. 6A again by acquiring new iris data and / or face data. have.

If (face score, iris score) satisfies a predetermined false recognition rate in step 630, the authentication apparatus determines in step 650 a combination of an iris score and a face score, or a fusion score fusion of an iris score and a face score. It may be determined whether the complex authentication condition is satisfied. If the complex authentication condition is satisfied, the authentication device may accept the authentication.

If the complex authentication condition is not satisfied, the authentication apparatus may acquire new iris data and face data and perform the complex authentication process illustrated in FIG. 6A again.

Although not shown in the figure, the authentication apparatus may determine that authentication has failed by counting a predetermined number of times or a predetermined time when the conditions do not match at step 630 or 650. For example, if the authentication device does not meet the conditions in step 630 or 650, the authentication device may accumulate and store the number of times that authentication fails, and then perform the complex authentication process again. The authentication device may determine that authentication has failed when the accumulated number of failures exceeds a predetermined threshold. Alternatively, the authentication device may accumulate and store time required for the complex authentication process, and may determine that authentication has failed when the accumulated time exceeds a predetermined threshold. If the authentication device does not meet the conditions in step 630 or 650, the authentication device may determine whether the accumulated time exceeds a predetermined threshold. The authentication device may provide feedback indicating the authentication failure to the user according to the determination that the authentication has failed.

6B is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 6B, the authentication apparatus extracts feature information of the modality in step 610. In step 690, the authentication apparatus performs sole authentication and complex authentication. For example, the authentication apparatus primarily performs face-only authentication, performs iris-only authentication secondly, and performs complex authentication of face and iris thirdly. When the authentication apparatus succeeds in the single authentication or the complex authentication in each step, the authentication device determines that the authentication succeeds in the end without performing subordinate authentication.

If the face authentication fails and the iris only authentication fails, the authentication device may determine an entry condition for whether to perform the complex authentication before performing the complex authentication.

The authentication apparatus may repeat the above-described authentication process based on the new input data when the entry condition or the condition for the complex authentication is not satisfied. If the authentication device does not succeed until a predetermined number of times or a predetermined time elapses, the authentication device may finally determine that authentication failed.

7 is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 7, there is shown a complex authentication process that can enhance user convenience and security.

Steps 610 to 650 of FIG. 6A indicate that the features (or feature vectors) corresponding to any combination of modalities (eg, a combination of face modality and iris modality) allow entry into complex authentication. Unlike judging whether a condition is satisfied and whether a condition for complex authentication of the fused feature vectors (acceptance) is satisfied, the process of steps 710 to 750 shown in FIG. There is a difference in determining whether the combi condition and the f (combi) condition are satisfied for various combinations. For convenience of description below, a condition for entering into the complex authentication is referred to as a 'combi condition', and a condition for complex authentication (acceptance) of the fused feature vectors is referred to as 'f (combi) condition'. The combi condition and / or f (combi) condition may be determined differently for each combination of corresponding modalities.

In FIG. 7, when the security level of the complex authentication through each combination of modalities is very high or the security level required by the complex authentication is not high, the condition determination for various authentication methods is added or omitted for each combination of modalities. can do.

When data of various modalities are input, the authentication apparatus may perform authentication by various combinations of modalities. For example, the authentication apparatus may perform complex authentication through M authentication methods including one or more modality combinations among N modalities. M may represent the number of authentication methods by single authentication or complex authentication.

However, prior to performing the composite authentication by each modality combination, the authentication apparatus determines whether or not the composite authentication is performed by each modality score ( s ₁ , s ₂ ,?, S _N ) and f (combi) condition and / or This can be determined by whether or not the combi condition is met. The f (combi) condition and / or combi condition may be in the form of, for example, a predetermined score above, below, below, above or a combination thereof. In this case, each modality score ( s ₁ , s ₂ ,?, S _N ) may correspond to a feature distance or similarity score between the feature vector corresponding to each modality and the registered feature vectors for each modality stored in the registration database 705. have. The condition for the modality score may be used as an entry condition for determining whether to enter the composite authentication.

The authentication apparatus may not perform authentication when all N modality scores for complex authentication are completed, but may perform complex authentication for each modality combination as each score for each modality is input. The authentication apparatus can determine a criterion for determining whether to enter such complex authentication by a secure modality.

The authentication apparatus determines whether the fused score of the combination of the first modalities satisfies the f ₁ (combi ₁ ) condition based on each modality score s ₁ , s ₂ ,?, S _N in step 750. Can be determined. In this case, the combination of the first modalities may correspond to one modality or may correspond to a combination of two or more modalities. In addition, the f ₁ (combi ₁ ) condition corresponds to a single authentication condition when the combination of the first modalities is configured as one modality, and the complex authentication condition when the combination of the first modalities is configured as a combination of two or more modalities. It may correspond to.

The authentication device determines whether the scores corresponding to the combination of the second modalities satisfy the combi ₂ condition when the fused score of the first combination of modalities does not satisfy the f ₁ (combi ₁ ) condition and / or the second It may be determined whether the fused feature vectors corresponding to the combination of the first modalities satisfy the f ₂ (combi ₂ ) condition. As described above, the process of determining the combi condition and the f (combi) condition for various authentication methods according to the security level of the complex authentication may be added for each combination of modalities. In addition, even when the security level of the complex authentication is low or the demand for convenience is high, the process of determining the combi condition and the f (combi) condition for various authentication methods may be added or omitted for each combination of modalities. .

8 is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 8, in operation 810, the authentication apparatus determines whether to perform complex authentication based on at least one of the first feature of the first modality and the second feature of the second modality. For example, the authentication apparatus may determine whether the first characteristic satisfies the first condition for single authentication. If the first feature does not satisfy the first condition, the authentication device may determine whether at least one of the first feature and the second feature satisfies a second condition different from the first condition. If the second condition is satisfied, the authentication apparatus may determine to perform complex authentication by a combination of the first modality and the second modalities. The second condition may be determined based on, for example, the false recognition rate (FAR) of the first modality or the second modality.

The authentication apparatus generates a third feature by combining the first feature and the second feature according to whether to perform the composite authentication (820).

The authentication device performs complex authentication based on the third feature (830). If the third characteristic satisfies the third condition for the composite authentication, the authentication device may perform the composite authentication.

9 is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 9, the authentication apparatus according to an embodiment determines whether a first entry condition corresponding to a first combination of modalities is satisfied (910). For example, the authentication apparatus may determine whether the first entry condition is satisfied based on whether the first combination of modalities satisfies a second condition different from the first condition for the single authentication. The first combination of modalities may comprise a first modality and a second modality that is different from the first modality. The authentication apparatus may determine whether at least one of the first characteristic of the first modality and the second characteristic of the second modality satisfies the second condition. When at least one of the first feature and the second feature satisfies the second condition, the authentication device may determine that the first entry condition is satisfied.

The authentication apparatus performs authentication by the first combination according to whether the first entry condition is satisfied (920).

If the authentication by the first combination fails, the authentication apparatus determines whether the second entry condition corresponding to the second combination of the modalities is satisfied (930). In this case, the first entry condition and the second entry condition may be determined differently for each combination of modalities.

The authentication apparatus performs authentication by the second combination in accordance with whether the second entry condition is satisfied (940).

10 is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 10, a complex authentication process according to an embodiment is illustrated. Since steps 1010 to 1050 of FIG. 10 are similar to steps 710 to 750 of FIG. 7, only operations different from those of FIG. 7 will be described below.

When data of several modalities are input, the authentication apparatus differs from FIG. 7 based on the scores s ₁ , s ₂ ,?, S _N for each modality in step 1030, and the combination of the first modalities is different. It may be determined whether the combi ₁ condition is satisfied. If the combination of the first modalities does not satisfy the combi ₁ condition, the authentication apparatus may determine whether the combination of the second modalities satisfies the combi ₂ condition. If each of the combinations of modalities does not satisfy the combi condition, the authentication device may compare the new modalities with the combi condition until the combi condition is satisfied.

For example, if combi _I condition = (M ₁ , M ₂ , M ₃ ) in step 1030, the entry conditions corresponding to the combi _I combination are, for example, M ₁ condition, M ₂ condition, M ₃ condition. Or various combinations thereof.

If the combination of the first modalities satisfies the combi ₁ condition, the authentication device may determine whether the fused score of the combination of the first modalities satisfies the f (combi ₁ ) condition in step 1050. In this case, the scores for each modality may be used to determine whether to enter the complex authentication. When the fused score of the combination of the first modalities does not satisfy the f (combi ₁ ) condition, the authentication apparatus may determine whether the combination of the second modalities satisfies the combi ₂ condition. If the fused score of the combination of first modalities satisfies the f (combi ₁ ) condition, the authentication device may accept the composite authentication.

11 is a flowchart illustrating a method of performing complex authentication according to an embodiment. Referring to FIG. 11, a process of performing complex authentication using an infrared image and a color face image when it is input is shown. For convenience of description, an application using a color image will be described. However, embodiments are not limited to this scenario and may be applied to an application using a black and white image or a depth image (or 3D image).

The authentication apparatus according to an embodiment corresponds to a plurality of features (feature vectors) of the iris modality acquired through the infrared image and the face modality acquired through the color face image through the feature extraction process in step 1110. Matching of a plurality of features (feature vectors) and a registered feature vector for each modality registered in a registration database may be performed. The authentication apparatus determines, via step 1110, scores i ₁ , i ₂ ,..., I _N corresponding to the plurality of features of the iris modality and scores corresponding to the plurality of features of the face modality f ₁ ,. f ₂ , .., f _O ) can be calculated. In this case, the plural features of the iris modality may include, for example, the Hamming distance of the iris, the bit count of the iris, the radius of the iris, the shape and color of the iris, the morphology of the retinal capillaries, and the like. It may include. Also, the plurality of features corresponding to the face modality may include a partial face, an entire face, and the like.

In operation 1130, the authentication apparatus determines that the combination of the first modalities is combi based on the scores for each modality i ₁ , i ₂ , .., i _N, f ₁ , f ₂ , .., f _{O. 1 It} can be determined whether the condition is satisfied. The combination of the first modalities may be, for example, a combination of the hamming distance of the iris and the partial face. If the combination of the first modalities does not satisfy the combi ₁ condition, the authentication device may determine whether the new second modalities combination satisfies the combi ₂ condition. The combination of the second modalities can be, for example, a combination of the morphology of the retinal capillary and the entire face.

If the combination of the second modalities satisfies the combi ₂ condition, the authentication apparatus may determine whether the fused score of the combination of the second modalities satisfies the f ₂ (combi ₂ ) condition. If the fused score of the combination of the second modalities satisfies the f ₂ (combi ₂ ) condition, the authentication device may accept the complex authentication by the combination of the second modalities.

According to an embodiment, the authentication apparatus may be configured such that the combi condition is always true or configured to be false. For example, the authentication apparatus is configured such that the combi ₁ condition corresponding to the face score f ₁ is always true among the scores corresponding to the face modality, so that independent authentication of the face score is always performed regardless of whether the combi ₁ condition is satisfied. You can do that. Or, according to example authentication apparatus face score by configuring such that the combi ₁ Conditions for the f ₁ is always false regardless of the happy with the combi ₁ condition always exclusive authentication of the face score is also such that the skip (skip) have.

In addition to the above-described embodiments, the authentication apparatus may perform authentication considering both user convenience and security by various combinations of combi conditions and f (combi) conditions. For example, the f (combi) conditions may be set to perform a single authentication (s) based on individual features and a composite authentication (s) combining a plurality of features in a predetermined order. The combi condition corresponding to each f (combi) condition may be set based on a condition different from a single authentication condition used for the f (combi) condition. In some cases, the combi condition may be configured to be true or false. have.

12 illustrates a user interface according to an exemplary embodiment. Referring to FIG. 12, when a user uses biometrics authentication, a type of modality used for unlocking may be displayed on a lock screen of a smart phone. For example, when the modality used for unlocking is a face modality, a face icon may be displayed on the lock screen as in the first screen 1210. Alternatively, when the modality used for unlocking is an iris modality, the iris icon may be displayed on the lock screen as shown in the second screen 1230. Alternatively, when the modality used for unlocking is a combination of face modality and iris modality, the face icon of the first screen 1210 and the iris icon of the second screen 1230 may be alternately displayed on the lock screen.

According to an embodiment, the first preview screen for input of the face image and the second preview screen for input of the iris image may be selectively displayed. Whether to display the first preview screen and whether to display the second preview screen may be variously set. For example, the first preview screen may not be displayed when using face authentication alone, but the second preview screen may be displayed when using alone authentication of iris modality. In addition, when the combined authentication of the face modality and the iris modality is used, both the first preview screen and the second preview screen may not be displayed.

Although face modality and iris modality have been described with reference to FIG. 12, embodiments may be modified or extended to use other modalities such as fingerprint modality.

13A and 13B illustrate a process of registering biometrics information, according to an exemplary embodiment. Referring to FIG. 13A, a user may register a face image using an image sensor attached to a smart phone (1310). In addition, the user may register the iris image using an infrared sensor attached to the smart phone (1330). The user who wears the glasses may take off the glasses to register the iris image.

According to an embodiment, a type of modality used for authentication for a specific application or function may be selected by a user. For example, the user can change the type of modality used for unlocking in the settings of the smartphone.

Referring to FIG. 13B, a screen for selecting a type of modality used for authentication through biometric information is shown. For example, the selection options include a first option 1370 corresponding to single authentication using face modality, a second option 1390 corresponding to single authentication using iris modality, and a combination authentication combining face modality and iris modality. It may include a third option 1350 corresponding to the. When the first option 1370 is selected, face modality is used for biometric authentication. When the second option 1390 is selected, iris modality is used for biometric authentication. When the third option 1350 is selected, biometric authentication is performed. Face modality and iris modality may be used together. The first option 1370, the second option 1390, and the third option 1350 may be selected exclusively from each other. For example, if either option is selected, the previously selected option may be released automatically.

In some embodiments, the third option 1350 corresponding to the complex authentication may not be separately displayed. In this case, the first option 1370 and the second option 1390 may be simultaneously selected. If the first option 1370 and the second option 1390 are selected at the same time, complex authentication combining face modality and iris modality may be performed for biometric authentication.

In FIG. 13B, the case of using the face modality and the iris modality for biometric authentication has been described. However, embodiments are not limited thereto, and may be modified to use another modality or to use three or more modalities.

When the type of modality is set, the authentication apparatus may determine whether registration data of the modality of the corresponding type is stored. If the registration data of the modality of the corresponding type is stored, the authentication device may immediately change the type of the modality for authentication without additional registration process. If the registration data of the modality of the corresponding type is not stored, the authentication device may additionally receive the registration data of the modality of the corresponding type.

For example, in a situation where there is no pre-stored registration data, a complex authentication of face modality and iris modality may be set. In this case, the authentication apparatus may acquire a face image and an iris image, and store registration data of face modality and registration data of iris modality.

As another example, a complex authentication of face modality and iris modality may be set in a situation where only registration data of face modality is stored. In this case, the authentication apparatus may acquire only the iris image and additionally store registration data of the iris modality.

If the user changes the setting from complex authentication to sole authentication of iris modality, the authentication apparatus determines whether registration data of iris modality is stored. In this case, since the registration data of the iris modality is stored, the authentication apparatus can change the setting to use the sole authentication of the iris modality without additional registration process.

According to an embodiment, registration data may be managed for each modality. For example, the user may selectively delete registration data having a specific modality among previously stored registration data. In this case, the authentication device may change the authentication type based on the currently set authentication type and the remaining registration data. For example, the registration data of the face modality and the registration data of the iris modality are stored, and the currently set authentication type may be complex authentication. If the registration data of the iris modality is deleted, the authentication device may change the authentication type to the single authentication of the face modality. In addition, when all the registration data of the bio-modality is deleted, the authentication type may be changed in such a manner that authentication is performed through other means such as a pattern or a password other than biometric authentication.

The above-described embodiments are applicable to devices other than smart phones, and also to other modalities other than face modality and / or iris modality.

14 is a diagram illustrating an operation of performing complex authentication by using correlation between different modalities according to an embodiment. The authentication apparatus may utilize a correlation between the first image and the second image in the process of acquiring the first image for the first modality and the second image for the second modality.

For example, when the first modality is the face modality and the second modality is the iris modality, the eye region in the first image may be expected to have a high correlation with the second image. In this case, when the single authentication using the first image fails, the authentication apparatus may determine whether an eye region within the first image is detected before performing the single authentication or the composite authentication using the second image. More specifically, the angle of view of the image sensor for the first image may be greater than the angle of view of the infrared sensor for the second image. In this case, if the eye region is not detected in the first image, the probability that the iris information is not included in the second image is high. If the iris information is not included in the second image, since the single authentication or the composite authentication using the second image cannot be performed, the authentication apparatus uses the first image by acquiring the first image again instead of acquiring the second image. You can retry exclusive authentication.

14 mainly describes, but is not limited to, a color image 1410 as a first image and an infrared image 1420 as a second image. The first image may be an infrared image, the second image may be a color image, and the first image and the second image may both be infrared images or both color images. In addition, the first image may be a depth image, and the second image may be a color image. In other words, the first image may be one of a color image, an infrared image, and a depth image, and the second image may also be one of a color image, an infrared image, and a depth image.

According to an embodiment, the authentication apparatus may acquire the color image 1410 as the first image. As illustrated in FIG. 14, the first image sensor of the authentication apparatus may generate a color image 1410 by photographing a face of a person as an object.

When the single authentication using the color image 1410 fails, the authentication apparatus may identify a landmark point of the object with respect to the color image 1410. For example, the authentication apparatus may extract a facial feature point of a person from the color image 1410 based on the object model. According to an embodiment, the authentication apparatus may check whether a landmark point is identified in the predetermined area 1411. The landmark point may be represented by a dot in FIG. 14. The predetermined area 1411 in the color image 1410 may be an area corresponding to the viewing angle of the infrared image 1420. When no landmark is detected in the predetermined area 1411 in the color image 1410, the authentication apparatus may predict that the iris information is not included in the infrared image 1420.

15 is a block diagram of an apparatus for performing complex authentication according to an embodiment. Referring to FIG. 15, an authentication apparatus 1500 according to an embodiment includes a processor 1510. The authentication device 1500 may further include a memory 1530, a communication interface 1550, and sensors 1570. The processor 1510, the memory 1530, the communication interface 1550, and the sensors 1570 may communicate with each other via the communication bus 1505.

The processor 1510 performs sole authentication based on the first modality among the modalities. When the sole authentication fails, the processor 1510 determines whether to perform the complex authentication based on a combination of modalities, based on a second condition different from the first condition for the single authentication. The processor 1510 performs complex authentication according to the determination of whether to perform complex authentication.

The memory 1530 may include a registration database including feature vectors for each modality. The registration database may correspond to the registration database 410, for example. The memory 1530 may be a volatile memory or a nonvolatile memory.

The communication interface 1550 may output a single authentication result and / or a result of performing a complex authentication to the display device (not shown) of the authentication device 1500 or the outside of the authentication device 1500. The communication interface 1550 may receive at least one modality from the outside of the authentication apparatus 1500, or may receive information about an environment in which the modalities have been collected from the user.

The sensors 1570 may include, for example, an image sensor, an infrared sensor, a fingerprint sensor, a voice recognition sensor, and the like. Sensors can collect various modalities.

According to an embodiment, the processor 1510 determines whether to perform complex authentication based on at least one of the first feature of the first modality and the second feature of the second modality. The processor 1510 generates a third feature by combining the first feature and the second feature according to whether or not the complex authentication is performed, and performs the complex authentication based on the third feature.

Alternatively, the processor 1510 may determine whether the first entry condition corresponding to the first combination of modalities is satisfied, and perform authentication by the first combination according to whether the first entry condition is satisfied. When the authentication by the first combination fails, the processor 1510 determines whether the second entry condition corresponding to the second combination of modalities is satisfied, and performs authentication by the second combination according to whether the second entry condition is satisfied. Perform.

In addition, the processor 1510 may perform an algorithm corresponding to at least one method or at least one method described above with reference to FIGS. 1 through 11. The processor 1510 may execute a program and control the authentication apparatus 1500. Program code executed by the processor 1510 may be stored in the memory 1230. The authentication device 1500 may be connected to an external device (eg, a personal computer or a network) through an input / output device (not shown), and may exchange data. The authentication device 1500 may include various electronic systems such as a smart television, a smart phone, a smart car, and the like.

The embodiments described above may be implemented as hardware components, software components, and / or combinations of hardware components and software components. For example, the devices, methods, and components described in the embodiments may include, for example, processors, controllers, arithmetic logic units (ALUs), digital signal processors, microcomputers, field programmable gates (FPGAs). It may be implemented using one or more general purpose or special purpose computers, such as an array, a programmable logic unit (PLU), a microprocessor, or any other device capable of executing and responding to instructions. The processing device may execute an operating system (OS) and one or more software applications running on the operating system. The processing device may also access, store, manipulate, process, and generate data in response to the execution of the software. For convenience of explanation, one processing device may be described as being used, but one of ordinary skill in the art will appreciate that the processing device includes a plurality of processing elements and / or a plurality of types of processing elements. It can be seen that it may include. For example, the processing device may include a plurality of processors or one processor and one controller. In addition, other processing configurations are possible, such as parallel processors.

The software may include a computer program, code, instructions, or a combination of one or more of the above, and configure the processing device to operate as desired, or process it independently or collectively. You can command the device. Software and / or data may be any type of machine, component, physical device, virtual equipment, computer storage medium or device in order to be interpreted by or to provide instructions or data to the processing device. Or may be permanently or temporarily embodied in a signal wave to be transmitted. The software may be distributed over networked computer systems so that they may be stored or executed in a distributed manner. Software and data may be stored on one or more computer readable recording media.

The method according to the embodiment may be embodied in the form of program instructions that can be executed by various computer means and recorded in a computer readable medium. The computer readable medium may include program instructions, data files, data structures, etc. alone or in combination. The program instructions recorded on the media may be those specially designed and constructed for the purposes of the embodiments, or they may be of the kind well-known and available to those having skill in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks, and magnetic tape, optical media such as CD-ROMs, DVDs, and magnetic disks, such as floppy disks. Magneto-optical media, and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device described above may be configured to operate as one or more software modules to perform the operations of the embodiments, and vice versa.

Although the embodiments have been described with reference to the accompanying drawings as described above, various modifications and variations are possible to those skilled in the art from the above description. For example, the described techniques may be performed in a different order than the described method, and / or components of the described systems, structures, devices, circuits, etc. may be combined or combined in a different form than the described method, or other components. Or even if replaced or substituted by equivalents, an appropriate result can be achieved. Therefore, other implementations, other embodiments, and equivalents to the claims are within the scope of the claims that follow.

Claims (30)

Performing sole authentication based on the first one of the modalities;
Determining whether to perform complex authentication based on a combination of the modalities when the single authentication fails; And
Performing the composite authentication according to the determination of whether to perform the composite authentication;
Including, the method of performing a composite authentication. The method of claim 1,
Determining whether to perform the composite authentication is
Determining whether a second condition for the composite authentication, which is distinguished from the first condition for the single authentication, is satisfied
Including, the method of performing a composite authentication. The method of claim 1,
Performing the exclusive authentication is
Determining whether a first characteristic of the first modality satisfies a first condition for the single authentication
Including, the method of performing a composite authentication. The method of claim 1,
Determining whether to perform the composite authentication is
Determining whether a first characteristic of the first modality satisfies a second condition different from the first condition for the single authentication
Including, the method of performing a composite authentication. The method of claim 1,
The modalities are
Includes a first modality and a second modality different from the first modality,
Determining whether to perform the composite authentication is
Determine whether the first feature of the first modality, the second feature of the second modality, or a combination of the first and second features satisfy a second condition different from the first condition for the single authentication Steps to
Including, the method of performing a composite authentication. The method of claim 1,
Performing the composite authentication is
Generating a third feature by fusion of a first feature of the first modality and a second feature of a second modality included in the modalities, in accordance with the determination of performing composite authentication; And
Based on the third feature, performing the composite authentication
Including, the method of performing a composite authentication. The method of claim 6,
Performing the composite authentication is
Determining whether the third feature satisfies a third condition for the complex authentication
Including, the method of performing a composite authentication. The method of claim 2,
The second condition is
And differently determined for each combination of the modalities. The method of claim 2,
The second condition is
Determined based on a False Acceptance Rate (FAR) of the highest security modalities among the modalities. How to perform complex authentication. The method of claim 2,
The second condition is
Wherein the convenience among the modalities is determined based on a false positive rate of the highest modality. The method of claim 1,
The combination of modalities is
And determined based on the security of the complex authentication or the convenience of the complex authentication. The method of claim 1,
The modalities are
Performing complex authentication, including any one or a combination of a face image, a fingerprint image, an iris image, a vein image, a palm image, a sign, a voice, a gait, a DNA structure of a user Way. Determining whether to perform complex authentication based on at least one of the first characteristic of the first modality and the second characteristic of the second modality;
Performing composite authentication based on the first feature and the second feature, according to whether to perform the composite authentication;
Including, the method of performing a composite authentication. The method of claim 13,
Performing the composite authentication is
Generating a third feature by combining the first feature and the second feature; And
Based on the third feature, performing the composite authentication
Including, the method of performing a composite authentication. The method of claim 13,
Determining whether to perform the composite authentication is
Determining whether the first feature, the second feature, or the combination of the first feature and the second feature satisfies a second condition different from the first condition for the sole authentication; And
Determining to perform complex authentication by a combination of the first modality and the second modality when the second condition is satisfied.
Including, the method of performing a composite authentication. The method of claim 15,
The second condition is
Determined based on a first false recognition rate (FAR) of the first modality, a second false recognition rate of the second modality, or a combination of the first false recognition rate and the second false recognition rate. How to perform complex authentication. The method of claim 14,
Performing the composite authentication is
Determining whether the third feature satisfies a third condition for the complex authentication
Including, the method of performing a composite authentication. Determining whether a first entry condition corresponding to the first combination of modalities is satisfied;
Performing authentication by the first combination according to whether the first entry condition is satisfied;
If authentication by the first combination fails, determining whether a second entry condition corresponding to the second combination of modalities is satisfied; And
Performing authentication by the second combination according to whether the second entry condition is satisfied;
Including, the method of performing a composite authentication. The method of claim 18,
The first entry condition and the second entry condition are
And differently determined for each combination of the modalities. The method of claim 18,
Determining whether the first entry condition is satisfied
Determining whether the first entry condition is satisfied based on whether the first combination of modalities satisfies a second condition different from the first condition for the single authentication
Including, the method of performing a composite authentication. The method of claim 20,
The first combination of modalities
A first modality and a second modality different from the first modality,
Determining whether the first entry condition is satisfied
Determining whether at least one of the first characteristic of the first modality and the second characteristic of the second modality satisfies the second condition; And
Determining that the first entry condition is satisfied when the second condition is satisfied;
Including, the method of performing a composite authentication. Performing sole authentication based on the first one of the modalities; And
If the single authentication fails, performing a composite authentication based on a combination of the modalities
Including, the method of performing a composite authentication. The method of claim 22,
If the sole authentication fails,
Prior to performing the complex authentication, performing a single authentication based on a second modality among the modalities
Further comprising, the method of performing a composite authentication. The method of claim 22,
Performing the composite authentication is
Generating a third feature by combining the first feature of the first modality and the second feature of the second modalities; And
Performing the composite authentication based on the third feature
Including, the method of performing a composite authentication. In the biometric authentication method for authenticating a user by using different first biomodality and second biomodality,
Determining whether the biometric information of the user satisfies any one of a first condition in which the feature of the first biomodality is reflected and a second condition in which the feature of the second biomodality is reflected;
Determining whether the biometric information of the user satisfies a complex condition in which the feature of the first biomodality and the feature of the second modality are reflected; And
Determining that the composite authentication is successful when the biometric information of the user satisfies any one of the first condition and the second condition and satisfies the compound condition.
Including, biometric authentication method. The method of claim 25,
The complex condition is a biometric authentication method in which a feature in which the feature of the first bio-modality and the feature of the second bio-modality is fused are reflected. The method of claim 25,
Wherein said composite condition is a condition in which a score based on a characteristic of said first biomodality and a score based on a characteristic of said second biomodality are combined. The method of claim 25,
Determining whether or not the complex condition is satisfied
Generating a third feature by fusion of the feature of the first bio-modality and the feature of the second bio-modality; And
Determining whether the complex condition with the third characteristic is satisfied;
Including, biometric authentication method. The method of claim 25,
Determining whether or not the complex condition is satisfied
Generating a third score by combining a first score based on the feature of the first biomodality and a second score based on the feature of the second biomodality; And
Determining whether the complex condition based on the third score is satisfied
Including, biometric authentication method. The method of claim 25,
And wherein the first biomodality is an iris modality and the second biomodality is a face modality.

Images