KR101208678B1 - Incremental personal autentication system and method using multi bio-data - Google Patents

Abstract

The present invention relates to personal authentication. More specifically, after fusion of facial image and voice feature information of a subject, the feature of visual and auditory convergence information is extracted with low eigenvectors and eigenvalues. The present invention relates to a gradual personal authentication system and method using multiple biometric information to authenticate a subject.

Description

Progressive personal authentication system and method using multiple biometric information {INCREMENTAL PERSONAL AUTENTICATION SYSTEM AND METHOD USING MULTI BIO-DATA}

The present invention relates to personal authentication, and more specifically, converges a subject's face image and voice feature information, converts visual and audio information into characteristic data of a unique vector, and then gradually learns new personal authentication information. It relates to a gradual personal authentication system and method using multiple biometric information to authenticate the subject.

Recently, interest and research on system security have been actively conducted.

Accordingly, interest in personal authentication is increasing day by day.

Conventionally, a method of inputting a password based on a keyboard is commonly used. The password input method is a method in which a user inputs a password by using a keyboard or a combination of numbers and letters in order to perform personal authentication.

However, such a password method has a problem that easily guessed numbers such as a person's birthday or a phone number are often used, and thus easily guessed and easily exposed by a means such as a camera eavesdropping by another person.

For this reason, a method of authenticating an individual using biometric information such as fingerprints, iris recognitions, and facial contours has emerged for strong security.

Biological information is unique because it varies from person to person.

Among these, personal authentication methods using face recognition have been widely used in recent years. However, people have a problem that the shape of the face changes with age and the shape also changes with weight.

In addition, there is a problem in that it is not easily recognized by constraints such as the shape of the face, facial expressions, makeup, etc. according to the light conditions.

In addition, there is a problem that accurate personal authentication is difficult because visual and auditory information are easily distorted in a complicated and noisy situation.

Therefore, in addition to personal authentication through face recognition, other means for improving the accuracy of personal authentication are urgently required.

The present invention has been made to solve the problems of the prior art as described above, the object of the present invention is to accurately authenticate the subject, in particular to significantly reduce the amount of data in performing the authentication, and gradually It is to provide a progressive personal authentication system and method using multiple biometric information for accurate authentication.

To this end, a gradual personal authentication system using multiple biometric information according to the present invention comprises: a collecting unit for collecting video information of a subject and voice information of the subject, extracting a face image from the video information, and voice from the voice information. An extraction unit for extracting feature information, and converting the high-dimensional data generated by fusing the face image and the voice feature information into low-dimensional data including representative components, obtaining a vector of the face image and the voice feature information, A conversion unit for gradually extracting only the newly input data of the face image and the voice feature information, and a determination unit for determining the subject by the information separated by the conversion unit.

The facial image is information for authentication of the subject, and extracts a facial region using a wavelet-based directional contour information (AdaBoost) algorithm on the image information.

The voice characteristic information is information for authentication of the subject, and is extracted to the voice information by a Mel-frequency cepstral coefficients (MFCC) method.

The transform unit uses incremental learning principle component analysis (IPCA), which enables progressive learning to accommodate new input data that has not been learned.

The determination unit uses the multi-biometric information, characterized in that to determine the subject by applying the RAN-LTM algorithm.

In addition, a progressive personal authentication method using multiple biometric information according to the present invention for this purpose, information collection step of collecting the subject image information and the subject voice information, extracting a facial image from the image information, An information extraction step of extracting and fusing voice feature information, a unique vector of visual / audio fusion information while converting high-dimensional data generated by fusing at least one or more of the face image and the voice feature information into low-dimensional data including representative components An information transformation step of extracting features of the information, a low dimensional extraction step of gradually extracting features of the visual and auditory data by adding or rotating the axes of the eigenvector representative components, and the visual and auditory fusion information feature extraction steps And a subject determining step of discriminating the subject by the information.

The representative component is a vector value or an eigen value of the low dimensional data.

The speech feature information is information for recognition of the subject, and is extracted by the Mel-frequency cepstral coefficients (MFCC) method.

) 과정, 상기 음성정보일 경우, 멜 로그 파워의 리스트에DCT(discrete cosine transform)를 적용하는 DCT과정, 및 상기 DCT 과정에서 추출된 정보의 상관관계를 제거하기 위해 Cepstral 변환하는 Cepstral filtering과정을 포함한다.The MFCC is a short time fourier transform (STFT) process for applying Fourier transform to the speech information, a mel-scale filter bank process for mapping power of a spectrum included on a mel scale using a triangle overlapping window, and a mel frequency. Apply a log of power to each of the Logs (

) Process, the DCT process of applying a discrete cosine transform (DCT) to the list of the melodic power, and the Cepstral filtering process of performing Cepstral transformation to remove the correlation of information extracted from the DCT process. do.

The low dimensional data extraction step uses an incremental learning principle component analysis (IPCA) capable of incremental learning that can accommodate new untrained input data.

According to various embodiments of the present disclosure, the use of multiple biometric information increases the accuracy of personal authentication.

In addition, according to various embodiments of the present disclosure, by significantly reducing the amount of data input for personal authentication, the load on the system may be reduced, and an efficient authentication system may be established at low cost.

Therefore, according to various embodiments of the present disclosure, the subject may be recognized in a real-life environment such as a security system, an access control system, an ePassport, and thus may be easily applied to the entire industry.

1 is a block diagram illustrating a gradual personal authentication system using multiple biometric information according to an embodiment of the present invention.
Figure 2 is a block diagram showing the operation of the extraction unit of the gradual personal authentication system using multiple biometric information according to an embodiment of the present invention.
3 is a block diagram showing a conversion unit of a gradual personal authentication system using multiple biometric information according to an embodiment of the present invention.
4 is a block diagram showing a determination unit of a gradual personal authentication system using multiple biometric information according to an embodiment of the present invention.
5 is a flowchart illustrating a gradual personal authentication method using multiple biometric information according to an embodiment of the present invention.
6 is a flowchart illustrating a process of extracting a facial image according to another embodiment of the present invention.
7 is an exemplary view showing an example of a facial image extracted by the method of FIG.
8 is a flowchart showing the application of the MLCC algorithm to extract the voice feature information according to another embodiment of the present invention.
9 is an exemplary view illustrating an information conversion step and an information separation step according to another embodiment of the present invention.
10 is a graph showing a result of authenticating a subject using a face image of a gradual personal authentication system using multiple biometric information according to an embodiment of the present invention.
11 is an authentication result of a subject of a personal authentication system using multiple biometric information (visual and audio fusion information) in an environment where Gaussian noise of an image is added step by step from 0.1 to 1.0, according to an embodiment of the present invention; A graph showing a comparison of the subject's authentication results using a single biometric information (visual information).

Hereinafter, a gradual personal authentication system and method using multiple biometric information according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

The same reference numerals are used for the same members in FIGS. 1 to 11.

In describing the present invention, visual / audio information, facial images, and voice feature information are used in the same sense.

The basic principle of the present invention is to perform personal authentication by separating the data by adding the axis of the eigenvector representative component extracted by converting the facial image and the voice feature information of the subject into low-dimensional data or rotating by a predetermined angle.

In the following description of the present invention, detailed description of known functions and configurations incorporated herein will be omitted when it may make the subject matter of the present invention rather unclear.

1 is a block diagram illustrating a gradual personal authentication system using multiple biometric information according to an embodiment of the present invention.

Referring to FIG. 1, a gradual personal authentication system 100 using multiple biometric information according to an embodiment of the present invention includes a collecting unit 110 and a collecting unit 110 for collecting image information of a subject and voice information of a subject. Facial information is extracted from the image information collected from the image), voice feature information is extracted from the voice information extractor 120, and the high-dimensional data generated by fusion in the extraction unit 120 is converted to low-dimensional data And a determining unit 140 for extracting representative components of the feature vector of the auditory convergence information to separate the data, and a determining unit 140 for determining the subject by the information converted by the converting unit 130.

Referring to the operation of a gradual personal authentication system using multiple biometric information according to an embodiment of the present invention configured as shown in Figure 1 as follows.

First, the collecting unit 110 collects the video information and the audio information of the subject. For example, an image including the subject's face is obtained by using a camera, and the subject's voice is recorded by using a microphone.

Then, the extractor 120 extracts a facial image from the image collected by the collector 110, and extracts and fuses voice feature information from the voice information. For example, the background image other than the face image for authentication of the subject is excluded from the collected image information.

On the other hand, the voice information may be divided into three types: linguistic meaning, subject information, and channel information of sound. Here, only the voice feature information necessary for authentication of the subject is extracted from the three types of information, and then fused with the extracted facial image.

The transform unit 130 extracts a representative component by applying a Principal Component Analysis (PCA) algorithm. Here, the representative component may be an eigen-vector or an eigen-value as a concept of a separation line for separating data. That is, after applying the PCA algorithm to extract the unique components represented from a certain number of personal information or more, apply the IPCA (Incremental Principal Component Analysis) algorithm.

Here, the IPCA algorithm is a gradual PCA method that separates new data by adding or rotating axes of eigen vectors and updates only the separated new data.

Finally, the determination unit 140 performs personal authentication by discriminating the extracted video and audio using a resource allocating network with long-term memory (RNA-LTM) model.

In this way, by authenticating the individual by using the biometric information of the subject's video and audio, it is possible to secure higher reliability, and the data calculation speed can be increased by converting the high-dimensional data of the biometric information into low-dimensional data. .

2 is a block diagram illustrating an operation of the extractor 120 of the gradual personal authentication system 100 using multiple biometric information according to an embodiment of the present invention.

Referring to FIG. 2, the extractor 120 of the gradual personal authentication system 100 using multiple biometric information according to an embodiment of the present invention may include a face image extracting unit 121 for extracting a face image from an image of a subject; And voice feature extraction means 122 for extracting voice feature information from the voice of the subject.

First, the face image extracting unit 121 uses a face detection method based on a selective attention model based on the visual information processing mechanism of the human brain.

The facial image extracting unit 121 according to an embodiment of the present invention mimics a function of extracting contour information and complementary color information of the human retina, and thus, in the protrusion map, basic input of color (RG) information for contour and complementary color information. Use as information. In addition, intensity information is used as an input of the protrusion map to reflect the hypothesis as a psychological element of human. In order to effectively detect the subject's face, the subject's region is selected by storing the subject's face color information and reflecting the weight to the facial region in real time. In the face candidate region thus selected, only the facial region is selected using a wavelet-based directional contour information (AdaBoost) algorithm.

On the other hand, the speech feature extraction means 122 uses the MFCC (Mel-Frequency Cepstral Coefficients) for authentication of speech feature information in three types of verbal meaning, subject information, and sound channel information. Extract

As such, the extractor 120 extracts and fuses the facial image of the subject and voice feature information.

3 is a block diagram illustrating a conversion unit 130 of a gradual personal authentication system 100 using multiple biometric information according to an embodiment of the present invention.

Referring to FIG. 3, the conversion unit 130 of the gradual personal authentication system 100 using multiple biometric information according to an embodiment of the present invention may include a PCA (Principal Component Analysis) unit 131 and an IPCA (Incremental Principal Component Analysis). Section 132.

Principal Component Analysis (PCA) is called Principal Component Analysis and is a statistical method that makes it easier to handle high-dimensional signals that are difficult to handle. The PCA method was used to obtain a low-dimensional image space with only a face image. The space for the facial image obtained was called eigenspace and represented by a vector corresponding to the coordinate system constituting the space.

When the PCA unit 131 of the present invention converts face information of a certain number of subjects into a coordinate system in a unique space, the IPCA unit 132 is an eigen-vector or eigen- value) to extract feature information of the data. That is, the PCA unit 131 converts high-dimensional visual / audio fusion information into low-dimensional data and extracts low-dimensional features that are eigenvectors or eigen-values. Then, the IPCA unit 132 extracts the characteristic data of the visual / audio fusion information by adding or rotating axes by learning only the newly added information using the eigen vector value and the eigen value extracted from the PCA unit 131. .

4 is a block diagram illustrating a determination unit 140 of a gradual personal authentication system 100 using multiple biometric information according to an embodiment of the present invention.

Referring to FIG. 4, the determination unit 140 of the gradual personal authentication system 100 using multiple biometric information according to an embodiment of the present invention includes a resource allocation network (RAN) 141 and a long-term memory (LTM) 142. do.

, 숨겨진 항목의 출력을

, 출력을

라고 나타내면, 숨겨진 항목의 출력 값 및 출력의 출력값을 다음과 같이 표시할 수 있다. RAN 141 is an extended model of the Radial Basis Function (RBF) network that automatically performs the assignment of hidden items. Here, the number of input items, the number of hidden items, and the number of output items are represented by I, J, and K, respectively.

, The output of hidden items

Output

, The output value of the hidden item and the output value of the output can be displayed as follows.

이고,

는 j번째 숨겨진 항목의 중심 및 분산이고,

는 j번째 숨겨진 항목으로부터 k번째 출력 항목의 연결 가중치이고,

는 k번째 출력 항목의 바이어스(bias)이다. here

ego,

Is the center and variance of the jth hidden item,

Is the concatenation weight of the kth output item from the jth hidden item,

Is the bias of the k th output item.

The LTM 142 may store an item called 'memory items' corresponding to the representative input-output pair. These input-output pairs can be selected from the training data, and learning can be performed with the newly given training data to prevent these pairs from being deleted.

In this learning algorithm, memory items are created when hidden items are allocated. That is, the RBF center and its corresponding output can be stored as a memory item.

The learning algorithm of this RAN-LTM learner is divided into two stages. Specifically, the method may be divided into an operation of allocating a hidden item (eg, selection of an RBF center) and an operation of calculating a connection weight between the hidden item and the output item. The former operation is the same as that in a typical RAN except that memory items are created at the same time. When a hidden item is assigned, its values are later fixed.

)는 출력 에러에 기초하여 갱신되는 유일한 파라미터가 된다. Therefore, connection weight (

) Is the only parameter that is updated based on the output error.

In order to minimize the error, a squares method such as Equation 3 may be used.

Here, D is a matrix of column vectors corresponding to the target output.

이 생성되어 있다고 가정하면, 타깃 행렬(D)은

이다. Given the training data (x, d), memory item (M)

Assume that is generated, the target matrix (D)

to be.

는 학습 데이터 및 메모리 항목을 이용하여 수학식 4와 같이 계산될 수 있다. therefore,

May be calculated as shown in Equation 4 using the training data and the memory item.

Meanwhile, in order to solve W in Equation 1, singular value decompostion (SVD) may be used.

5 is a flowchart illustrating a gradual personal authentication method using multiple biometric information according to an embodiment of the present invention.

Referring to FIG. 5, in the progressive personal authentication method 500 using multiple biometric information according to another embodiment of the present invention, an information collection step (S510) of collecting subject image information and voice information, and extracting facial information from the image information is performed. Information extraction step (S520) of extracting and fusing voice feature information from voice information and converting high-dimensional data generated by fusing facial information and voice feature information into low-dimensional data including representative components (S530). ), An information separation step of separating the newly input data by using the representative component (S540), and a subject identification step (S550) of determining the subject by using the separated data.

Performance of the progressive personal authentication method 500 using multiple biometric information according to another embodiment of the present invention configured as shown in FIG. 5 is as follows.

First, the image of the subject is captured by using an imaging unit such as a camera, and the subject's voice is recorded using a recording unit such as a microphone (S510).

Thereafter, a facial image is extracted from the image of the subject, and voice feature information necessary for authentication is extracted from the voice and fused (S520).

The process of extracting the facial image from the image of the subject is described below with reference to FIG. 6.

6 is a flowchart illustrating a process of extracting a facial image according to another embodiment of the present invention.

Referring to FIG. 6, first, r (Red), g (Green), and b (Blue) components are separated from an image of a subject, the brightness information (I, Intensity) is classified, and face color filtering is performed. Thereafter, the extracted real-R and real-G values are combined into one and the edge components E and Edge are extracted (S521).

(intensity feature map),

(edge feature map),

(color feature map)을 모두 합한다(S522).The extracted information is extracted by performing Center Surround Difference & Normalization (CSD & N) process.

intensity feature map,

(edge feature map),

(color feature map) is summed (S522).

Finally, a salient map (SM) is generated from the summed image, face candidate regions are set in the generated salient map, and the facial image regions are extracted by applying the AdaBoost algorithm to the set face candidate regions (S523). .

Therefore, by applying this method, only the face image can be extracted from the subject's image.

7 is an exemplary view illustrating an example of a facial image extracted by the method of FIG. 6.

Referring to FIG. 7, (a) is a resultant image output by applying the AdaBoost algorithm, (b) is an image showing a candidate region of the protrusion map, and (c) is an extracted image of a face image according to the present invention.

(d) is an experimental result showing facial images extracted in the same manner as (c).

Meanwhile, FIG. 8 is a flowchart showing the application of the MLCC algorithm to extract voice feature information according to another embodiment of the present invention.

) 과정(S526), 음성정보일 경우, 멜 로그 파워의 리스트에 DCT(discrete cosine transform)를 적용하는 DCT과정(S527), 및 DCT 과정에서 추출된 정보의 상관관계를 제거하기 위해 Cepstral 변환하는 Cepstral filtering과정(S528)을 수행하여 피인증자의 음성특징 정보를 추출할 수 있다.Referring to FIG. 8, a short time fourier transform (STFT) process of applying Fourier transform to speech information (S524), and a Mel-scale filter that maps power of a spectrum included on a mel scale by using overlapping windows of triangles. In the banking process (S525), a log that applies a log of power to each of the mel frequencies

) Process (S526), the DCT process (S527) to apply a DCT (discrete cosine transform) to the list of the melody power, and the Cepstral transform to remove the correlation of the information extracted in the DCT process By performing the filtering process (S528) it is possible to extract the voice feature information of the subject.

The facial image and voice feature information extracted as described above are combined with each other and then converted into data of a low dimensional visual / audio fusion feature vector component including a representative component using a PCA algorithm (S530). Then, the feature is gradually extracted with respect to the newly input data based on the representative component using the IPCA algorithm (S540).

9 is an exemplary view showing an information conversion step (S530) and information separation step (S540) according to another embodiment of the present invention.

Referring to FIG. 9, when data (Gray Image) input through the information extraction step (S520) is input, a PCA algorithm is applied to a predetermined number or more images and converted into a low-dimensional image having an eigen vector value of n-dimension (S530). For the newly applied data, the IPCA algorithm is applied to extract an image with the eigen vector of the new m-dimension. Thereafter, the feature vector value is gradually updated to learn the updated feature (S540).

As such, the updated data is used as the means for discriminating the subject by applying the RAN-LTM algorithm as shown in FIG. 4 (S550).

<Experimental Example>

The following describes the experimental results for the gradual personal authentication system using multiple biometric information according to an embodiment of the present invention.

First, FIG. 10 and FIG. 11 illustrate the present invention using a brain recognition mechanism based face detection model, voice feature extraction using MFCC, and progressive feature extraction model and RAN-LTM classifier model. Experimental results of a gradual personal authentication system are shown.

FIG. 10 is a graph showing a result of authenticating a subject using a face image of a gradual personal authentication system using multiple biometric information according to an exemplary embodiment of the present invention.

Referring to FIG. 10, the dotted line indicates that the data performance gradually learned is performed up to 91.96%, and the solid line indicates that up to 97.0% is performed in a comparison test between the trained subject and 10 other images.

11 is an authentication result of a subject of a personal authentication system using multiple biometric information (visual and audio fusion information) in an environment where Gaussian noise of an image is added step by step from 0.1 to 1.0, according to an embodiment of the present invention; It is a graph showing the comparison of the authentication results of the subject of the personal authentication system using a single biometric information (visual information).

Referring to FIG. 11, the dotted line shows 20.83% of the performance results when only the image is tested when the noise is added step by step, and the solid line shows the results of the image and voice test when the noise is added to the image by the step.

The performance result here means the authentication accuracy of the subject.

While the invention has been shown and described with reference to certain preferred embodiments thereof, it will be understood by those of ordinary skill in the art that various changes in form and details may be made therein without departing from the spirit and scope of the invention as defined by the appended claims. This is possible.

Therefore, the scope of the present invention should not be limited by the described embodiments, but should be determined by the scope of the appended claims as well as the appended claims.

110: collector, 120: extractor
130: conversion unit, 140: determination unit

Claims (10)

Collecting unit for collecting the video information of the subject and the voice information of the subject;
An extraction unit for extracting a facial image from the image information and extracting voice feature information from the voice information;
While converting the high-dimensional data generated by fusing the face image and the voice feature information into low-dimensional data including representative components, a vector of the face image and the voice feature information is obtained and only newly input data is gradually obtained. A converter configured to extract the facial image and the voice feature information; And
And a discrimination unit for discriminating the subject by the information separated by the converting unit. The method of claim 1, wherein the facial image is
The personal information system of claim 1, wherein the facial information is extracted from the image information using a wavelet-based directional contour information (AdaBoost) algorithm. The method of claim 1, wherein the voice characteristic information
As the information for the authentication of the subject, a progressive personal authentication system using multiple biometric information, characterized in that the extracted by the MFCC (Mel-frequency cepstral coefficients) method. The method of claim 1, wherein the conversion unit
A gradual personal authentication system using multiple biometrics, characterized by using Incremental Learning Principal Component Analysis (IPCA), which enables progressive learning to accommodate new untrained input data. The method of claim 1, wherein the determining unit
A gradual personal authentication system using multiple biometric information, characterized by determining the subject by applying the RAN-LTM algorithm. An information collecting step of collecting the subject image information and the subject voice information;
An information extraction step of extracting a facial image from the image information and extracting and fusing voice feature information from the voice information;
Information for extracting a unique vector feature of the fusion data of the face image and the voice feature information while converting high-dimensional data generated by fusing at least one or more of the face image and the voice feature information into low-dimensional data including a representative component Conversion step;
A low dimensional extraction step for progressively extracting the subject information by adding or rotating an axis of a representative component including the eigenvector; And
And a subject identification step of discriminating the subject by the information extracted in the information conversion step. The method of claim 6, wherein the representative component
Gradual personal authentication method using multiple biometric information, characterized in that it comprises the eigen vector value or eigen value of the low-dimensional data. The method of claim 6, wherein the voice characteristic information
As the information for the recognition of the subject, a progressive personal authentication method using multiple biometric information, characterized in that extracted in the speech information by the MFCC (Mel-frequency cepstral coefficients) method. The method of claim 8, wherein the MFCC
A short time fourier transform (STFT) process for applying a Fourier transform to the speech information;
A Mel-scale filter bank process that maps the power of the spectrum contained on top of the mel scale using triangle overlapping windows;
Log to apply a log of power to each of the mel frequencies

) process;
A DCT process of applying a discrete cosine transform (DCT) to a list of mellog power in the case of the voice information; And
Cepstral filtering process for converting the Cepstral to remove the correlation of information extracted in the DCT process. The method of claim 6, wherein the low dimensional extraction step
Extracting low-order eigenvectors and eigenvalues; multi-biometric information characterized by using Incremental Learning Principal Component Analysis (IPCA) capable of incremental learning to accommodate new untrained input data Progressive personal authentication method using.

Images