KR20140107946A - Authentication apparatus and authentication method thereof - Google Patents

Abstract

An authentication apparatus is disclosed. The authentication apparatus according to the present invention includes a storage unit which stores the gesture information and face information of a user which is previously registered, a photographing unit which photographs an image of the user, a face recognizing unit which extracts a face image from the photographed user image and measures a face similarity by comparing the face image with the face information of the user which is previously registered, a gesture recognizing unit which calculates a difference image for each frame of the photographed user image and measures a gesture similarity by comparing a gesture feature extracted using the calculated difference image with the gesture information of the user which is previously registered, and a control unit which determines whether the user is registered based on the measured gesture and face similarities.

Description

BACKGROUND OF THE INVENTION 1. Field of the Invention [0001] The present invention relates to an authentication apparatus,

The present invention relates to an authentication apparatus and an authentication method thereof, and more particularly to an authentication apparatus and an authentication method using the user's face information and gesture information as authentication elements.

Personal information is frequently leaked through malicious hacking of financial services or portal sites and online service companies in recent years. As a result, interest in information protection is increasing in the society as a whole. Recently, the emergence of various smart devices that can access information through online has increased the need for strengthening the user authentication system for information security. In particular, authentication methods such as passwords, keys, and security cards being used for information security have limitations due to problems such as exposure, theft, and loss, while biometric authentication methods based on physical characteristics or behavioral characteristics Is expected to replace existing methods.

Face recognition is a user authentication method using physical features and is a technique widely used in addition to fingerprinting. In particular, since the face does not require contact with a dedicated device or a recognition device such as a fingerprint or an iris, there is no fear of obtaining biometric information, and a camera attached to a webcam or a smart device can be utilized. However, the accuracy of recognition may be greatly affected by lighting, pose, facial expression, and occlusion, and there is a disadvantage in that the authentication process can be bypassed using an authorized user's face photograph.

Gesture recognition is a user authentication method based on behavioral characteristics. It is developed in early gait analysis and is extended to hand gestures and sign language recognition for control of robot or system as a means of human - computer interaction. Recently, a user authentication system using an accelerometer attached to a smart device and a gyroscope or a touch panel has been developed. Because these methods require the use of an accelerometer, gyroscope or multi-touchable touch panel for each device, there is a compatibility problem for user authentication depending on the smart device.

SUMMARY OF THE INVENTION Accordingly, the present invention has been made keeping in mind the above problems occurring in the prior art, and it is an object of the present invention to provide a smart device, which analyzes image information obtained through an image sensor installed in a smart device and extracts face information and gesture information of a user An authentication device for performing compatibility of a user authentication system according to a smart device and robust user authentication compared to a single biometric authentication, and an authentication method thereof.

In order to achieve the above object, an authentication apparatus according to an embodiment of the present invention includes a storage unit for storing face information and gesture information of a previously registered user, a photographing unit for photographing a user image, A facial recognition unit for extracting a facial image and measuring facial similarity by comparing the facial image with facial information of the previously registered user; calculating a difference image for each frame of the captured user image; A gesture recognition unit for measuring the gesture similarity by comparing the gesture feature extracted using the difference image with the gesture information of the already registered user, and a controller for determining whether the user is a registered user based on the measured face similarity and the gesture similarity do.

In this case, the photographing unit may convert the photographed user image into a grayscale image.

Meanwhile, the face recognition unit may extract a face image from a user image using a Viola-Jones face detector, and extract a Histogram of Oriented Gradient (HOG) descriptor from the extracted face image to measure face similarity.

Here, the face recognizing unit may include:

을 이용하여 얼굴 유사도를 계산할 수 있다. (여기서, S_FACE는 얼굴 유사도,

는 기 등록된 사용자의 얼굴 정보에 대한 HOG 기술자 ,

는 촬영된 사용자 영상에서 추출된 얼굴 영상에 대한 HOG 기술자,

는 기 등록된 사용자의 얼굴 정보에 대한 i번째 윈도우의 HOG 기술자값,

는 촬영된 사용자 영상에서 추출된 얼굴 영상에 대한 i번째 윈도우의 HOG 기술자값, i는 1~M의 자연수 값이다.)

Can be used to calculate the face similarity. (Where S _FACE is face similarity,

A HOG descriptor for the face information of the previously registered user,

A HOG descriptor for the face image extracted from the photographed user image,

The HOG descriptor value of the i-th window with respect to the face information of the previously registered user,

Is the HOG descriptor value of the i-th window for the face image extracted from the photographed user image, and i is a natural number value of 1 to M.)

Meanwhile, the gesture recognition unit performs dynamic time warping to adjust the sequence length of the extracted gesture characteristic and the gesture information of the previously registered user equally. Gesture similarity can be measured through the distance scale based on mutual information.

Here, the gesture recognizing unit may include:

을 이용하여 제스처 유사도를 측정할 수 있다.(여기서, S_GESTURE는 제스처 유사도,

는 기 등록된 사용자의 제스처 특징 정보,

는 촬영된 사용자 영상에서 추출된 차영상의 제스처 특징 정보이다. )

Can be used to measure gesture similarity (where S _GESTURE is the gesture similarity,

Gesture feature information of a previously registered user,

Is the gesture feature information of the difference image extracted from the photographed user image. )

Meanwhile, the controller may determine that the user is registered only if the measured face similarity and the gesture similarity satisfy the respective authentication criteria.

The apparatus may further include a user interface unit for displaying the determination result.

Meanwhile, the authentication method of the authentication apparatus according to the present embodiment may include: storing face information and gesture information of a previously registered user; capturing a user image; extracting a face image from the photographed user image; The method of claim 1, further comprising the steps of: measuring face similarity by comparing facial images with facial information of the previously registered user; calculating a difference image for each frame of the captured user image; Measuring gesture similarity by comparing gesture information of a previously registered user and determining whether the user is a registered user based on the measured face similarity and the gesture similarity.

Here, the photographing may convert the photographed user image into a gray scale image.

The step of measuring the face similarity may include extracting a face image from a user image using a Viola-Jones face detector, extracting a HOG (His togram of Oriented Gradient) descriptor from the extracted face image, can do.

Here, the step of measuring the face similarity may include:

을 이용하여 얼굴 유사도를 계산할 수 있다. (여기서, S_FACE는 얼굴 유사도,

는 기 등록된 사용자의 얼굴 정보에 대한 HOG 기술자 ,

는 촬영된 사용자 영상에서 추출된 얼굴 영상에 대한 HOG 기술자,

는 기 등록된 사용자의 얼굴 정보에 대한 i번째 윈도우의 HOG 기술자값,

는 촬영된 사용자 영상에서 추출된 얼굴 영상에 대한 i번째 윈도우의 HOG 기술자값, i는 1~M의 자연수 값이다.)

Can be used to calculate the face similarity. (Where S _FACE is face similarity,

A HOG descriptor for the face information of the previously registered user,

A HOG descriptor for the face image extracted from the photographed user image,

The HOG descriptor value of the i-th window with respect to the face information of the previously registered user,

Is the HOG descriptor value of the i-th window for the face image extracted from the photographed user image, and i is a natural number value of 1 to M.)

The step of measuring the gesture similarity may include performing dynamic time warping in which a sequence length of the extracted gesture feature and a gesture information of the previously registered user are adjusted to be equal to each other. Gesture similarity can be measured through the distance scale based on mutual information.

 Here, the step of measuring the gesture similarity may include:

을 이용하여 제스처 유사도를 측정할 수 있다.(여기서, S_GESTURE는 제스처 유사도,

는 기 등록된 사용자의 제스처 특징 정보,

는 촬영된 사용자 영상에서 추출된 차영상의 제스처 특징 정보이다.)

Can be used to measure gesture similarity (where S _GESTURE is the gesture similarity,

Gesture feature information of a previously registered user,

Is the gesture feature information of the difference image extracted from the photographed user image.)

The step of determining whether the user is the registered user may be determined as a registered user only when the measured face similarity and the gesture similarity satisfy the respective authentication criteria.

The authentication method may further include displaying the determination result.

1 is a block diagram showing the configuration of an authentication apparatus according to an embodiment of the present invention;
2 is a view for explaining a face recognition method according to an embodiment of the present invention;
3 is a view for explaining a gesture recognition method according to an embodiment of the present invention;
4 is a view for explaining a FAR / FRR graph for a face according to an embodiment of the present invention;
5 is a view for explaining a FAR / FRR graph for a gesture according to an embodiment of the present invention;
6 is a view for explaining scattering of a sample according to an embodiment of the present invention,
FIG. 7 is a view for explaining FRR graph of 0% FAR for each authentication element according to an embodiment of the present invention, FIG.
FIG. 8 is a flowchart for explaining processing of an authentication method according to an embodiment of the present invention,
9 is a flowchart illustrating an authentication method according to an embodiment of the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings.

The recognition apparatus according to an embodiment of the present invention may be implemented by electronic devices such as a security system, an access management system, a server system, a PC (personal computer), a tablet, and a mobile phone, which require user authentication.

1 is a block diagram showing the configuration of an authentication apparatus according to an embodiment of the present invention.

1, the authentication apparatus 100 includes a storage unit 110, a photographing unit 120, a control unit 130, a face recognition unit 140, a gesture recognition unit 150, and a user interface unit 160 .

The storage unit 110 stores face information and gesture information of a previously registered user. Specifically, the storage unit 110 may store face information and gesture information of the user to determine whether the user is permitted to access.

Here, the face information is a face image for a user who is allowed to access, and the gesture information is a gesture image set by a user who is allowed to access. This may be photographed at a separate step and stored in the storage unit 110. [

Specifically, the storage unit 110 may store characteristics of the gesture image input as the gesture information. For example, assuming that a gesture characteristic of a difference image of an image shaking the hand in left and right is A, and a gesture characteristic of a difference image of an image shaking the hand up and down is B, the storage unit 110 stores the gesture characteristics A and B Can be stored. The stored gesture characteristics A and B can be used in the gesture recognition unit 150 to be described later.

Here, the gesture characteristic and difference image will be described in detail in the gesture recognition unit 150, which will be described later.

The storage unit 110 may temporarily store the user image captured by the photographing unit 120 and the face recognition unit 140 and the gesture recognition unit 150, which will be described later.

The storage unit 110 may include a storage medium in the recognition apparatus 100 and an external storage medium such as a removable disk including a USB memory, a storage medium connected to a host, a Web server via a network, Lt; / RTI >

The photographing unit 120 photographs a user image. Specifically, the photographing unit 120 is implemented as an image capturing device, and can photograph a user approaching the authentication device 100. [

Here, the user image photographed by the photographing unit 120 may be generated as a moving image file and stored in the storage unit 110. [

Meanwhile, the photographing unit 120 may perform a preprocessing process of converting the photographed user image into a predetermined size and converting it into a grayscale image.

The face recognition unit 140 extracts a face image from the photographed user image, and measures the face similarity by comparing the face image with the face information of the previously registered user.

Specifically, the face recognition unit 140 extracts a face image from a user image using a Viola-Jones face detector, and extracts a Histogram of Oriented Gradient (HOG) descriptor from the extracted face image to measure the face similarity have.

Here, a Viola-Jones face detector refers to a detector using a Viola-Jones algorithm to detect a specific object (or face) within a particular image or image. The Viola-Jones algorithm has a high performance of object detection, and is especially suitable for real-time face detection because the speed of face detection is very fast, and the detection rate is very high even when the pixels of the detection background are almost the same value.

The Viola-Jones algorithm is described in detail in PAUL VIOLA, MICHAEL J. JONES, 2004, "ROBUST REAL-TIME FACE DETECTION", International Journal of Computer Vision 57 (2) do.

Meanwhile, the face recognition unit 140 may extract a Histogram of Oriented Gradient (HOG) descriptor from the extracted user face image using the Viola-Jones face detector, and compare the extracted face image with the previously stored user image. Here, the HOG descriptor is a feature information descriptor designed for the purpose of object detection, and represents the intensity gradient or direction of a pixel in a local area.

HOG engineers are described in detail in N. Dalal and B. Triggs, 2005, "Histograms of oriented gradients for human detection", IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

The face recognition unit 140 according to an exemplary embodiment of the present invention first searches for a user's face image in a user image photographed by the photographing unit 120 using a Viola-Jones face detector, The size of the face image can be adjusted to 32 x 32. In addition, it is possible to extract features from facial images by applying an algorithm that performs statistical learning on face data represented by a histogram-based local feature descriptor (HOG descriptor) robust to illumination change for face recognition. Then, the face recognition unit 140 can measure similarity between the registered face information of the user stored in the storage unit 110 and the feature of the extracted face image.

Meanwhile, the face recognition unit 140 according to an embodiment of the present invention can measure the degree of similarity using the following equation (1).

는 기 등록된 사용자의 얼굴 정보에 대한 HOG 기술자 ,

는 촬영된 사용자 영상에서 추출된 얼굴 영상에 대한 HOG 기술자,

는 기 등록된 사용자의 얼굴 정보에 대한 i번째 윈도우의 HOG 기술자값,

는 촬영된 사용자 영상에서 추출된 얼굴 영상에 대한 i번째 윈도우의 HOG 기술자값, i는 1~M의 자연수 값이다. 한편, Dist() 함수는 두 데이터간의 거리를 산출하는 L1-Norm을 포함한 일반적인 거리 함수이다.Here, S _FACE is face similarity,

A HOG descriptor for the face information of the previously registered user,

A HOG descriptor for the face image extracted from the photographed user image,

The HOG descriptor value of the i-th window with respect to the face information of the previously registered user,

Is the HOG descriptor value of the i-th window for the face image extracted from the photographed user image, and i is a natural number value of 1 to M. On the other hand, the Dist () function is a general distance function including an L1-Norm that calculates the distance between two data.

On the other hand, FIG. 2 is a diagram for explaining the above-described HOG descriptor extraction.

Referring to FIG. 2, the face image 210 extracted through the Viola-Jones face detector is divided by the 4 × 5-shaped lattice region 220. Here, the shape of the lattice region is not fixed but can be changed, and the face image can be divided into a 4x4-shaped lattice region. The HOG descriptor for 20 partial regions can be extracted and the distance between the HOG descriptors of the facial image to be compared can be calculated. To calculate the actual face distance, the above-mentioned N. Dalal and B. Triggs used L1-Norm, but other types of distance scale can be used.

The face similarity measured by the face recognizing unit 140 may be transmitted to the controller 130 to be described later and used to determine whether the registered user is a registered user.

The gesture recognition unit 150 calculates a difference image for each frame of the photographed user image, and measures the gesture similarity by comparing the gesture feature extracted using the calculated difference image with the gesture information of the pre-registered user .

In this case, the gesture recognition unit 150 performs a dynamic time warping in which the sequence length of the extracted gesture characteristic and the gesture information of the previously registered user are adjusted to be equal to each other. Gesture similarity can be measured through the distance scale based on mutual information.

Specifically, the gesture recognition unit 150 calculates a difference image sequentially by dividing the photographed user image by a frame unit, and applies an algorithm for performing statistical feature extraction on the image difference, that is, Principle Component analysis, Can be extracted. In order to measure the similarity of two gesture features with different lengths, dynamic time warping is performed to adjust the length of the gesture data to the same length, and then, through a distance scale based on mutual information, The degree of similarity between gesture features can be measured.

Here, the difference image can be obtained by performing a subtract operation between adjacent frames among a plurality of frames constituting the captured user image. Specifically, when the pixel information of the immediately preceding frame is retrieved based on the current frame, the pixel information of the current frame is retrieved, and the pixel amount is temporarily determined and subtracted, the difference image can be extracted have.

On the other hand, the information appearing in the difference image may mean a point where a change occurs in the image. A gesture feature is a sequence in which features are extracted by performing statistical analysis (PCA) on the difference image of the learning data, and the extracted features are successively connected to one gesture.

3 is a view for explaining an operation of extracting difference images and gesture characteristics.

Referring to FIG. 3, the size adjustment and the RGB image are converted into a gray image in a plurality of frames 310-1, 310-2, and 310-3 to 310-n constituting a user image photographed by the photographing unit 110 The preprocessors 320-1, 320-2 and 320-3 to 320-n for performing the conversion are performed and subtraction operations are performed between adjacent frames to generate the difference images 330-1 and 330-2 to 330- n).

The gesture recognition unit 150 extracts a gesture feature of the obtained difference image and calculates a statistical distance between the gesture feature of the extracted difference image and the gesture feature of the gesture information stored in the storage unit 110. [

The method of extracting the gesture feature of the difference image can be classified by various criteria. As a representative criterion, there is a global feature extraction method and a regional feature extraction method according to the range of the feature extraction region.

The global feature extraction method uses statistical values such as covariance for the whole difference image. It is general to apply eigenvalues and eigenvectors to satisfy a specific objective function. Therefore, the features obtained from arbitrary learning data are expressed in such a way that the entire description of the learning data is possible. Representative methods include Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA).

The regional feature extraction method extracts local features that can explain the feature of the difference image, and does not consider the feature of the region other than the specific region in the difference image. Local feature extraction methods are typically SIFT (Scale Invariant Feature Transform), LPP (Locality Preserving Projection), and dense-SIFT.

Meanwhile, the gesture recognition unit 150 adjusts two data having different lengths using dynamic time warping (DTW), calculates a statistical distance between the gesture characteristic of the extracted difference image and the stored gesture characteristic do.

If the captured user image and the stored gesture image have different lengths, there is a problem that the two images can not be compared.

Accordingly, the gesture recognition unit 150 adjusts the length of the two images using the dynamic time matching before comparing the gesture feature of the difference image extracted from the captured user image with the pre-stored gesture feature.

Here, dynamic time alignment is a method that can calculate an optimal path for sorting two sequence data and can be used for calculating the distance between time series data. In the present embodiment, a statistical distance is used to calculate the distance between the gesture characteristics of the difference image, as shown in Equation 2 below.

는 기 등록된 사용자의 제스처 특징 정보,

는 촬영된 사용자 영상에서 추출된 차영상의 제스처 특징 정보이다. 한편, Dist() 함수는 두 데이터간의 통계적 거리를 산출하는 함수이며, cov() 공분산을 산출하는 함수, std()는 표준편차를 산출하는 함수이다.Here, S _GESTURE is the gesture similarity,

Gesture feature information of a previously registered user,

Is the gesture feature information of the difference image extracted from the photographed user image. On the other hand, the Dist () function is a function that calculates the statistical distance between two pieces of data. The cov () covariance function and std () is a function that calculates the standard deviation.

Here, the calculated gesture similarity is closer to 0, the statistical distance is shorter, and it can be judged that the gestures are mutually similar.

For example, when the gesture characteristic of the car image of the image shaking the hand is X, the gesture characteristic of the car image of the image shaking up and down is Y, and the gesture characteristic of the car image of the circle image is Z, If the statistical distance of the gesture feature X of the difference image of the image and the gesture feature X of the stored difference image is the shortest, the gesture of the user can be recognized as a shaking motion to the left and right corresponding to the feature point X. [

The controller 130 determines whether the user is a registered user based on the measured face similarity and gesture similarity. Specifically, the controller 130 may determine whether the user is a registered user based on the gesture similarity measured by the face recognition unit 140 and the gesture recognition unit 150. [

Specifically, the controller 130 can determine that the user is registered only when the measured face similarity and the gesture similarity satisfy the respective authentication criteria. That is, if both the face of the user and the gesture similarity photographed by the photographing unit 110 are lower than respective thresholds, the user is authenticated as 'accept', and any one of the two similarities is higher than the threshold You can 'reject' it.

On the other hand, the control unit 130 controls each configuration in the authentication apparatus 100. [

The user interface unit 160 displays the determination result. Specifically, the user interface unit 160 may display the authentication result determined by the control unit 130. FIG.

For example, when it is determined that the user is a registered user, 'accept' can be displayed through a display, and a control signal can be sent to a separate device to perform an operation such as 'unlock'.

Meanwhile, the user interface unit 160 may include a plurality of function keys that a user can set or select various functions supported by the authentication apparatus 100, and may display various information provided by the authentication apparatus 110 . The user interface 160 may be implemented by a combination of a monitor and a keyboard, or may be implemented as an apparatus in which inputs and outputs are simultaneously realized, such as a touch pad.

As described above, the authentication apparatus 100 using both the face similarity degree and the gesture similarity degree can perform stronger user authentication than when using one similarity degree. The effect will be described in detail with reference to FIGS. 4 to 7 below.

In addition, the authentication apparatus 100 according to an exemplary embodiment of the present invention uses a photographing unit commonly installed in a smart device, so that it is compatible with user authentication in various smart devices and utilizes multiple biometric information to construct a robust user authentication system can do.

Hereinafter, simulation results of the authentication apparatus 100 according to an embodiment of the present invention will be described with reference to FIGS. 4 to 7. FIG.

A total of 20 users are regarded as impersonators for the remaining 19 users, and two images for all 20 persons are used as test data to measure the similarity with the registered user data. Observation results.

Here, FAR (False Accept Rate) and FRR (False Reject Rate) are used as evaluation scales. FAR is a ratio of misclassifying the impostor as a genuine user, FRR is a genuine user Impostor ").

Although the authentication device 100 may generally be allowed to classify a user as an imposter, it must select a criterion with a minimum FRR at FAR 0%, since it should not be allowed to misclassify the impersonator as a user.

FIGS. 4 and 5 are views for explaining FAR / FRR graphs for faces and gestures according to an embodiment of the present invention.

Referring to FIG. 4, the vertical line 430 in the graph indicates a threshold at which the FAR is 0%. If the criterion for the face is 12.28 (430), the FAR 410 is 0% and the FRR 420 is 42.5%.

On the other hand, referring to FIG. 5, FAR 510 is 0% and FRR 520 is 15% when the criterion for gesture is 0.78 (530).

The FAR and FRR generally vary according to the criterion. In the simulation, it is assumed that the security is strict and the FAR is set to 0%.

4 and 5, when the FAR is 0% for the face information, the FRR is very high. This is appropriate for classification problem when using face information, but it may cause problems because authentication problem is sensitive to various variations of each person.

On the other hand, when the FAR is 0% in the case of the gesture information, the FRR shows a result of 15%, which is more suitable for the authentication problem than the face. This is because the variance between classes due to the use of different gestures for each user is larger than the variance within the class.

FIGS. 6 and 7 are summarized FAR / FRR results according to data used for authentication. FIG. 6 is a view for explaining a scattering diagram of a sample according to an embodiment of the present invention, and FIG. FIG. 5 is a diagram for explaining the FRR graph of the 0% FAR for each authentication factor according to the example.

Referring to FIG. 6, when FAR is set to 0% in both face authentication and gesture authentication, FRR shows a result of 10%. Here, the judgment standard of the face similarity degree may be set to 12.28 (610) and the judgment standard of the gesture similarity degree may be set to 0.78 (620) in order to reflect the experimental results of FIGS.

On the other hand, FIG. 7 compares the FAR / FRR for each authentication element.

Referring to FIG. 7, FRR is shown when face and gesture information are used separately and when both pieces of information are used together. If both information are used together (Combination), only if the information is all 'Accepted', and if any of the information is 'Deny', the FRR is calculated with 'Final Decision' as 'Denied' It is confirmed that the FRR is improved.

According to the simulation result of FIG. 7, the authentication apparatus 100 according to the embodiment of the present invention can perform sophisticated user authentication using the image-based gesture information and face information that have not been used before .

8 is a flowchart for explaining the processing of the authentication method according to the embodiment of the present invention.

Referring to FIG. 8, the authentication apparatus 100 can capture a user image (S810).

Then, the authentication apparatus 100 may perform a preprocessing process of converting the photographed image into a grayscale image (S820).

Then, the face image is searched in the photographed user image (S830), and the size of the searched face image can be converted into the predetermined size (S831). Then, the HOG descriptor is extracted (S832), and the face similarity degree can be measured by comparing the face image with the face information of the already registered user (S833).

At the same time, a difference image for each frame of the captured user image is calculated (S840), and the gesture feature (S841) extracted using the calculated difference image and the gesture information of the already registered user are subjected to dynamic time matching (S842), and the gesture similarity can be measured (S843).

The facial similarity measurement and the gesture similarity measurement are described in detail in FIG. 1, but a duplicate explanation will be omitted.

Then, the authentication apparatus 100 can determine whether the user is a registered user based on the measured face similarity and the gesture similarity (S850).

Specifically, the authentication apparatus 100 can determine that the user is a registered user only if the measured face similarity and the gesture similarity satisfy the respective authentication criteria. That is, if both of the captured face and gesture similarity are lower than the respective thresholds, the user is authenticated as 'accept', and if any of the two similarities is higher than the threshold value, can do.

Then, the authentication apparatus 100 can display the determination result (S860).

The recognition method as shown in Fig. 8 can be executed on the authentication device having the configuration of Fig. 1, or on the authentication device having other configurations.

9 is a flowchart illustrating an authentication method according to an embodiment of the present invention.

Referring to FIG. 9, the authentication apparatus 100 may store face information and gesture information of a previously registered user (S910).

Then, the authentication apparatus 100 can capture a user image (S920). Specifically, when the user accesses the authentication apparatus 100, the authentication apparatus 100 can take a user image for authentication.

Then, the facial image is extracted from the captured user image, and the facial similarity degree can be measured by comparing the facial image with the facial information of the already registered user (S930).

At the same time, it is possible to calculate the difference image for each frame of the captured user image, and to measure the gesture similarity by comparing the gesture feature extracted using the calculated difference image with the gesture information of the pre-registered user (S940 ).

The facial similarity measurement and the gesture similarity measurement are described in detail in FIG. 1, but a duplicate explanation will be omitted.

Then, the authentication apparatus 100 can determine whether the user is a registered user based on the measured face similarity and the gesture similarity (S950).

Specifically, the authentication apparatus 100 can determine that the user is a registered user only if the measured face similarity and the gesture similarity satisfy the respective authentication criteria. That is, if both of the captured face and gesture similarity are lower than the respective thresholds, the user is authenticated as 'accept', and if any of the two similarities is higher than the threshold value, can do.

Then, the authentication apparatus 100 can display the determination result (S960).

The recognition method as shown in Fig. 9 can be executed on the authentication device having the configuration of Fig. 1, or on the authentication device having other configurations as well.

Furthermore, the authentication method as described above can be implemented as at least one executable program for executing the above-described authentication method, and the executable program can be stored in a computer readable recording medium.

Thus, the authentication methods of the present invention may be embodied as computer-writable code on a computer-readable recording medium. The computer-readable recording medium may be a device capable of storing data that can be read by a computer system.

While the present invention has been particularly shown and described with reference to exemplary embodiments thereof, it is to be understood that the invention is not limited to the disclosed exemplary embodiments, but, on the contrary, It will be understood by those skilled in the art that various changes in form and details may be made therein without departing from the scope of the appended claims.

110: storage unit 120: photographing unit
130: control unit 140: face recognition unit
150: gesture recognition unit 160: user interface unit

Claims (16)

In the authentication apparatus,
A storage unit for storing face information and gesture information of a previously registered user;
A photographing unit for photographing a user image;
A facial recognition unit for extracting a facial image from the photographed user image and measuring facial similarity by comparing the facial image with facial information of the previously registered user;
A gesture recognition unit for calculating a difference image for each frame of the captured user image and measuring gesture similarity by comparing the gesture feature extracted using the calculated difference image with gesture information of a previously registered user; And
And a controller for determining whether the user is a registered user based on the measured face similarity and the gesture similarity. The method according to claim 1,
Wherein,
And converts the photographed user image into a grayscale image. The method according to claim 1,
Wherein the face recognizing unit comprises:
Extracts a face image from a user image using a Viola-Jones face detector, and extracts a Histogram of Oriented Gradient (HOG) descriptor from the extracted face image to measure face similarity. The method of claim 3,
Wherein the face recognizing unit comprises:

And the face similarity calculating unit calculates the face similarity by using the face similarity. (Where S _FACE is face similarity,

A HOG descriptor for the face information of the previously registered user,

A HOG descriptor for the face image extracted from the photographed user image,

The HOG descriptor value of the i-th window with respect to the face information of the previously registered user,

Is the HOG descriptor value of the i-th window for the face image extracted from the photographed user image, and i is a natural number value of 1 to M.) The method according to claim 1,
The gesture recognition unit
Performing dynamic time warping to adjust the sequence length of the extracted gesture characteristic and the gesture information of the pre-registered user equally. Wherein the gesture similarity degree is measured through a distance scale based on mutual information. 6. The method of claim 5,
The gesture recognizing unit recognizes,

_Wherein the gesture _resemblance is measured using the gesture _resemblance ,

Gesture feature information of a previously registered user,

Is the gesture feature information of the difference image extracted from the photographed user image.) The method according to claim 1,
Wherein,
And judges that the user is a registered user only when both the measured face similarity and the gesture similarity satisfy respective authentication criteria. The method according to claim 1,
And a user interface unit for displaying the result of the determination. An authentication method of an authentication apparatus,
Storing face information and gesture information of a previously registered user;
Capturing a user image;
Extracting a face image from the photographed user image, and measuring face similarity by comparing the face image with face information of the pre-registered user;
Calculating a difference image for each frame of the photographed user image, measuring gesture similarity by comparing gesture characteristics extracted using the calculated difference image with gesture information of a previously registered user; And
And determining whether the user is a registered user based on the measured face similarity and the gesture similarity. 10. The method of claim 9,
Wherein the photographing step comprises:
And converting the photographed user image into a grayscale image. 10. The method of claim 9,
Wherein the step of measuring the face similarity comprises:
Extracting a face image from a user image using a Viola-Jones face detector, and extracting a Histogram of Oriented Gradient (HOG) descriptor from the extracted face image to measure face similarity. 12. The method of claim 11,
Wherein the step of measuring the face similarity comprises:

Wherein the face similarity is calculated by using S _FACE ,

A HOG descriptor for the face information of the previously registered user,

A HOG descriptor for the face image extracted from the photographed user image,

The HOG descriptor value of the i-th window with respect to the face information of the previously registered user,

Is the HOG descriptor value of the i-th window for the face image extracted from the photographed user image, and i is a natural number value of 1 to M.) 10. The method of claim 9,
Wherein the step of measuring the gesture similarity comprises:
Performing dynamic time warping to adjust the sequence length of the extracted gesture characteristic and the gesture information of the pre-registered user equally. Wherein the gesture similarity is measured through a distance scale based on mutual information. 14. The method of claim 13,
Wherein the step of measuring the gesture similarity comprises:

_Wherein the gesture _resemblance is measured using the Gesture similarity,

Gesture feature information of a previously registered user,

Is the gesture feature information of the difference image extracted from the photographed user image.) 10. The method of claim 9,
Determining whether the user is the registered user,
Wherein the determined user is determined to be a registered user only when both the measured face similarity and the gesture similarity satisfy respective authentication criteria. 10. The method of claim 9,
And displaying the result of the determination.

Images