KR101057720B1 - User Authentication System and Method - Google Patents

Abstract

According to an aspect of the present invention, there is provided a user authentication method that simultaneously uses number authentication by a password input by a user and face recognition by face image by a user image of photographing a user: receiving a user authentication request from the user; Obtaining an image, receiving a password from the user, re-photographing the user to obtain a second image, authenticating a number using the received password, the first image, the second image Determining whether the first image and the first image and the second image are suitable images applicable to face recognition, and when the image is determined to be suitable, the first image, the second image, or the first image and the second image Performing face recognition by using a user, if both the result of the number authentication and the result of the face recognition pass, the user Determining success of authentication.

User authentication, face recognition, password, template

Description

User Authentication System and Method {SYSTEM AND METHOD FOR USER IDENTIFYING}

The present invention relates to a system and a method for authenticating a user, and more particularly, to succeed in authenticating a user only when a user passes simultaneously an authentication by a password and an authentication by a user's face in order to access a security application requiring security management. The present invention relates to a user authentication system and a method for allowing access to the security application device.

In a device requiring security management, such as a general home automation system, a user authentication method using a password is widely used as a user authentication method for allowing access from only the right user from the outside.

However, such a user authentication method has a problem in that it is possible to steal the number by an invalid user and thus, the security is poor.

Therefore, when a password is stolen by an unjust user, a system has been proposed that allows the user to extract an unjust user later by photographing an unjust user. However, in this case, if an unfair user approaches the state wearing a face mask, eye patch, glasses, hat, etc., there is a problem in that it is difficult to accurately identify the unjust user because the face is not clearly recognized.

Therefore, when the user covers the face more than necessary, it is desirable to detect the face and to require the user to expose the face.

On the other hand, as a further improved user authentication method, in recent years, a system for performing user authentication through face recognition using an image of a user's face has been popularized.

However, in the conventional system, if there is a part that cannot be recognized at least one of the face parts, the security system refuses to recognize the face, and the user inevitably covers a part of the face with a mask, eye patch, glasses, hat, or the like. Or there may be a case where the image is not clear due to the face angle, ambient lighting conditions, etc. at the time of shooting, it is also necessary to accurately determine the case to eliminate the inconvenience of user authentication fails.

The present invention is to provide a user authentication system and method which can perform more thorough security by determining that the user authentication succeeds only when both the face recognition and the authentication by password are successful.

Also, even if a part of the user's face photographed for face recognition is obscured or not clear by surrounding conditions, a user who can perform face recognition without requiring the user to retake the picture if the minimum conditions necessary for face recognition are met. An authentication system and method are provided.

In addition, the present invention is to provide a user authentication system and method that can perform more thorough security by determining whether an image photographing a user is stolen by an illegal user and attempting user authentication.

According to an aspect of the present invention, there is provided a user authentication method that simultaneously uses number authentication based on a password input by a user and face recognition using a user image photographed by a user, including: receiving a user authentication request from the user; Photographing the user to obtain a first image; receiving a password from the user; re-photographing the user to obtain a second image; authenticating a number using the received password; Determining whether the first image, the second image, or the first image and the second image are suitable images applicable to face recognition; when determined as a suitable image, the first image, the second image, or the Performing face recognition using the first image and the second image, the result of the number authentication and the determination of the face recognition If the all-pass, there is provided a user authentication method comprising the step of determining the user authentication is successful.

The first image, the second image, or the first image and the second image may be obtained by photographing at a moment that the user does not recognize.

The determining of whether the image is suitable may include: extracting a face region image of a specific face region from the photographed first image, the second image, or the first image and the second image; Computing the shape information of the face region from the face image, and calculating the similarity by comparing the calculated shape information with template data stored in advance for the face region, and calculating the similarity. And determining the first image, the second image, or the first image and the second image as a suitable image when the reference similarity is greater than or equal to the reference similarity.

The determining of whether the image is a suitable image may include: extracting respective face region images of a plurality of face portions from the photographed first image, the second image, or the first image and the second image; Computing the shape information of the corresponding face part from the extracted facial part image by numerically comparing the calculated shape data with the template data stored for the corresponding face part in advance and comparing the similarity for each part. Calculating a combination similarity by combining the calculated similarities for each site by a predetermined arithmetic expression; and when the calculated combined similarity is equal to or greater than a predetermined reference combined similarity, the first image and the second And determining the first image and the first image and the second image as a suitable image.

The determining of whether the image is a suitable image may include: extracting respective face region images of all face regions from the photographed first image, the second image, or the first image and the second image, wherein the extraction is performed. Calculating the shape information of the corresponding face part from each of the face image obtained, and calculating the similarity for each part by comparing the calculated shape data with the template data previously stored for the face part; Calculating a total similarity by combining all of the calculated similarities for each site by a predetermined arithmetic expression, and when the calculated total similarity is equal to or greater than a predetermined reference total similarity, the first image, the second image, or And determining the first image and the second image as a suitable image.

Further, the step of number authentication and the face recognition may include: performing number authentication by comparing the password inputted from the user with a preset authentication password, and if the number authentication is successful, in response to the authentication password. Reading a stored authentication face image, and comparing the read authentication face image with the first image, the second image, or the first image and the second image to perform face recognition; It features.

Also, an image theft determining step of determining whether the first image, the second image, or the first image and the second image are generated by being photographed from an image stolen by an illegal user, and the image is stolen. If it is determined that the generated from the image, characterized in that it further comprises the step of determining that the user authentication failure.

The determining whether the image is stolen may include: extracting a background image from the first image, the second image, or the first image and the second image, and extracting at least one line shape from the extracted background image. And when the extracted line form forms a closed figure surrounding a part of a user's face in the first image, the second image, or the first image and the second image, the first image and the second image. And determining that the first image and the second image are generated from the stolen image.

The determining whether the image is stolen may include: extracting a user's body outline from the first image, the second image, or the first image and the second image; When the first image, the second image, or the bottom edges of the first image and the second image do not coincide with each other, the first image, the second image, or the first image and the second image Determining that it is generated from the stolen image.

The determining whether the image is stolen may include: extracting respective background images from the first image and the second image, extracting at least one line shape from each of the extracted background images, and the first image. When the linear form extracted from the background image of the image and the linear form extracted from the background image of the second image do not coincide with each other, the first image, the second image, or the first image and the second image are stolen. And determining that the image is generated from the captured image.

The determining whether the image is stolen may include: extracting a background image from which the user's body shape is removed from the first image, the second image, or the first image and the second image, at least one of the extracted background image Extracting a linear form of the first image, the second image, or the first image when the extracted linear form does not match a reference linear form extracted from a surrounding image photographed without a user at a current position; And determining that the image and the second image are generated from the stolen image.

In addition, the present invention for achieving the above object, the user authentication when passing through the number authentication by the password input by the user who wants to access a predetermined security application device and the face recognition by the user image photographing the user at the same time A user authentication system regarded as success of: A first image of photographing the user, a second image of re-photographing the user at a predetermined time interval from the time of photographing the first image, and a password input from the user is transmitted. A user terminal; Upon receiving the first image and the second image, it is determined whether the first image, the second image, or the first image and the second image are suitable images applicable to face recognition, and the Number authentication is performed, face recognition is performed using the first image, the second image, or the first image and the second image, and both the result of the suitable image determination, the number authentication result, and the face recognition result And an authentication processing device for determining that the user authentication is successful when the user passes, and allowing the user to access the security application device.

The authentication processing apparatus may calculate and calculate shape information about a specific face part in the first image, the second image, or the first image and the second image, and calculate the calculated shape information and the corresponding face. Similarity is calculated by comparing template data stored in advance with respect to a part, and when the calculated similarity is equal to or greater than a predetermined reference similarity, the first image, the second image, or the first image and the second image are suitable. It is characterized by judging by an image.

The authentication processing apparatus may calculate and calculate respective shape information of a plurality of face parts in the first image, the second image, or the first image and the second image, and calculate the respective shapes. The similarity for each part is calculated by comparing the template and pre-stored template data with respect to the corresponding face part, and the combined similarity is calculated by combining the calculated similarity for each part by a predetermined arithmetic formula, and calculating The first image, the second image, or the first image and the second image are determined to be suitable images when the combined similarity degree is equal to or greater than a predetermined reference combination similarity degree.

The authentication processing apparatus may calculate and calculate respective shape information of all face parts in the first image, the second image, or the first image and the second image, and calculate the calculated shape information. Compares the template data stored in advance with respect to each of the corresponding face parts to calculate similarity for each part, and calculates the total similarity by combining all of the calculated similarities for each part by a predetermined arithmetic formula. When the calculated total similarity is equal to or greater than a predetermined reference total similarity, the first image, the second image, or the first image and the second image may be determined as a suitable image.

Further, number authentication and face recognition are performed only when the first image, the second image, or the first image and the second image are determined to be suitable images.

In addition, the authentication processing apparatus determines whether the first image, the second image, or the first image and the second image are photographed and generated from an image stolen by an improper user. When it is determined that the second image, or the first image and the second image are generated from the stolen image, it is determined that the user authentication has failed.

According to the user authentication method of the present invention configured as described above and a user authentication system for executing the method, the user is photographed twice for user authentication to generate a user image, and each user image is capable of face recognition. It will first check if it is taken enough. This makes it possible to determine in advance the face recognition failure due to the inappropriate video.

Also, even if a part of the user's face taken for face recognition is obscured or not clear by the surrounding conditions, if the minimum conditions necessary for face recognition are met, face recognition can be performed without requiring the user to retake the face. Thus, user convenience is improved.

In addition, since user authentication is performed by simultaneously applying number authentication and face recognition using a password, further improved security can be provided.

In addition, since it is possible to determine the case where the video of the user is stolen by an illegal user, it is possible to prevent the illegal access.

Hereinafter, a user authentication system and method according to an embodiment of the present invention will be described in detail with reference to the accompanying drawings.

1 is a block diagram schematically showing the configuration of a user authentication system according to an embodiment of the present invention. Referring to FIG. 1, a user authentication system includes a user terminal 100, an authentication processing device 200, and a security application device 300.

The security application device 300 is a system that processes user authentication in various ways in order to allow and block a user's access to a system requiring security management, such as a home automation system, an office automation system, a personal computer system, a server system, and the like. It may be itself or a part thereof.

The user terminal 100 may be a mobile phone, a home video phone, a remote control, a personal computer, or the like that accesses the security application device 300 and provides various commands.

On the other hand, the user terminal 100 at least, the camera that can be used for number authentication through a password by inputting a camera, a number or a character that can be used for later face recognition by taking a background of the user's face and surroundings of the face And display means capable of displaying various information visually or audibly. In addition, the user terminal 100 is preferably further provided with a wired or wireless communication means to remotely access the security application device 300 via a wired or wireless communication network.

The authentication processing apparatus 200 operates in close coupling with the security application apparatus 300, and authenticates the device to the user terminal 100 that is accessed through the wired / wireless communication network and the user to the user who operates the user terminal 100. The authentication is performed, and the user who succeeds in user authentication is allowed to access the security application device 300 only through the user terminal 100 in which the device is authenticated. To this end, the authentication processing device 200 is configured to process number authentication by password and face recognition by user face image.

When the security application device 300 is a home automation system and the user terminal 100 is a mobile phone in the user authentication system configured as described above, the security application device 300 and the authentication processing device 200 are Nearly installed in the house and wired, the user terminal 100 and the authentication processing device 200 may be configured to communicate through a mobile communication network or a wired or wireless Internet network of the mobile phone.

The operation of the user authentication system according to the present invention will be described with reference to FIG. 2 is a flowchart schematically illustrating a user authentication method by a user authentication system according to an exemplary embodiment of the present invention.

First, when a user requests processing of user authentication to attempt to access the security application device 300 through the user terminal 100 (S10), the user who operates the user terminal 100 in the authentication processing device 200 The user authentication process for.

That is, the authentication processing apparatus 200 allows the user terminal 100 to photograph the user twice and transmit them as the first image and the second image, and simultaneously receive and transmit the password. Accordingly, the user terminal 100 immediately photographs and transmits the first image, instructs the user to input a password, and then transmits the password received from the user, and also takes a predetermined time interval from the time when the first image is captured. After taking the picture, the second image is generated and transmitted to the authentication processing apparatus 200.

After obtaining the transmitted first image, the second image, and the password (S20, S30, S40), the authentication processing apparatus 200 determines whether the image is suitable for face recognition of the first image and / or the second image, Number authentication for the password is performed (S50). If the first image and / or the second image is determined to be a suitable image and the password number authentication is passed, face recognition is performed using the first image and / or the second image (S60).

If the face recognition succeeds, the user authentication process is determined to be regarded as an access by a legitimate user, and the access to the security application device 300 requested by the user is allowed (S70).

In this case, in the above-described step, the time point at which the first image is taken, the time point at which the password is input, and the time point at which the second image is taken may be changed in any order.

In addition, it is also possible to perform a suitable video determination on only one of the first video and the second video.

In addition, in the above-mentioned steps, the viewpoint of determining the suitable video, the viewpoint of number authentication, and the viewpoint of face recognition may be changed in arbitrary order with each other.

In face recognition, face recognition may be performed by a first image or by a second image, and face recognition may be performed by generating an average image of two images.

In addition, the first image and / or the second image may be generated by capturing the photographed image so that the user does not recognize the photographing moment when the user does not recognize the photographed image.

3 is a view for explaining the user authentication method as described above in more detail.

First, the user terminal 100 transmits a user authentication request for access to the security application apparatus 300 to the authentication processing apparatus 200 (S112). The authentication processing apparatus 200 requests the user terminal 100 to provide a plurality of pieces of authentication information necessary for user authentication in order to perform user authentication processing before allowing access (S114).

In response to this request, the user terminal 100 immediately transmits a first image mainly photographed of the user's face (S122). In addition, the user is guided to input the password, and transmits the password entered by the user (S132). In addition, the second image is generated by re-photographing the user at a predetermined time interval from the point of time when the first image is captured, and the generated second image is transmitted (S142).

In this case, the second image may be photographed while the user inputs a password, or may be photographed after the first image is photographed but before the user inputs the password. As such, the time interval for generating the second image may be arbitrarily set.

Next, when the user terminal 100 transmits the first image, the second image, and the password, the authentication processing apparatus 200 performs a process for user authentication based on this information.

First, the authentication processing apparatus 200 performs a suitable image determination process using either or both of the first image and the second image (S152).

The suitable image determination process determines whether the first image and the second image, which are images generated by photographing the user, are photographed to visually identify the user or to display a clear face shape to be used for face recognition. will be.

Here, the image photographed to be used for face recognition becomes a suitable image, and the image photographed to be unable to be used for face recognition becomes an inappropriate image.

A detailed method of determining a good image will be described with reference to FIGS. 4 to 7.

4 is a flowchart illustrating a procedure of a method for determining a suitable image by a user authentication system according to an exemplary embodiment of the present invention. First, the authentication processing apparatus 200 receives one of the first image and the second image photographing the user as a user image (S210). In operation S220, a user face image including only a face portion of the user is obtained from the provided user image. When the user's face image is obtained, the image of each part is extracted from each of the parts (eye, nose, mouth, eyebrow, etc.) forming the face form from the user's face image (S230).

Subsequently, shape information for each region is calculated by specifying and quantifying a feature point of a facial region included in the region-specific image, or generating and digitizing a vector representing the shape of the facial region from the region-specific image extracted for each region. (S240). When the shape information for each part is calculated, the information is compared with the shape information for each part (template data pre-stored for each face part) based on this information, and a similarity indicating the degree of correspondence with each part is calculated, and the similarity is calculated. The identifiable fit ratio for each site is calculated based on the step (S250).

The discriminable fit ratio for each part is that a face image included in a user image generated by photographing each part of the face at an appropriate angle with respect to the camera, under appropriate lighting, at an appropriate resolution, and used for face recognition for user identification is used. It is a measure indicating whether or not the user can be identified by the extracted user face image.

On the other hand, when the identifiable fit rate for each part is calculated for each part, the size-specific identifiable fit rate for each part is compared with the standard fit rate for each part, which is a preset standard fit ratio for each corresponding part. S260). At this time, if it is determined that the calculated identifiable fit ratio for each part is larger than the corresponding standard fit ratio for each part, it is determined that the user image is a suitable image that can be used for face recognition, and subsequent user authentication is continued (S270). If at least any one of the part-identifiable fit ratios calculated for each part is smaller than the corresponding part-specific standard fit rate, it is determined that the user image is an unsuitable image that is unclearly captured so that it cannot be used for face recognition. (S280). If it is determined that the image is not suitable, the process proceeds to FIG.

At this time, the determination of the unsuitable image may occur because the user's face may not be able to temporarily adequately capture due to the influence of the surrounding environment, so that a predetermined guide is displayed to the user to induce re-shooting or conform to the criteria for each part of the corresponding part. The value of the rate may be reduced and compared again, or the process may be continued with the relaxed fit image determination step (C). Step C corresponds to FIG. 6.

As an example of such a situation, first, a face such as an eyebrow, an eye, an ear, etc. when a user is photographed by a photographing means while covering the nose and mouth with a mask or the like while setting the standard fitting ratio for each part as 70%. The part may be used for face recognition, but since the nose and mouth cannot be photographed, each identifiable fit ratio is 0%, and thus the user image is determined to be an inappropriate image.

On the other hand, when the surrounding environment where the user is photographed is dark or reflected light is generated by the user, or when the user wears a hat or the like, the specific face part of the user is not accurately photographed, and thus the identifiable fit ratio is low for the specific part. (For example, the identifiable fit rate for the left eyebrow can be calculated as 70%), in which case, by reducing and recompiling the baseline fit for the eyebrow area by 60% Give you the flexibility to deal with these situations.

Here, it is preferable that the standard conformance ratio for each part is set to a lower limit of a value that can identify the user by identifying each face part at least by visual observation.

FIG. 5 is a flowchart for explaining a procedure for calculating the discriminable fit ratio for each part in FIG. 4. First, when part-specific shape information of at least one face part is calculated from the user's face image (S214; corresponding to S240), predetermined template storage means may store the template data for each part previously set and maintained in correspondence with the corresponding part ( Not shown) (S215), and comparing the acquired template data for each site with the calculated site-specific shape information (S216) to calculate the similarity, and based on the similarity, the recognizable fit rate for each site for the corresponding site It is calculated (S217).

6 is a flowchart for explaining a procedure of determining a suitable image by combining the identifiable fit ratios for a plurality of sites. The procedure of FIG. 6 is applied as step C of FIG. 4, and may be used as a method of applying a relaxed criterion when it is determined as an inappropriate image.

First, an identifiable fit rate for each part calculated for each part constituting the user's face is calculated (S225), and the identifiable fit rate for each part calculated for two or more parts is calculated in a specific arithmetic formula. By combining, the combination suitability is calculated (S226).

Here, the arithmetic formula for calculating the combination fit ratio can be arbitrarily set. In other words, a combinational fit rate may be calculated by a simple sum or by weighting each fit rate and adding the discriminable fit rate by a plurality of parts, or by combining various arithmetic operations or through more complex arithmetic expressions. You can also calculate the rate. The calculated combination fit rate may further include information indicating which face parts can be combined with each other.

The combination fit rate calculated as described above is compared with the reference combination fit rate set for each part used in the combination (S227). When it is determined that the calculated combination fit rate is greater than the reference combination fit rate, the user face image of the user image is displayed. If it is determined that the user is a suitable image that can be identified (S228), if it is determined that the user is small, it is determined that the user image is not suitable (S229).

Further, if it is determined that the image is not suitable, the user can be provided with guidance of not being able to retake or authenticate the user, or can proceed to the further relaxed suitable image determination step (B). The relaxed step D corresponds to FIG. 7.

For example, in this case, the combined fit rate is calculated by adding and averaging the identifiable fit rates of the respective parts of the left eyebrow and the right eyebrow, the left eye and the right eye in the user's face image, and the calculated combined fit rate is If the eyebrows and both eyes are larger than a preset standard combination fit ratio, it is determined that the user's identity can be identified by the naked eye or that the user's face image can be used for face recognition.

In other words, when the user wears sunglasses or the like and both eyes are almost covered, the fit ratio of the left eyebrow is 80%, the fit ratio of the right eyebrow is 90%, and the fit ratio of the left and right eyes is 10%. have. At this time, the combined suitability calculated by a simple sum and an average is 47.5%. On the other hand, if the reference combination fit ratio for both eyebrows and eyes is set to 60%, it is determined that the user face image generated by photographing the user is an inappropriate image.

On the other hand, if the user puts on one eye due to eye disease or the like, the fit rate of the left eyebrow is 80%, the fit rate of the right eyebrow is 90%, the fit rate of the left eye is 80%, and the fit rate of the right eye is 0% can be determined, and the combined fit ratio can be 62.5%, which can be determined as a suitable image.

When the combination fit rate is applied in this way, even when a part of the face part of the user is inevitably covered by a disease, an ornament, or the surrounding environment, the user can be identified.

FIG. 7 is a flowchart illustrating a procedure of determining whether a suitable image is obtained after combining all identifiable fit ratios for all face parts. The procedure of FIG. 7 may be used as a method of applying a relaxed criterion when it is determined as an inappropriate image in the procedure of FIG. 6. It may also be applied as step C of FIG.

First, an identifiable fit ratio for each part is calculated for all parts constituting the face (parts used for face recognition) (S235), and all of the identifiable fit ratios for all parts are all determined by a predetermined arithmetic formula. In combination, the total fit ratio is calculated (S236). The calculated total fit ratio is compared with the reference total fit ratio (S237). If the total fit ratio is greater than the reference total fit ratio, the user's face image is determined as a suitable image (S238). It determines (S239). When the inappropriate image is determined as described above, a predetermined guide is output to notify that the user face image generated by photographing the user cannot be used for face recognition for user identification.

By using the total fit ratio for the entire part of the face as described above, similarly to using the combination fit ratio described above, a flexible response to the occlusion of the user's face is possible.

By determining whether or not it is a suitable image as described above, it is possible to determine whether a part or all of the photographed user's face is photographed clearly and at a precise angle to the extent that the face can be recognized. In addition, it is also possible to determine whether part or all of the user's face is covered by an illegal purpose.

Therefore, in case the user who wants to access the security device 300 is covering the face illegally, the administrator cannot restrict the use of the device smoothly or report it to the security center or the like to determine whether the administrator uses the device incorrectly. Processing such as notifying to confirm is possible.

On the contrary, even when a part of the face is hidden, the user can be smoothly identified by using the remaining part of the normal photographing, so that user authentication by face recognition can continue even if the face is not completely seen. You can do that.

Meanwhile, in the flow of FIG. 3, when it is determined that the video is not suitable for the first image and / or the second image, the process proceeds to procedure A, which is referred to FIG. 8.

That is, when one or all of the captured images are determined to be inappropriate images (S303), a predetermined warning or guide message may be displayed to the user through the user terminal 100 (S304). Such a warning message may show an indication of a face part determined as an inappropriate image, and to remove the prop covering the part or to operate the user terminal 100 under brighter light, or to accurately maintain a facial expression or the like. , Or guide the user to remove the stolen image and try to retake it.

In this case, when the user responds to the reshooting (S305), the user retakes the first image and / or the second image (S306), and repeats the determination of the suitable image (S307). If it is rejected, it is regarded as a failure of user authentication and the user authentication procedure ends (S308).

On the other hand, if it is determined that at least one procedure of number authentication using a password and face recognition by a user image is determined to fail, the procedure B is performed, which is described with reference to FIG. 9.

That is, if it is determined that the authentication failed in the user authentication procedure (S313), the user is asked whether or not to retry the user authentication procedure (S314), and when attempting to reauthenticate, the process returns to step S114 to the user terminal 100 side. Re-request information for user authentication (S315), and if the re-authentication is rejected, it is determined that the user authentication has failed and ends.

As described above, the user authentication method according to the embodiment of the present invention may further include determining whether the user steals a legitimate user's face image. That is, it is determined that face recognition is attempted in an unfair manner using the first image and / or the second image generated by photographing the user.

10 is a flowchart illustrating a method of determining whether the user image is stolen. Here, a method of determining a case where a user's face attempts to identify a user using a photographed picture is disclosed.

First, referring to FIG. 10, in the authentication processing apparatus 200, a background image of a background excluding a face and a body part of a user is selected from one of a first image and a second image (called a user image) photographed and provided. Extracting (S351), and extracting a linear form including a straight line and / or artificially formed curve from the extracted background image (S352). Preferably, the straight form is extracted. The extracted line shape is analyzed to determine whether the user image is a stolen image or a normal image generated by directly photographing an actual user (S353).

11 and 12 are flowcharts illustrating the method for determining whether the image is stolen in FIG. 10 in detail.

First, FIG. 11 detects a state in which all edges of a stolen image, such as a photograph of a user, are shown in a user image.

First, the authentication processing apparatus 200 determines whether the linear form extracted from the background image forms a closed figure surrounding the user's face (S363, S364). In this case, when there is a closed figure surrounding the face of the user, the outer edge of the output, such as an ID photo including the face of the user and the body part of the user, or the edge of the display means on which the user's face is displayed, Since it means a state in which all of the images are captured in the photographing area (see FIG. 14), it is determined that an image of the user is being stolen by an illegal user (S365). If not, it is determined as a normal image (S366).

Next, FIG. 12 detects a state in which at least the lower edge of the stolen image is seen in the user image.

First, the authentication processing apparatus 200 extracts a body outline representing the outline of the user's body (S372), and determines whether the bottom line of the extracted body outline overlaps the bottom edge of the user's image. Examine (S373, S374). As a result of the examination, when it is confirmed that the bottom line of the extracted body outline does not overlap with the bottom border of the captured user image, it is determined that the stolen image is captured (S375). On the other hand, when the bottom line of the extracted body outline overlaps the bottom edge of the captured user image, it is determined that the user is normally photographed (S376).

In this case, as shown in FIG. 15, when an unjust user steals a photograph of a legitimate user's face and / or body part and approaches the photographing means, the lower edge of the stolen image is approached. Is a case where the image is viewed in the photographing area of the photographing means. For example, at least part of the user's chest is not shown because the ID photo is shown as a straight line in the outline of the body. If the straight line does not coincide with the lower edge of the user's image, the image is clearly displayed. It may be stolen.

The procedure for determining the image theft as described above will be described with reference to FIGS. 13 to 115.

First, FIG. 13 shows a normal image captured by a camera normally by a legitimate user in a state where all faces are exposed. There is no closed figure surrounding the user's face, and the bottom line of the body outline coincides with the bottom edge of the user image.

14 illustrates a state in which an invalid user attempts to identify a user by using a photograph taken by a valid user. At this time, a rectangular line shape (a portion indicated by a red rectangle) was extracted in a form surrounding a face portion (a portion indicated by a green circle) of the user image, and this state is determined as an image stolen by the procedure of FIG. 11.

Next, FIG. 15 illustrates a state in which an unjust user attempts an illegal access to a picture of a legitimate user. Here, the body outline of the lower part of the chest of the user's body outline does not overlap the bottom edge of the user image. It is shown. This state is determined to be the stolen image by the procedure of FIG.

Meanwhile, another method for determining the case where the stolen image is used will be described with reference to FIGS. 16A and 16B.

First, the authentication processing apparatus 200 extracts and maintains a plurality of line shapes from the surrounding image by photographing the front in the state where there is no user at the current position of the user terminal 100. Subsequently, when the user approaches for user authentication, if the line shape of the background image extracted from the user image does not match the line shape from the surrounding image previously held, it may be determined that the stolen image is applied.

FIG. 16A illustrates an example of the surrounding image and the extracted line form, and FIG. 16B illustrates an example of the user image and the extracted line form.

According to the user authentication method including the above-described process and the user authentication system implementing the method, the user is photographed twice to generate an image for user authentication, and each image is first checked whether it is photographed to the extent possible for face recognition. Done. Therefore, since the face recognition failure due to the inappropriate image can be determined in advance, it is possible to determine when the face is randomly hidden or when the face is obscured, and thus the case where the face recognition fails inevitably fails due to this masking. Will be.

In addition, since user authentication is performed by interworking a password and face recognition, higher security can be provided.

In addition, since the case where the video is stolen can be determined, it is possible to reliably prevent access by an unjust user.

If the user authentication system and method according to the present invention as described above are applied to a home automation system, the following settings are possible.

First, a user transmits a request to access a home automation system using a specific access code using his or her user terminal 100, for example, a mobile phone, through a communication network. The access request thus sent arrives at the authentication processing apparatus 200 serving as a gateway for relaying the information of the home automation, and the authentication processing apparatus 200 performs user authentication processing for the user who requested access. If the user authentication is successful by such processing, the user can access the home automation system and perform desired home control.

1 is a block diagram schematically showing the configuration of a user authentication system according to an embodiment of the present invention.

2 is a flowchart schematically illustrating a user authentication method by a user authentication system according to an exemplary embodiment of the present invention.

3 is a view for explaining the user authentication method as described above in more detail.

4 is a flowchart illustrating a procedure of a method for determining a suitable image by a user authentication system according to an exemplary embodiment of the present invention.

FIG. 5 is a flowchart for explaining a procedure for calculating the discriminable fit ratio for each part in FIG. 4.

6 is a flowchart for explaining a procedure of determining a suitable image by combining the identifiable fit ratios for a plurality of sites.

FIG. 7 is a flowchart illustrating a procedure of determining whether a suitable image is obtained after combining all identifiable fit ratios for all face parts.

FIG. 8 is a flowchart for explaining a procedure after A in the flowchart of FIG. 3.

FIG. 9 is a flowchart for explaining the procedure after the B procedure in the flowchart of FIG. 3.

10 is a flowchart for explaining a method of determining whether a user image is stolen.

11 and 12 are flowcharts illustrating a method for determining whether the image is stolen in FIG. 10.

FIG. 13 is a diagram illustrating a normal image captured by a legitimate user with all faces exposed.

14 is a diagram illustrating a state in which an invalid user attempts to identify a user by using a photograph taken by a legitimate user.

15 is a diagram illustrating a state in which an illegal user attempts an illegal access to a picture of a legitimate user.

16 (a) and 16 (b) are diagrams for explaining another method of determining the case where the stolen image is used.

Claims (17)

delete delete A user authentication method that simultaneously uses number authentication by a password entered by a user and face recognition by a user image photographing a user: Receiving a user authentication request from the user, Photographing the user to obtain a first image; Receiving a password from the user; Re-photographing the user to obtain a second image; Performing number authentication using the input password; Extracting a face region image of a specific face region from the photographed first image, the second image, or the first image and the second image; Calculating and calculating the shape information of the corresponding face part from the extracted face part image; Calculating similarity by comparing the calculated shape information with template data stored in advance with respect to the face part; Determining that the first image, the second image, or the first image and the second image are suitable images applicable to face recognition when the calculated similarity is equal to or greater than a predetermined reference similarity; Performing face recognition using the first image, the second image, or the first image and the second image when it is determined to be a suitable image; And if both the result of the number authentication and the result of the face recognition pass, determining that the user authentication succeeds. A user authentication method that simultaneously uses number authentication by a password entered by a user and face recognition by a user image photographing a user: Receiving a user authentication request from the user, Photographing the user to obtain a first image; Receiving a password from the user; Re-photographing the user to obtain a second image; Performing number authentication using the input password; Extracting a face part image of a plurality of face parts from the photographed first image, the second image, or the first image and the second image; Digitizing and calculating shape information of a corresponding facial part from each of the extracted facial part images; Calculating similarity for each part by comparing the calculated shape information with template data previously stored for the corresponding face part; Calculating the combined similarity by combining the calculated similarity for each part by a predetermined arithmetic expression, Determining that the first image, the second image, or the first image and the second image are suitable images applicable to face recognition when the calculated combined similarity is equal to or greater than a predetermined reference combined similarity; Performing face recognition using the first image, the second image, or the first image and the second image when it is determined to be a suitable image; And if both the result of the number authentication and the result of the face recognition pass, determining that the user authentication succeeds. A user authentication method that simultaneously uses number authentication by a password entered by a user and face recognition by a user image photographing a user: Receiving a user authentication request from the user, Photographing the user to obtain a first image; Receiving a password from the user; Re-photographing the user to obtain a second image; Performing number authentication using the input password; Extracting respective face part images of all face parts from the first image, the second image, or the first image and the second image which are photographed; Digitizing and calculating shape information of the corresponding face part from each of the extracted face part images; Calculating similarity for each part by comparing the calculated shape information with previously stored template data for the corresponding face part; Calculating the total similarity by combining all of the calculated similarities for each part by a predetermined arithmetic expression, Determining that the first image, the second image, or the first image and the second image are suitable images applicable to face recognition when the calculated total similarity is equal to or greater than a predetermined reference total similarity; Performing face recognition using the first image, the second image, or the first image and the second image when it is determined to be a suitable image; And if both the result of the number authentication and the result of the face recognition pass, determining that the user authentication succeeds. delete delete A user authentication method using both number authentication by a password input by a user and face recognition by a user image photographing a user at the same time: Receiving a user authentication request from the user, Photographing the user to obtain a first image; Receiving a password from the user; Re-photographing the user to obtain a second image; Performing number authentication using the input password; Determining whether the first image, the second image, or the first image and the second image are suitable images applicable to face recognition; Performing face recognition using the first image, the second image, or the first image and the second image when it is determined to be a suitable image; Extracting a background image from the first image, the second image, or the first image and the second image, extracting at least one linear form from the extracted background image, and the extracted linear form is the first image. In the case of forming the closed figure surrounding the face of the user in the image, the second image, or the first image and the second image, the first image, the second image, or the first image and the second image Determining whether the image is stolen from the stolen image; If it is determined that the first image, the second image, or the first image and the second image are generated from the stolen image, determining that the user authentication has failed, otherwise determining that the face recognition is possible; And if it is determined that the face can be recognized and both the result of the number authentication and the result of the face recognition pass, determining that the user authentication succeeds. The method of claim 8, The determining whether the image is theft is: Extracting a user body outline from the first image, the second image, or the first image and the second image; When the shape of the bottom of the extracted user's body outline does not coincide with the first image, the second image, or the bottom edge of the first image and the second image, the first image and the second image And determining that an image, or the first image and the second image are generated from the stolen image. The method of claim 8, The determining whether the image is theft is: Extracting respective background images from the first image and the second image, Extracting at least one line shape from each of the extracted background images; When the linear form extracted from the background image of the first image and the linear form extracted from the background image of the second image do not coincide with each other, the first image, the second image, or the first image and the second image Determining that the image was generated from a stolen image. The method of claim 8, The determining whether the image is theft is: Extracting a background image from which the user's body shape is removed from the first image, the second image, or the first image and the second image; Extracting at least one line shape from the extracted background image; When the extracted line shape does not match the reference line shape extracted from the surrounding image photographed without the user at the current position, the first image, the second image, or the first image and the second image are stolen. And determining that it is generated from the image. delete A user authentication system which considers success of user authentication when a user who wants to access a certain security device simultaneously passes number authentication using a password entered and face recognition by a user image photographing the user. A user terminal for transmitting the first image photographing the user, a second image rephotographing the user at a predetermined time interval from a time point at which the first image is photographed, and a password received from the user; The number authentication is performed on the received password, and the shape information of a specific face part is calculated by numerically calculating the first image, the second image, or the first image and the second image, and the calculated form information Similarity is calculated by comparing template data stored in advance with respect to the corresponding face part, and when the calculated similarity is equal to or greater than a predetermined reference similarity, the first image, the second image, or the first image and the second image. It is determined that the image is a suitable image applicable to face recognition, face recognition is performed using the first image, the second image, or the first image and the second image determined as a suitable image, and the suitable image is determined. When both the result, the number authentication result and the face recognition result pass, it is determined that the user authentication is successful and the user User authentication system comprising an authentication apparatus to gain access to the device. A user authentication system which considers success of user authentication when a user who wants to access a certain security device simultaneously passes number authentication using a password entered and face recognition by a user image photographing the user. A user terminal for transmitting the first image photographing the user, a second image rephotographing the user at a predetermined time interval from a time point at which the first image is photographed, and a password received from the user; Upon receiving the first image and the second image, it is determined whether the first image, the second image, or the first image and the second image are suitable images applicable to face recognition, and the Number authentication is performed, and the respective shape information of a plurality of face parts in the first image, the second image, or the first image and the second image is numerically calculated, and the calculated shape information and corresponding Comparing the template data stored in advance with respect to the face parts to calculate the similarity for each part, and calculate the combined similarity by combining the calculated similarity for each part by a predetermined arithmetic formula, and the calculated combined similarity Is a suitable image to which the first image, the second image, or the first image and the second image are applicable to face recognition when is equal to or greater than a predetermined reference combination similarity. And face recognition using the first image, the second image, or the first image and the second image determined as a suitable image, and the fit image determination result, the number authentication result, and the face. And an authentication processing device for determining that the user authentication succeeds when all of the recognition results have passed, so that the user can access the security application device. A user authentication system which considers success of user authentication when a user who wants to access a certain security device simultaneously passes number authentication using a password entered and face recognition by a user image photographing the user. A user terminal for transmitting the first image photographing the user, a second image rephotographing the user at a predetermined time interval from a time point at which the first image is photographed, and a password received from the user; Number authentication is performed on the received password, and the respective shape information of all the face parts of the first image, the second image, or the first image and the second image is numerically calculated and calculated, respectively. Comparing the template information stored in advance with respect to the shape information of each of the face parts and calculates the similarity for each part, and calculates the total similarity by combining all of the calculated similarity for each part by a predetermined arithmetic formula The first image, the second image, or the first image and the second image are determined to be suitable images applicable to face recognition when the calculated total similarity is equal to or greater than a predetermined reference total similarity. The face recognition is performed using the first image, the second image, or the first image and the second image determined to be, and the fitted image plate Result and the code verification result and the user authentication system by the face recognition result is determined as the user authentication is successful when the pass all characterized in that it includes a verification processing unit that allows the user to have access to the secure application device. delete The method according to any one of claims 13 to 15, The authentication processing device, It is determined whether the first image, the second image, or the first image and the second image are photographed and generated from an image stolen by an illegal user, and the first image, the second image, or the first image is generated. And if it is determined that the first image and the second image are generated from the stolen image, the user authentication system is determined to fail.

Images