JP7402940B2 - User authentication methods, systems and programs - Google Patents

Description

Application of Article 30, Paragraph 2 of the Patent Act Published on October 23, 2019 on the homepage listed in the certificate for application of the exceptions to loss of novelty of invention.

The present invention relates to a method and system for authenticating users.

As technology develops, the use of electronic devices is increasing. In particular, electronic devices are utilized as means for authenticating users in various situations.

For example, there is a need to authenticate true users in various situations, such as confirming the attendance of participants at a conference, confirming the arrival of a patient at a hospital, and confirming users for payment at a cafe. The use of electronic devices in situations where user authentication is required in this way has the effect of reducing unnecessary labor and time required for user authentication.

There are various user authentication methods that utilize electronic devices. For example, users can directly identify themselves by providing identification information (e.g., phone number, email address, etc.) or biometric information (e.g., fingerprint information, etc.) to the electronic device. There is a method in which authentication is performed by inputting the following information:

In recent years, much research has been conducted on methods for more convenient and intuitive user authentication without the user directly inputting information into an electronic device.

The present invention provides a user authentication method and system that can grasp the user's intention and perform user authentication more intuitively.

Further, the present invention provides a user authentication method and system that allows user authentication to be performed without the user directly inputting information.

Furthermore, the present invention provides a user authentication method and system that can complete user authentication while grasping the user's true intention as to whether or not to perform user authentication.

In order to solve the above problems, a user authentication method according to the present invention includes the steps of: detecting a facial image of a target user using a camera; extracting user information corresponding to the facial image from a user database; performing user authentication for the target user using user information; in the step of performing user authentication, the user authentication is completed based on a gesture of the target user detected by the camera; It is characterized by determining whether or not.

Further, the user authentication system according to the present invention includes a display unit, a camera that detects a face image of a target user, and a control unit that extracts user information corresponding to the face image from a user database, and the control unit includes: The present invention is characterized in that user authentication is performed for the target user using the user information, and it is determined whether or not to complete the user authentication based on the gesture of the target user detected by the camera.

Further, a program storable on a computer-readable medium according to the present invention is executed by one or more processes on an electronic device, and includes the steps of: detecting a facial image of a target user using a camera; and detecting user information corresponding to said facial image. a command for executing a step of extracting from a user database and a step of performing user authentication for the target user using the user information, and in the step of performing user authentication, the gesture of the target user detected by the camera; The method is characterized in that it is determined whether or not to complete the user authentication based on the above.

As described above, in the user authentication method and system according to the present invention, a camera is used to detect a user who is a target of user authentication, and a facial image of the detected user is used to perform user authentication for the detected user. It can be performed. In this way, according to the present invention, user authentication can be performed based on the fact that the user is near the electronic device, without the user having to directly input information. Therefore, the user can more conveniently and intuitively communicate his/her intention for user authentication to the electronic device simply by being in the vicinity of the electronic device.

Further, in the user authentication method and system according to the present invention, it is possible to understand the user's intention regarding user authentication based not only on the detected face image of the user but also on the user's gesture. Therefore, in the user authentication method and system according to the present invention, the authenticity of the user who is attempting to perform user authentication can be ascertained based on the user's gesture, and the user authentication can be performed based on the user's intention.

1 is a conceptual diagram for explaining a user authentication system according to the present invention. 3 is a flowchart for explaining a user authentication method according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention. 1 is a conceptual diagram for explaining a user authentication method and system according to the present invention.

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings, but the same or similar components will be denoted by the same reference numerals regardless of the drawing numbers, and redundant explanation thereof will be omitted. The suffixes "module" and "part" used in the following explanations are given or mixed to facilitate the preparation of specifications, and they themselves have significance and usefulness. It's not something you have. Furthermore, when describing the embodiments of the present invention, if it is determined that detailed explanation of related known techniques would obscure the gist of the embodiments of the present invention, the detailed explanation will be omitted. Furthermore, the attached drawings are only for helping understanding of the embodiments of the present invention, and the technical idea of the present invention is not limited by the attached drawings, and any changes that fall within the idea and technical scope of the present invention may be made. , should be understood to include equivalents or substitutes.

Terms including ordinal numbers such as "first", "second", etc. are used to describe various components, but the components are not limited to the above terms. These terms are only used to distinguish one component from another.

When a component is referred to as being "coupled" or "connected" to another component, it may be directly coupled or connected to the other component, and there may be other components in between. It should be understood that it is permissible to do so. In contrast, when an element is referred to as being "directly coupled" or "directly connected" to another element, there are no intermediate elements present.

Singular expressions include plural expressions unless otherwise specified.

As used herein, terms such as "comprising" and "having" are intended to specify the presence of features, numbers, steps, acts, components, parts, or combinations thereof described herein. It should be understood that this does not exclude in advance the presence or possibility of adding one or more other features, figures, steps, acts, components, parts or combinations thereof.

The present invention is intended to provide a user authentication method and system that can grasp the user's intention and perform user authentication more intuitively.The present invention will be specifically described below with reference to the accompanying drawings. explain. FIG. 1 is a conceptual diagram for explaining a user authentication system according to the present invention, and FIG. 2 is a flowchart for explaining a user authentication method according to the present invention. 3a, 3b, 4a, 4b, 5, 6a, 6b, 6c, 7a, and 7b are conceptual diagrams for explaining the user authentication method and system according to the present invention.

As shown in FIG. 1, a user authentication system 100 according to the present invention may include at least one of a communication unit 110, a storage unit 120, and a control unit 130. Here, the user authentication system 100 according to the present invention is not limited to the above components, but may further include components that play the same or similar functions as those described in this specification.

Note that the user authentication system 100 according to the present invention may operate in conjunction with the electronic device 10. Here, the electronic devices 10 include smartphones, mobile phones, tablet PCs, kiosks, computers, notebook computers, digital broadcasting terminals, PDAs (Personal Digital Assistants), and PMPs (Portable
Multimedia Player).

Further, the user authentication system 100 according to the present invention may be realized by an application or software. Here, the application or software of the user authentication system 100 may be installed on the electronic device 10 to perform a process according to the user authentication method described below.

For example, the communication unit 110, storage unit 120, and control unit 130 of the user authentication system 100 according to the present invention may use the hardware configuration of the electronic device 10 to execute a series of functions for user authentication. . For example, the communication unit 110 uses the configuration of a communication module (for example, a mobile communication module, a short-range communication module, a wireless internet module, a location information module, a broadcast receiving module, etc.) included in the communication unit of the electronic device 10. Data may be exchanged with at least one external server (for example, the database server (DB server) 140 shown in FIG. 1, a cloud server, etc.).

The storage unit 120 may store data related to user authentication according to the present invention using a memory included in (or inserted into) the electronic device 10 or a cloud server.

The control unit 130 may use at least one of a CPU and a cloud server provided in the electronic device 10 to control each configuration and perform overall control according to the present invention. Furthermore, the control unit 130 may have control authority over at least one component of the electronic device 10 in order to realize the present invention. Further, the control unit 130 may use information received or detected by the components of the electronic device 10 to implement the present invention. For example, the control unit 130 may use information received or detected by the components of the electronic device 10. User authentication according to the present invention may be performed using the received image.

Furthermore, the control unit 130 may output information according to the present invention using the display unit 12 of the electronic device 10.

In addition, the control unit 130 performs user authentication according to the present invention using other components included in the electronic device 10 (for example, a proximity sensor, an infrared sensor, a touch screen, an input unit, etc.). Good too.

Note that the user authentication system 100 according to the present invention may be in the form of not only a program or an application, but also various forms capable of executing the configuration and functions described below.

Therefore, the user authentication system 100 according to the present invention will be described below without placing strict limitations on these.

As shown in FIG. 1, a user authentication system 100 according to the present invention may detect a user using a camera 11 of an electronic device 10 and perform user authentication for the detected user.

For convenience of explanation, in the present invention, a user who is a target of user authentication is referred to as a "target user."

Furthermore, in the present invention, "user authentication" means user authentication for a target user.

In the present invention, "user authentication" may be used to approve or execute "events or functions subject to user authentication (hereinafter collectively referred to as "target functions")".

Here, the "target function" has a wide variety of types, and means a function that requires user authentication, such as a payment function, an attendance confirmation function, a hospital visit confirmation function, and the like.

In the present invention, the control unit 130 acquires the face image of the target user 1000 from the image received from the camera 11, and uses the user information corresponding to the face image stored in the user database (user DB) to acquire the face image of the target user 1000. User authentication for 1000 may be performed.

Here, the target function to be subjected to user authentication may be specified in advance. The control unit 130 may perform user authentication related to the target function using the face image of the target user 1000 and user information extracted from the user database. As an example, when the target function is an attendance confirmation function, the control unit 130 may perform either an attendance confirmation approval process or an attendance confirmation rejection process for the target user 1000.

As another example, when the target function is a payment function, the control unit 130 may perform either payment approval processing or payment rejection processing for the target user 1000.

Here, the control unit 130 may extract the face image of the target user 1000 from the image received from the camera 11, and extract user information corresponding to (or matching) the extracted face image from the user database. good.

Here, the user information stored in the user database may be referred to as "user-related information," "user identity information," or "user identification information."

The user database may be included in at least one of the storage unit 120, the database server 140, and the cloud server.

The control unit 130 may search the user database for a person corresponding to the face image of the target user 1000, and extract user information that matches the searched person.

The user database may store user information regarding each individual person (or persons). That is, the user database may store matching user information corresponding to different people.

Here, the user information includes the name, date of birth, address, telephone number, email address, account information, employee number, ID, facial image, biometric information (fingerprint information, iris information, etc.) of the person corresponding to each person. ), a reference face image, electronic device identification information, and schedule information.

Here, the reference face image is a face image of a specific person, and may be one or more. In the present invention, the "reference face image" may be referred to as "reference data."

The control unit 130 compares the target user's face image acquired by the camera 11 with a reference face image stored in the user database, and searches for a person who matches the reference face image corresponding to the target user's face image. You can do it like this. Further, the control unit 130 may specify the searched person as a target user, and extract user information matching the searched person from the user database.

Furthermore, the control unit 130 may use the extracted user information to perform user authentication regarding the target function.

As shown in FIG. 1, the electronic device 10 can be placed at any location, and the target user 1000 can perform user authentication just by being around the electronic device 10. Here, various target functions for user authentication can be set depending on the location where the electronic device 10 is placed.

As an example, if the place where the electronic device 10 is placed is a cafe, the target function may be specified as a payment function. In this case, the user authentication system 100 according to the present invention may be programmed such that payment functions are performed based on user authentication.

As another example, if the place where the electronic device 10 is placed is a hospital, the target function may be specified as a function for checking patients (or customers in the case of a store) who have visited the hospital. In this case, the user authentication system 100 according to the present invention may be programmed to perform the visit confirmation function based on user authentication.

The user authentication method according to the present invention will be described in more detail below with reference to flowcharts.

As shown in FIG. 2, in the user authentication method according to the present invention, first, a process of detecting a target user's face image from an image received from the camera 11 (see FIG. 1) is performed (S210).

That is, the control unit 130 may detect the target user's face image using the camera 11 of the electronic device 10.

More specifically, as shown in (a) of FIG. 3A, the control unit 130 may detect a face image of the target user 1000 around the electronic device 10 using the camera 11.

The control unit 130 acquires a face image of the target user 1000 from the image received from the camera 11, and performs user authentication for the target user 1000 using user information corresponding to the face image stored in the user database. It's okay.

The point in time at which the facial image of the target user 1000 is detected (or detected) by the camera 11 varies widely.

As an example, the control unit 130 controls the camera 11 to photograph the face of the target user 1000 when a target function (or a specific function) that requires user authentication is activated or executed.

Here, activation or execution of the target function may be performed based on a request from the target user 1000 or a third party, or may be performed under the control of the control unit 130.

For example, a request to activate or perform a target function that requires user authentication is received via an input unit (eg, a touch screen) of the electronic device 10, as shown in (a) of FIG. 3a. The target user 1000 or a third party can request activation of a specific function that requires user authentication via an input section of the electronic device 10. At this time, the touch screen of the electronic device 10 displays guide information (e.g., "touch to start payment") on how to input a request to the electronic device 10 to activate the target function, as shown. It may also be output. The target user 1000 or a third party can refer to the guide information and perform user input for user authentication (for example, touch input on a touch screen) to the electronic device 10.

Hereinafter, the terms display section and touch screen will not be differentiated, and will be collectively described as the display section. That is, it goes without saying that the display unit 12 described below may include a touch sensor.

On the other hand, upon receiving the activation request by the electronic device 10, the control unit 130 detects the face image of the target user 1000 from the image received from the camera 11, and uses the detected face image to perform user authentication. A series of processes may be performed.

In contrast, when it is detected that there is a person around the electronic device 10, the control unit 130 displays an image of the target user 1000 using the camera 10, as shown in (a) and (b) of FIG. The camera 10 may be controlled so that the information is received. Here, if the target user 1000 is within a predetermined range (or distance) from the electronic device 10, the control unit 130 may control the camera 10 to detect the face image of the target user 1000. The control unit 130 uses at least one sensor (for example, a proximity sensor, an infrared sensor, a lidar sensor, etc.) included in the electronic device 10 to place a target around the electronic device 10 or within a predetermined range from the electronic device 10. The presence of the user 1000 may be detected (or sensed).

On the other hand, as shown in FIG. 4A (a), when the target user 1000 is not within a predetermined range from the electronic device 10, the control section 130 controls the display so that the display section 12 of the electronic device 10 becomes inactive. The unit 12 may also be controlled. The display unit 12 in an inactive state may be in at least one of a power-off state, a standby state, and a power saving state.

Further, the control unit 130 controls the display unit so that the display unit 12 of the electronic device 10 becomes active when the target user 1000 is within a predetermined range from the electronic device 10, as shown in FIG. 4A (b). 12 may be controlled. At this time, guide information (for example, "touch to start payment") regarding a method for requesting activation of a target function that requires user authentication may be output to the display unit 12.

Alternatively, the control unit 130 may determine whether the image received from the camera 11 includes the face image of the target user 1000 at predetermined intervals or in real time. As a result of the determination, if the face image of the target user 1000 is included in the image received from the camera 11, a series of processes for user authentication related to the target function may be performed.

On the other hand, the control unit 130 may detect the face image of the target user 1000 from the image received from the camera 11 using at least one face recognition algorithm. The type of face recognition algorithm used in the present invention is not limited, and algorithms that operate based on artificial intelligence or deep learning can also be used.

On the other hand, even if the face image of the target user 1000 is detected by the camera 11, the control unit 130 determines whether the detected face image satisfies a predetermined condition (or a predetermined authentication condition) for performing user authentication. If not, guide information for acquiring a face image that satisfies predetermined conditions may be provided.

More specifically, the control unit 130 may determine whether the face image of the target user 1000 detected by the camera 11 satisfies a predetermined condition. Here, the predetermined conditions include the size of the face image included in the image, the angle of the face corresponding to the face image, the position of the face image in the image, and facial feature elements (e.g., eyes, nose, mouth) in the image. may be related to at least one of the following:

As shown in FIG. 4b, the control unit 130 may output at least one piece of information regarding the posture, position, etc. of the target user 1000 as guide information so that a face image that satisfies a predetermined condition can be obtained. . The guide information may relate to at least one of the position and distance between the camera 11 and the target user 1000.

For example, the control unit 130 may include, as guide information, i) location information of the target user 1000 with respect to the electronic device (for example, please approach, please move behind the ii) posture information of the target user 1000 (e.g., please look at the screen), as shown in FIG. 4b(c); iii) FIG. 4b(d) As shown in , when the face of the target user 1000 is hidden by an object (for example, a mask), information (for example, please remove the mask) that guides the removal of the object may be provided.

When the face image of the target user 1000 is detected by the camera 11 in this manner, the present invention performs a process of extracting user information corresponding to the detected face image from the user database (S220).

More specifically, the control unit 130 extracts the face image of the target user 1000 from the image received from the camera 11, and extracts user information corresponding to (or matches) the extracted face image from the user database. You can also do this.

As mentioned above, the user information stored in the user database may be referred to as "user-related information," "user identity information," or "user identification information."

The user database may be included in at least one of the storage unit 120, the database server 140, and the cloud server.

The control unit 130 may search the user database for a person corresponding to the face image of the target user 1000, and extract user information that matches the searched person.

The user database may store user information regarding each individual person (or persons). That is, the user database may store matching user information corresponding to different people.

Here, the user information includes the name, date of birth, address, telephone number, email address, account information, employee number, ID, facial image, biometric information (fingerprint information, iris information, etc.) of the person corresponding to each person. ), a reference facial image, electronic device identification information, schedule information, and authority information regarding a specific function (or specific event).

Here, the reference face image is a face image of a specific person, and may be one or more. In the present invention, the "reference face image" may be referred to as "reference data."

The control unit 130 compares the target user's face image acquired by the camera 11 with a reference face image stored in the user database, and searches for a person who matches the reference face image corresponding to the target user's face image. You can do it like this. Further, the control unit 130 may specify the searched person as a target user, and extract user information matching the searched person from the user database.

The control unit 130 may extract from the user database a specific reference face image having a degree of similarity that satisfies a predetermined condition with respect to the detected face image of the target user 1000. Further, the control unit 130 may specify a person who matches the extracted specific reference face image as the target user.

As shown in FIG. 3A (b), while the search for the target user 1000 is being performed, the control unit 130 displays guidance information regarding the current progress (for example, "Searching for you") on the electronic device. The display section 12 may be controlled so that the information is output to the screen 10. The control unit 130 may provide the display unit 12 with guidance information regarding the current progress of user authentication (eg, “Searching for you”).

As illustrated, an image taken by the camera 11 may be output to the display unit 12. At this time, the image output to the display unit 12 may include a facial image of the target user 1000.

On the other hand, as described above, when a person corresponding to the target user 1000 is specified and user information regarding the specified person (for example, Sato*Mi) is output, the control unit 130 outputs at least part of the user information. The information may be provided to the display unit 12.

That is, as shown in FIG. 3A (c), at least part of the user information (for example, name information (Sato*Mi)) corresponding to the person corresponding to the target user 1000 is output to the display unit 12. It's okay. Therefore, the target user 1000 can refer to the information output to the display unit 12 and check whether or not user authentication regarding the user himself/herself has been performed.

When the user information regarding the target user 1000 is extracted in this manner, the present invention performs a process of user authentication for the target user 1000 using the extracted user information (S230).

More specifically, the control unit 130 may use the extracted user information to perform user authentication regarding the target function. For example, as shown in FIG. 3A (c), if the place where the electronic device 10 is placed is a cafe, the target function may be specified as a payment function. In this case, the user authentication system 100 according to the present invention may be programmed such that payment functions are performed based on user authentication.

In the present invention, user authentication is performed in cooperation with a target function, and the control unit 130 determines the success or failure of user authentication by i) approving or rejecting, ii) activating or inactivating a specific function. iii) may be used as a control command for execution or non-execution.

For example, if the user authentication is successful, the control unit 130 may approve, activate, or execute the target function. Furthermore, if user authentication fails, the control unit 130 may deny, deactivate, or not execute a specific function.

On the other hand, the control unit 130 may control the display unit 12 so that information (target function information, specific function information) regarding the target function (or specific function) for which user authentication is performed is output to the display unit 12. good.

As shown in FIG. 3A(c), the display unit 12 displays user information regarding the person searched based on the face image of the target user 1000, as well as target function information regarding the target function to be subject to user authentication (for example, ``Do you want to pay for 700 yen for a cafe latte?'') may also be output. Therefore, the target user 1000 can intuitively understand for which function the user authentication currently performed in the electronic device 10 is for.

On the other hand, the control unit 130 may perform user authentication based not only on the facial image of the target user 1000 but also on the gesture of the target user 1000.

The control unit 130 may detect not only the facial image of the target user 1000 but also the gesture of the target user 1000 using the camera 11.

Here, the gestures of the target user 1000 may include at least one of the target user's 1000 gestures, actions, and mouth movements.

The control unit 130 may determine whether to complete user authentication for the target user 1000 based on the gesture of the target user 1000.

At this time, the control unit 130 compares the face image of the subject who made the gesture with the already detected face image of the target user 1000, and determines whether the gesture is that of the target user 1000. It's okay. Further, the control unit 130 may complete the user authentication only when the subject who made the gesture is the target user 1000.

Here, "completion of user authentication" refers to the final state of user authentication, and the final state of user authentication includes at least one of i) successful user authentication and ii) failed user authentication. You can do it like this.

In the present invention, the final state of user authentication means searching for a person corresponding to the target user 1000, and then approving, rejecting, executing, or not executing the target function (or a specific function) based on the gesture of the target user 1000. It means a state in which activation or deactivation has been determined.

That is, even when the control unit 130 searches for a person corresponding to the target user 1000 and extracts user information regarding the person, the control unit 130 performs a specific function based on the gesture of the target user 1000. (e.g., payment functions).

Here, as shown in FIG. 3A (c), the control unit 130 stores at least the user information (for example, name information) regarding the target user 1000 and the information (for example, payment function information) regarding the target function that requires user authentication. One may be output to the display unit 12, and then the gesture of the target user 1000 may be detected. This is to detect a gesture that the target user 1000 performs after confirming the current progress status of user authentication from the display unit 12. That is, the target user 1000 can grasp the status of which function the user authentication currently being performed is based on the information output to the display unit 12, and after confirming the status, , users can indicate their intention to authenticate with gestures.

The control unit 130 receives the image from the camera 11 while at least one of user information (for example, name information) and information regarding a target function that requires user authentication (for example, payment function information) is output to the display unit 12. You may also do so. Further, the control unit 130 may detect a gesture of the target user 1000 from the received image.

Note that, in the present invention, in order to detect the gesture of the target user 1000, it goes without saying that the time point at which the image is received from the camera 11 and the time point at which the gesture is detected from the received image may be changed in various ways.

Meanwhile, the storage unit 120 may store information regarding gestures corresponding to different control commands. Therefore, the control unit 130 may detect the gesture of the target user 1000 from the received image and extract from the storage unit 120 a control command that matches the detected gesture. Furthermore, the control unit 130 may control a target function that requires user authentication in accordance with the extracted control command.

As an example, the control unit 130 may process the user authentication related to the target function as successful if the gesture of the target user 1000 corresponds to a predetermined first gesture that matches a control command agreeing to user authentication. good. For example, as shown in (a) of FIG. 3b, when a first gesture (for example, an action of shaking the head vertically) by the target user 1000 is detected from the image received from the camera 11, the control unit 130 specifies the The user authentication related to the function (or the target function) may be treated as successful. For example, the particular function may be a payment function, as shown. As shown in the figure, the control unit 130 may provide the display unit 12 with information 340 regarding the user authentication processing result (for example, “Payment has been completed”). Therefore, information 340 regarding the user authentication processing result is output to the display unit 12.

As another example, if the gesture of the target user 1000 corresponds to a predetermined second gesture that matches a control command to deny user authentication (disagree with user authentication), the control unit 130 performs user authentication related to the target function. may be treated as a failure. For example, as shown in FIG. 3B (b), when a second gesture (for example, shaking the head) by the target user 1000 is detected from the image received from the camera 11, the control unit 130 specifies the The user authentication related to the function (or target function) may be treated as a failure. For example, the particular function may be a payment function, as shown. As shown in the figure, the control unit 130 may provide the display unit 12 with information 350 regarding the user authentication processing result (for example, “payment has been cancelled”). Therefore, information 350 regarding the user authentication processing result is output to the display unit 12.

Therefore, in the user authentication method and system according to the present invention, it is possible to understand the user's intention regarding user authentication based not only on the detected face image of the user but also on the user's gesture. Therefore, in the user authentication method and system according to the present invention, the authenticity of the user who is attempting to perform user authentication can be ascertained based on the user's gesture, and the user authentication can be performed based on the user's intention.

Meanwhile, the control unit 130 may provide the display unit 12 with guide information 320 regarding a control command that matches the gesture, as shown in FIG. 3A (c). Therefore, the target user 1000 can intuitively understand which gesture to perform in order to obtain a desired result in a specific function.

On the other hand, the control unit 130 may cause the face image of the target user 1000 acquired by the camera 11 to be included in the user database when user authentication is completed. The control unit 130 may update the user database so that the user information of the person corresponding to the target user 1000 includes a facial image.

Hereinafter, a case will be described in which another user is recognized in the process of user authentication for a target user recognized by a camera.

In the present invention, as shown in FIGS. 5(a) and 5(b), while user authentication is being performed for the target user 1000a, as shown in FIG. 5(c), the target user is A face image of another user 1000b may be detected together with the face image of user 1000a. Here, the other user 1000b is a user who attempts user authentication or a user who hides the target user 1000a.

In order to complete user authentication for the target user 1000a, the control unit 130 may continue to perform user authentication for the target user 1000a even if another user 1000b is recognized. For example, even if the target user 1000a is recognized by the camera 11 while being hidden behind another user 1000b, the control unit 130 can perform user authentication for the target user 1000a as long as at least a part of the target user 1000a is recognized. You may continue to do so. That is, in the present invention, the control unit 130 may grasp the situation in which the target user 1000a continues to be around the camera 11 as indicating that the target user 1000a intends to continue user authentication.

On the other hand, the control unit 130 may terminate the user authentication for the target user 1000a when another user 1000b is detected by the camera 11 for a predetermined period of time or longer, as shown in FIG. 5(c). Further, the control unit 130 may detect the face image of the other user 1000b from the image received from the camera 11, as shown in FIG. 5(d). Further, the control unit 130 may compare the detected face image of the other user 1000b with a reference face image included in the user database to identify the person corresponding to the other user 1000b. Further, the control unit 130 extracts user information that matches the identified person from the user database, and uses the extracted user information to perform the above-described series of user authentication processes for the other user 1000b. It's okay. In this case, the control unit 130 may end the user authentication for the target user 1000a even if the user authentication for the target user 1000a is not completed.

In this way, the control unit 130 determines the intention of the target user 1000a and other users 1000b to perform user authentication based on the information regarding the target user 1000a and other users 1000b detected by the camera 11. It's okay. Further, the control unit 130 may perform user authentication for either the target user 1000a or the other user 1000b based on the determination result.

The control unit 130 may perform user authentication for either the target user 1000a or the other user 1000b based on predetermined detection conditions. Here, the predetermined detection condition is determined based on at least one of the length of time, posture, and occupied area during which the face images of the target user 1000a and the other users 1000b were detected from the images received from the camera 11. You can do it like this.

For example, the predetermined detection condition is i) whether or not the face image of another user 1000b has been detected by the camera 11 for a predetermined time or longer while user authentication is being performed for the target user 1000a; and ii) if the other user iii) whether the target user 1000a is not looking at the camera 11; iv) the face image of the target user 1000a occupies the image received from the camera 11; At least one sub-condition, such as whether the ratio is smaller than the ratio occupied by the face image of another user 1000b, may be set in combination.

Therefore, the control unit 130 counts the detection time or calculates the occupied area of the face image of at least one of the target user 1000a and the other user 1000b in the image received from the camera 11, and calculates the area occupied by the target user 1000a and the other user 1000b. User authentication may be performed for any user of 1000b.

In this way, the control unit 130 determines the intention of the target user 1000a and other users 1000b to perform user authentication based on the information regarding the target user 1000a and other users 1000b detected by the camera 11. , can provide appropriate user authentication for the user.

The following describes a processing method when the person corresponding to the target user 1000 in the user database is incorrectly specified or a plurality of persons are specified.

As described above, the control unit 130 compares the target user's face image acquired by the camera 11 with the reference face image stored in the user database, and determines whether the target user's face image matches the reference face image corresponding to the target user's face image. You may also search for a person who does. Further, the control unit 130 may specify the searched person as a target user, and extract user information matching the searched person from the user database.

At this time, the degree of similarity between the face image of the target user 1000 acquired by the camera 11 and the reference face image stored in the user database is low, or the current appearance of the target user 1000 is different from the appearance of the reference face image. In this case, the control unit 130 cannot accurately search the user database for a person corresponding to the target user 1000.

Therefore, in the present invention, guide information may be provided so that the target user 1000 can input feedback thereto into the electronic device 10 or the control unit 130 at any time.

For example, as shown in (a) and (b) of FIG. 6a, the control unit 130 controls the control unit 130 when user information regarding a person corresponding to the target user (for example, Hisashi Suzuki**, Osamu Takahashi*) is extracted. However, the display unit 12 may be provided with a GUI 610, 620 or a graphic object for receiving an input as to whether the extracted user information is incorrect.

When the GUI 610 or 620 is selected, the control unit 130 may photograph the target user again using the camera 11 and detect the face image of the target user from the photographed image again. Further, the control unit 130 may search the user database again for the person corresponding to the detected face image, and output user information regarding the person searched again to the display unit 12.

Here, if the person searched again and the person searched before the search again are the same person, the control unit 130 executes the user authentication process using another method instead of the face recognition method. Good too. The user information regarding the person searched again may not be output to the display unit 12.

On the other hand, if the person searched again and the person searched before the search again are the same person, the control unit 130 transfers the target user's identification information (for example, phone number, employee number, account information, The display unit 12 may be provided with a GUI for receiving input of address, name, etc.). Further, the control unit 130 uses the identification information of the target user input through the GUI to search the user database for a person corresponding to the target user, and uses the user information corresponding to the searched person to search for a person corresponding to the target user. User authentication may also be performed.

The control unit 130 may extract from the user database a specific reference face image having a degree of similarity that satisfies a predetermined condition with respect to the detected face image of the target user 1000. Further, the control unit 130 may specify a person who matches the extracted specific reference face image as the target user.

As described above, the control unit 130 compares the target user's face image acquired by the camera 11 with the reference face image stored in the user database, and determines whether the target user's face image matches the reference face image corresponding to the target user's face image. You may also search for a person who does. Further, the control unit 130 may specify the searched person as a target user, and extract user information matching the searched person from the user database.

On the other hand, the control unit 130 searches the user database for a reference face image having a degree of similarity that satisfies a predetermined condition (or a predetermined similarity condition) with respect to the target user's face image, and controls The person who does this may be identified as the person who corresponds to the target user. At this time, a plurality of reference face images may be searched from the user database. Here, the plurality of reference face images may be images corresponding to different people.

In this case, the control unit 130 extracts user information of a plurality of people corresponding to the plurality of searched reference face images, and uses the extracted user information to, as shown in (a) of FIG. The display unit 12 may be provided with a GUI for receiving selection of user information corresponding to the user from the target user. As shown in the figure, graphic objects 630 and 640 may be output to the display unit 12 for receiving the selection of one of a plurality of people corresponding to a plurality of reference face images, respectively.

The output graphic objects 630, 640 may correspond to different people, and each may include user information (or identification information, such as account information, email address, name, etc.) of the corresponding person.

Further, as shown in (a) of FIG. 6b, the control unit 130 identifies a specific person who matches the graphic object 630 selected by the target user among the graphic objects 630 and 640 as a person corresponding to the target user. You may also do so. Further, as shown in FIG. 6B (b), the control unit 130 may perform user authentication for the target user using user information that matches the identified person.

On the other hand, the control unit 130 may cause the target user's face image acquired by the camera 11 to be included in the user database when user authentication is completed. The control unit 130 may update the user database so that the user information corresponding to the identified person includes a facial image.

Further, the control unit 130 may set the target user's face image as a reference face image of a person corresponding to the target user. Furthermore, when the next user authentication for the target user is performed, the control unit 130 may specify the target user using the newly set reference face image.

As another example, the control unit 130 searches the user database for a reference face image that has a degree of similarity that satisfies a predetermined condition (or a predetermined similarity condition) with respect to the target user's face image, and The person corresponding to the image may be identified as the person corresponding to the target user. At this time, a plurality of reference face images may be searched from the user database. Here, the plurality of reference face images may be images corresponding to different people.

In this case, the control unit 130 may output a plurality of reference face images 671, 672, and 673 corresponding to different people to the display unit, as shown in FIG. 6c.

Further, when one of the plurality of reference face images 671, 672, and 673 is selected by the target user, the control unit 130 specifies a person matching the selected reference face image as a person corresponding to the target user. You may also do so.

Furthermore, when there are a plurality of searched reference face images, the control unit 130 controls the control unit 130 to control the control unit 130 to control the control unit 130 to determine whether the searched reference face images have a higher degree of similarity (or the highest degree of similarity) with the target user's face image photographed by the camera 11 among the searched reference face images. The user information of a specific person (or a first person) matching a specific reference face image (or a first reference face image) may be extracted from the user database.

Furthermore, the control unit 130 includes user information 650 about the specific person (or the first person), and also controls the user information 650 about the specific person (or the first person), as well as other information about the specific person (or the first reference facial image) that is different from the specific reference facial image (or the first reference facial image) among the plurality of searched reference facial images. Reference face images (or second reference face images) 671, 672, and 673 may be provided to the display unit.

That is, the control unit 130 primarily identifies the person corresponding to the reference face image with the highest degree of similarity to the target user's face image as the person corresponding to the target user, but other people may correspond to the target user. Gender is not excluded either.

On the other hand, as shown in FIG. 6c, the control unit 130 displays other reference face images (or second reference face images) 671, 672, and 673 on the display unit along with the user information 650 of the specific person (or the first person). When the camera 110 detects a gesture of the target user that satisfies a predetermined condition (for example, a gesture related to user authentication) in a state where the target user is output as the first person, the target user may be identified as the first person. Further, the control unit 130 may perform user authentication using the user information 650 of a specific person (or the first person).

On the other hand, unlike this, in a state where other reference face images (or second reference face images) 671, 672, 673 are output to the display unit together with the user information 650 of the specific person (or the first person), Any of the other reference face images (or second reference face images) 671, 672, and 673 may be selected by the target user. In this case, the control unit 130 converts the specific person (or second person) corresponding to the other reference face image (or second reference face image) selected from the user database by the target user to the person corresponding to the target user. It may be specified as . The control unit 130 also extracts user information of a specific person (or second person) corresponding to another reference face image (or second reference face image) selected from the user database by the target user, and User information may be used to authenticate the target user.

In this case as well, the control unit 130 controls the user information corresponding to the person (first person or second person) identified as the target user to include the face image of the target user acquired by the camera 11. , the user database may be updated. Further, the control unit 130 may set the target user's face image as a reference face image of a person corresponding to the target user. Furthermore, when the next user authentication for the target user is performed, the control unit 130 may specify the target user using the newly set reference face image.

Hereinafter, a method for performing user authentication when the user database does not contain user information of a person corresponding to the target user 1000 will be described.

As described above, the control unit 130 compares the target user's face image acquired by the camera 11 with the reference face image stored in the user database, and determines whether the target user's face image matches the reference face image corresponding to the target user's face image. You may also search for a person who does. Further, the control unit 130 may specify the searched person as a target user, and extract user information matching the searched person from the user database.

Here, the reference face image corresponding to the face image of the target user 1000 acquired by the camera 11 may not exist in the user database.

In this case, the control unit 130 may execute the user authentication process using another method instead of the face recognition method, as shown in FIGS. 7A and 7B. As shown in FIG. 7A, the control unit 130 may output guide information 710 to the display unit 12 to notify that the reference face image does not exist in the user database.

In addition, as shown in FIG. 7A (b), the control unit 130 uses a GUI 720 for receiving the target user's identification information (for example, telephone number, employee number, account information, address, name, etc.) from the target user. may be provided to the display unit 12.

In this way, when there is no user information corresponding to the target user's face image in the user database, the GUI 720 for receiving the input of the target user's identification information is displayed on the display unit 12, as shown in FIG. It may be displayed. The control unit 130 searches the user database for a person corresponding to the target user using the identification information of the target user inputted through the GUI 720, and uses the user information corresponding to the searched person (for example, Osamu Takahashi*). Alternatively, the target user may be authenticated.

Further, the control unit 130 may perform a process for registering the target user's face image detected by the camera 11 as a reference face image of the searched person. That is, the control unit 130 may include the target user's face image in the user information of the searched person so that the target user can perform user authentication using a face recognition method next time.

To this end, the control unit 130 causes the display unit 12 to output a GUI 740 for receiving a selection from the target user to register the target user's face image, as shown in (a) and (b) of FIG. 7b. It's okay. When the GUI 740 is selected by the target user, the control unit 130 updates the user database so that the target user's face image is registered as the reference face image of the person corresponding to the target user's user information. good.

On the other hand, if the target user performs user authentication a predetermined number of times using another user authentication method instead of the face recognition method in a state where the target user's reference face image is not registered in the user database, the control unit 130 controls the target user's A process for registering a face image as a reference face image may be performed.

More specifically, the control unit 130 may analyze the degree of similarity between a plurality of facial images of the target user while user authentication is performed a predetermined number of times. That is, even if the target user is authenticated using another user authentication method, the control unit 130 acquires the facial image of the target user using the camera 11 and stores the acquired facial image in the storage unit 120. You may also save it in

Further, the control unit 130 may analyze the degree of similarity between a plurality of facial images of the target user acquired in a situation where user authentication has been performed a predetermined number of times using another user authentication method. Furthermore, if the degree of similarity between the plurality of facial images of the target user satisfies a predetermined similarity criterion, a process for registering the facial image of the target user may be performed.

That is, the control unit 130 uses target users acquired at different times to confirm whether or not a target user who has performed user authentication using another user authentication method is a true user who is eligible for user authentication. A comparison may be made between the user's face images. Therefore, the control unit 130 performs a process for registering a face image only when the same target user performs user authentication for the same person multiple times (or a predetermined number of times) as a result of comparing the face images. You can do it like this.

As described above, the user authentication method and system according to the present invention are intended to provide a user authentication method and system that can grasp the user's intention and perform user authentication more intuitively.

Further, the present invention provides a user authentication method and system that allows user authentication to be performed without the user directly inputting information.

Furthermore, the present invention provides a user authentication method and system that can complete user authentication while grasping the user's true intention as to whether or not to perform user authentication.

On the other hand, the computer-readable medium includes any type of recording device on which data that can be read by a computer system is recorded. Examples of computer-readable media include HDD (Hard Disk Drive), SSD (Solid State Disk), SDD (Silicon Disk Drive), ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical data storage device, etc. It will be done.

The computer-readable medium also includes storage and can be a server or cloud storage that the electronic device can access via communication. In this case, the computer can download the program according to the invention from the server or cloud storage by wired or wireless communication.

Furthermore, in the present invention, the above-mentioned computer has a processor, that is, a central processing unit (Central Processing Unit).
It is an electronic device equipped with a CPU (Unit, CPU), and its type is not particularly limited.

On the other hand, the detailed description of the present invention is illustrative and should not be construed as restrictive in any respect. The scope of the present invention should be determined by reasonable interpretation of the appended claims, and all modifications within the range of equivalency of the present invention are included within the scope of the present invention.

<<< Additional notes >>>
(Additional note 1)
detecting a facial image of the target user using a camera;
extracting user information corresponding to the facial image from a user database;
performing user authentication for the target user using the user information,
In the step of performing user authentication,
A user authentication method, comprising determining whether to complete the user authentication based on a gesture of the target user detected by the camera.
(Additional note 2)
further comprising the step of providing at least a portion of the user information to a display unit of the electronic device that performs the user authentication;
In the step of performing user authentication,
The user authentication method according to appendix 1, wherein the user authentication is performed based on a gesture of the target user detected after at least a part of the user information is provided to the display unit.
(Additional note 3)
The display section includes:
In addition to at least a part of the user information, target function information regarding a target function to be subject to the user authentication is further provided;
In the step of performing user authentication,
if the target user's gesture corresponds to a predetermined first gesture that matches a control instruction agreeing to the user authentication;
The user authentication method according to appendix 2, characterized in that user authentication in the target function is treated as successful.
(Additional note 4)
In the step of performing user authentication,
when the target user's gesture corresponds to a predetermined second gesture that matches the control instruction to deny the user authentication;
The user authentication method according to appendix 3, characterized in that user authentication in the target function is treated as a failure.
(Appendix 5)
In the step of performing user authentication,
Supplementary note 1, characterized in that even if the camera detects a face image of another user different from the target user while the user authentication for the target user is being performed, the user authentication for the target user is performed. The user authentication method described in any one of ~4.
(Appendix 6)
In the step of performing user authentication,
The user authentication method according to appendix 5, characterized in that when the face image of the other user is detected for a predetermined period of time or more, user authentication for the target user is terminated.
(Appendix 7)
When user authentication for the target user is completed,
In the step of performing user authentication,
Supplementary Note 6, characterized in that user information corresponding to the face image of the other user is extracted from the user database, and user authentication for the other user is performed using the user information of the other user. User authentication method.
(Appendix 8)
In the step of extracting the user information,
Comparing the target user's facial image and a reference facial image stored in the user database, searching for a reference facial image that satisfies a predetermined similarity criterion with respect to the target user's facial image;
The user information is
8. The user authentication method according to any one of appendices 1 to 7, wherein the user information is user information of a person corresponding to the searched reference face image.
(Appendix 9)
If the searched reference face images are multiple,
The display section of the electronic device that performs the user authentication includes:
outputting a graphic object for receiving selection of one of the plurality of people corresponding to the plurality of searched reference face images;
In the step of performing user authentication,
The user authentication method according to appendix 8, characterized in that user authentication for the target user is performed using user information that matches a specific person selected by the graphic object among the plurality of people.
(Appendix 10)
The user authentication method according to appendix 9, further comprising the step of updating the user database so that the user information matching the specific person includes a face image of the target user.
(Appendix 11)
If the searched reference face images are multiple,
In the step of extracting the user information,
extracting from the user database user information of a first person that matches a first reference face image with a higher degree of similarity to the target user's face image from among the plurality of searched reference face images;
A second reference face image different from the first reference face image among the plurality of searched reference face images is provided together with the user information of the first person to a display unit of the electronic device that performs the user authentication. The user authentication method according to appendix 8, characterized in that:
(Appendix 12)
In the step of performing user authentication,
When a gesture of the target user is detected while the user information of the first person and the second reference face image are provided on the display unit,
identifying the target user as the first person;
The user authentication method according to appendix 11, characterized in that the user authentication is performed using user information of the first person.
(Appendix 13)
When the second reference face image is selected by the target user while the user information of the first person and the second reference face image are provided on the display unit,
extracting user information of a second person corresponding to the second reference face image from the user database;
The user authentication method according to appendix 12, characterized in that the user authentication is performed using user information of the second person.
(Appendix 14)
If user information corresponding to the target user's face image does not exist in the user database,
The display section of the electronic device that performs the user authentication includes:
A GUI is provided for receiving input of identification information of the target user,
In the step of extracting the user information,
14. The user authentication method according to any one of appendices 1 to 13, characterized in that the user information is extracted from the user database using identification information of the target user input through the GUI.
(Appendix 15)
15. The user authentication method according to appendix 14, further comprising the step of updating the user database so that the target user's face image is registered as a reference face image of the person corresponding to the user information.
(Appendix 16)
In the step of detecting the face image,
determining whether the target user's face image detected by the camera satisfies predetermined authentication conditions;
Supplementary notes 1 to 15, characterized in that if the predetermined authentication condition is not satisfied as a result of the determination, guide information for satisfying the predetermined authentication condition is provided to a display unit of the electronic device that performs the user authentication. User authentication method described in any of the above.
(Appendix 17)
The guide information is
The user authentication method according to appendix 16, characterized in that the method relates to at least one of the location and distance between the camera and the target user.
(Appendix 18)
a display section;
A camera that detects the target user's face image,
a control unit that extracts user information corresponding to the face image from a user database,
The control unit includes:
Performing user authentication for the target user using the user information,
A user authentication system, characterized in that it is determined whether or not to complete the user authentication based on a gesture of the target user detected by the camera.
(Appendix 19)
A program that operates a computer in an electronic device,
The program is
A command for executing the steps of detecting a facial image of a target user using a camera, extracting user information corresponding to the facial image from a user database, and performing user authentication for the target user using the user information. including;
In the step of performing user authentication,
A program for determining whether to complete the user authentication based on a gesture of the target user detected by the camera.

Japanese Patent Application Publication No. 2018-173891 Japanese Patent Application Publication No. 2014-048937 Japanese Patent Application Publication No. 2020-115361 International Publication No. 2020/136795 International Publication No. 2018/235191 Japanese Patent Application Publication No. 2009-093399

Claims (9)

A user authentication method executed by an electronic device including a camera, a display unit, and a control unit,
detecting a facial image of a target user using the camera;
the control unit extracting user information corresponding to the face image from a user database;
the control unit providing the display unit with at least part of the user information and target function information regarding a target function for which user authentication is required;
detecting gestures of the target user using the camera;
A user authentication method comprising: the control unit performing user authentication for approval or rejection of the target function using gestures of the target user detected by the camera. In the step of performing user authentication,
The user authentication according to claim 1, wherein the user authentication is performed based on a gesture of the target user detected after at least a part of the user information and the target function information are provided to the display unit. User authentication method. In the step of performing user authentication,
The result of the user authentication for approval of the target function differs depending on whether the gesture of the target user detected by the camera is a gesture that has been matched in advance for approval of the target function. The user authentication method according to claim 1. In the step of performing user authentication,
Claim characterized in that even if the camera detects a face image of another user different from the target user while the user authentication for the target user is being performed, the user authentication for the target user is performed. 1. The user authentication method described in 1. In the step of performing user authentication,
5. The user authentication method according to claim 4, wherein when the face image of the other user is detected for a predetermined period of time or more, user authentication for the target user is terminated. When user authentication for the target user is completed,
In the step of performing user authentication,
6. The user information corresponding to the other user's face image is extracted from the user database, and the user information of the other user is used to perform user authentication for the other user. user authentication method. If user information corresponding to the face image of the target user does not exist in the user database, a GUI for receiving the input of identification information of the target user is displayed on a display unit of the electronic device that performs the user authentication;
In the step of extracting the user information,
2. The user authentication method according to claim 1, wherein the control unit extracts the user information from the user database using identification information of the target user input through the GUI. a display section;
A camera that detects the target user's face image,
a control unit that extracts user information corresponding to the face image from a user database,
The control unit includes:
providing at least part of the user information and target function information regarding a target function for which user authentication is required to the display unit;
A user authentication system characterized by performing user authentication for approving or rejecting the target function using gestures of the target user detected by the camera. A program executed by one or more processes on an electronic device including a camera, a display unit, and a control unit,
The program is
detecting a facial image of a target user using the camera;
the control unit extracting user information corresponding to the face image from a user database;
the control unit providing the display unit with at least part of the user information and target function information regarding a target function for which user authentication is required;
detecting gestures of the target user using the camera;
A program characterized in that the control unit performs user authentication for approving or rejecting the target function using a gesture of the target user detected by the camera.

Images