TWI578183B - Identity verification method, apparatus and system and computer program product - Google Patents

Description

Identity verification method, device, system and computer program product

The present invention relates to an identity verification mechanism, and more particularly to an identity verification method, apparatus, system, and computer program product using eye movement input.

At present, eye tracking technology can be mainly divided into two types: invasive and non-invasive. Invasive eye tracking technology is mainly to set up a search Coil in the eye or use an electrooculogram. Non-invasive eye tracking technology can be distinguished as a free-head or head-mount eye tracking technology. With the development of technology, eye tracking technology has been widely used in various fields, such as neuroscience, psychology, industrial engineering, human factors engineering, marketing advertising, computer science and so on.

Eye tracking technology is used to help communicate and assist people with physical difficulties, which brings more convenience to those with reduced mobility. For example, with the aid of eye-moving electronic products, it is possible to replace communication with the mouse to complete communication, Internet access, and audio-visual entertainment activities.

The invention provides an identity verification method, device, system and computer program product, which uses an eye movement state to input a password, so that the password input is more diversified.

The method for verifying the identity of the present invention comprises: analyzing a facial image sequence of the user to determine whether the biometric feature conforms to the preset feature; displaying the input interface after the biometric feature conforms to the preset feature; performing eye tracking on the facial image sequence The algorithm is used to detect the blinking action of the user; the blinking action is used to perform the mental power confirmation; when the blinking action conforms to the preset action, the determination is confirmed by the mental force.

In an embodiment of the present invention, after determining the confirmation by the mental stimuli, the method further includes: performing an eye tracking algorithm on the facial image sequence to detect the eye movement state of the user; and performing the voluntary will based on the eye movement state The confirmation program is used to determine whether the user is in an autonomous will state or an involuntary will state; when the user is determined to be in autonomous will state, the operating procedure is initiated; and when the user is determined to be in a non-autonomous will state, a warning signal is generated and an operation is initiated. program.

In an embodiment of the present invention, in the step of performing an autonomous will confirmation process based on an eye movement state, determining whether an alert option in the input interface is triggered based on an eye movement state; and determining a user when the alert option is triggered In a state of non-autonomous will.

In an embodiment of the present invention, in the step of performing the autonomous will confirmation process based on the eye movement state, when the password string conforms to the first string, it is determined that the user is in an autonomous will state, and all use of the open operation program is performed. Permission; and when the password string conforms to the second string, it is determined that the user is in a non-autonomous will state, and part of the usage rights of the operating program are opened.

In an embodiment of the invention, the input interface includes a password input block and a non-password input block. The password input block is used to provide input of a password string. In the step of performing the autonomous will confirmation process based on the eye movement state, determining whether the non-password point selection block is selected based on the eye movement state; and determining the user when determining that the number of times the non-selection block is selected corresponds to the preset number of times In a state of non-autonomous will.

In an embodiment of the invention, when the password string is input through the input interface, the eye movement state of the user's eyes when looking at one of the plurality of input units in the input interface is determined, wherein the eye movement state includes At least one of a gaze time, a pupil displacement amount, and a pupil movement direction; and an input unit for determining the desired output based on the blink action.

In an embodiment of the invention, the biometric feature is a facial feature, an iris feature, or an ocular white vein feature.

The identity verification device of the present invention comprises: a display unit, an image capture unit, and a control unit. The control unit is coupled to the display unit and the image capturing unit. The display unit displays the input interface. The image capturing unit captures a sequence of facial images of the user. The control unit analyzes the user's facial image sequence to determine whether the biometric features conform to the preset features, and displays the input interface after the biometric features conform to the preset features. The control unit performs an eye tracking algorithm on the facial image sequence to detect the blinking motion of the user, and performs the mental intelligence confirmation through the blinking motion, and when the blinking motion conforms to the preset motion, the control unit determines to confirm by the mental power .

The computer program product of the present invention comprises a computer readable recording medium, and the computer readable recording medium records at least one program instruction, and the program instructions are used to execute the steps of the identity verification method after loading the electronic device.

The identity verification system of the present invention includes: a local end device and a server device. The local device includes: a display unit that displays an input interface; and an image capturing unit that captures a sequence of facial images. The server device is connected to the local device through the Internet to receive a facial image sequence from the local device. The server device includes: a control unit. The control unit analyzes the user's facial image sequence to determine whether the biometric feature conforms to the preset feature, and after determining that the biometric feature conforms to the preset feature, causes the user device to display the input interface to the display unit, and the control unit faces the face The image sequence performs an eye tracking algorithm to detect a blinking motion of the user, and performs a mental power confirmation through the blinking motion, and when the blinking motion conforms to the preset motion, the control unit determines to confirm by the mental power.

Based on the above, combined with biometrics and eye tracking algorithms for identity verification, not only improves security, but also increases the variety of operations.

The above described features and advantages of the invention will be apparent from the following description.

1 is a block diagram of an identity verification device in accordance with an embodiment of the present invention. Referring to FIG. 1 , the identity verification apparatus 100 includes a display unit 110 , an image capturing unit 120 , a storage unit 130 , and a control unit 140 . The control unit 140 is coupled to the display unit 110, the image capturing unit 120, and the storage unit 130.

The display unit 110 is configured to display a graphical interface such as an input interface to facilitate the user to view the screen of the display unit 110 for operation. The display unit 110 can be any type of display, such as a liquid crystal display (LCD), a light emitting diode (LED) display, or a flexible display.

The image capturing unit 120 is configured to capture a facial image sequence of the user. The image capturing unit 120 is, for example, a camera, a camera, or the like using a charge coupled device (CCD) lens or a complementary metal oxide semiconductor transistor (CMOS) lens.

The storage unit 130 is, for example, any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory (Flash memory), hard Disc or other similar device or a combination of these devices.

The control unit 140 is, for example, a central processing unit (CPU), an image processing unit (GPU), a physical processing unit (PPU), a programmable microprocessor (Microprocessor), Embedded control chip, Digital Signal Processor (DSP), Application Specific Integrated Circuits (ASIC) or other similar devices.

The storage unit 130 includes an image recognition module 131, a password verification module 132, and an eyeball tracking module 133, which are respectively composed of one or more code segments, and the control unit 140 executes the modules to perform multiple functions. The storage unit 130 may further include a database 134 for storing account data of the qualified user. In other embodiments, the image recognition module 131, the password verification module 132, and the eyeball tracking module 133 may each be a chip group composed of a plurality of digital logic gates. The control unit 140 drives and executes the image recognition module 131, the password verification module 132, and the eyeball tracking module 133 to implement related operational procedures.

The eye tracking module 133 can detect the blinking motion or the eye displacement of the user by performing an eye tracking algorithm on the facial image sequence. When the blinking motion conforms to the preset motion, the eyeball tracking module 133 determines to confirm by the mental power.

In addition, the eyeball tracking module 133 detects a plurality of gaze locations on the input interface of the user and detects an eye movement state, thereby inputting a password string through the input interface. The eye movement state includes at least one of a gaze time, a pupil displacement amount, and a pupil movement direction. The eye tracking module 133 determines an eye movement state when the user's eyes are looking at one of a plurality of input units (for example, numbers or symbols) included in the input interface, and determines one of the desired outputs based on the blinking motion. Number or symbol. For example, the pupil displacement and the pupil movement direction and the gaze position are used to judge the selected candidate character, and the preset blink action is used as the determination input command. Blinking actions include the number of blinks, the frequency of blinking, and the time of blinking.

The password verification module 132 performs a password verification procedure. The password verification module 132 verifies whether the entered password string matches the qualified password of the qualified user. The image recognition module 131 is configured to analyze the facial image sequence of the user to determine whether the user is a qualified user allowed by the identity verification device 100. For example, it is determined whether the biometric of the user conforms to the preset feature. The above biometric features are, for example, facial features, iris features or ocular white vein features.

The steps of the identity verification method are described below with the above-described identity verification device 100. 2 is a flow chart of an identity verification method in accordance with an embodiment of the present invention. Referring to FIG. 1 and FIG. 2, in step S205, the image recognition module 131 analyzes the facial image sequence to determine whether the biometric feature of the user conforms to the preset feature. That is, the image recognition module 131 first determines whether the user is a qualified user. If it is determined that the user is a qualified user, the input interface is displayed as shown in step S215. If it is determined that the user is a non-qualified user, as shown in step S230, the operation flow is ended.

After determining that the biometrics meet the preset features, the control unit 140 displays the input interface in the display unit 110 in step S215.

Here, the above biometric features are one or a combination of facial features, iris features, and ocular white vein features. For example, the image recognition module 131 determines whether the user is a qualified user based on the preset features in the database 134, that is, determines biometric characteristics such as one of the facial features, the iris features, and the white blood vessel lines, or a combination thereof. Whether it matches the preset feature. If it is determined that the user is not a qualified user, the image recognition module 131 ends the flow of the identity verification (step S230). If the biometric is consistent with the preset feature, the user is initially determined to be a qualified user, and the input interface is displayed, and the eye tracking module 133 is driven. The image recognition module 131 can also selectively read the preset password corresponding to the qualified user from the database 134, and transmit the corresponding preset password to the password verification module 132 for subsequent comparison.

Next, in step S220, an eye tracking algorithm is performed on the user's facial image sequence to detect the user's blinking motion. That is, the control unit 140 drives the eyeball tracking module 133 to perform an eye tracking algorithm. Thereafter, in step S225, the eyeball tracking module 133 performs cardiac smart force confirmation through the blinking motion. When the blinking action conforms to the preset action, the determination is confirmed by the mental power. The blinking actions described above include the number of blinks, the frequency of blinks, and the blink time.

For example, in the case where the blinking operation is the number of blinks, when the eye tracking module 133 detects that the number of blinks is three, it is determined that the eye intelligence is confirmed. The preset number of blinks (3 times) is merely an example enumerated for convenience of explanation, and the present invention is not limited thereto. Furthermore, the default number of blinks can be N times, and the value of N is a positive integer that can be selected by the designer.

For example, in the case of the blinking motion as the blink frequency, it is assumed that the preset blink frequency range is set to 1.9 to 2.1 seconds/blink times. When the blink frequency detected by the eye tracking module 133 is within the preset blink frequency range, it is determined to be confirmed by the mental power. The preset blink frequency range (1.9 to 2.1 seconds/blink times) is merely an example enumerated for convenience of explanation, but the present invention is not limited thereto. Furthermore, the upper/lower limit of the preset preset blink frequency range can be selected by the designer and can also be a specific frequency (for example, 2 seconds/blink times).

For example, when the blinking action is the blink length, when the eye tracking module 133 detects that the continuous blink time is within the preset blink time range, it is determined to be confirmed by the mental power. The preset blink time range is set, for example, to 3 to 5 seconds. That is, when the eye tracking module 133 detects that the user continues to blink for 3 to 5 seconds, it is determined to be confirmed by the mental power. Here, the preset blink time range (3 to 5 seconds) is merely an example for convenience of explanation, but the present invention is not limited thereto. Furthermore, the upper/lower limit of the set blink time range can be selected by the designer, or only the lower limit of the blink time range can be set without setting the upper limit.

The identity verification apparatus 100 may be further configured to request the user to input a password string to perform the identity verification procedure after determining that the mental power is confirmed. For example, enter a password in the input interface through eye movement input. The password verification module 132 determines whether the entered password string is correct. If the entered password string is correct, start the operating program. Here, the operation program is, for example, an unlocking program, a financial transaction program, or the like. If the entered password string is incorrect, the current operation flow is ended. However, the above-mentioned identity verification procedure by the password string is optional, and is not necessarily limited.

Further, in order to prevent the user from performing the identity verification program under coercion, the identity verification device 100 also performs an autonomous will confirmation program to judge whether the user is in an autonomous will state or a non-autonomous will state. For example, the control unit 140 determines whether the eye movement state meets the preset condition, thereby determining whether the user is in an autonomous will state or in a non-autonomous will state. An embodiment will be described below.

3 is a flow chart of an identity verification method in accordance with another embodiment of the present invention. In the present embodiment, the same steps as those in FIG. 2 are given the same reference numerals, and the description thereof will be omitted.

After step S225, in step S305, the control unit 140 performs an autonomous will confirmation process based on the eye movement state, thereby determining whether the user is in an autonomous will state or a non-autonomous will state. For example, the control unit 140 determines whether the eye movement state meets the preset condition, thereby determining whether the user is in an autonomous will state or in a non-autonomous will state.

Thereafter, in step S310, the eyeball tracking module 133 determines whether or not it is confirmed by the mental power. If it is not confirmed by the mental power, as shown in step S230, the current operation flow is ended. If it is confirmed by the mental power, in step S315, it is judged whether or not it is in the state of autonomy.

When it is determined that the user is in the state of autonomy, the control unit 140 starts the operation procedure as shown in step S320. In step S320, the control unit 140 does not generate an alert signal. When it is determined that the user is in the non-autonomous will state, as shown in step S325, the control unit 140 generates an alert signal and starts the operating procedure.

The following is an example to illustrate how to perform an autonomous will confirmation procedure based on the eye movement state.

An alert option is set in the input interface, and when the user has a non-autonomous operation such as being threatened, the eye movement input can be used to trigger the alert option. The control unit 140 determines whether the alert option in the input interface is triggered based on the eye movement state. When the alert option is triggered, the control unit 140 determines that the user is currently in a non-autonomous will state. In turn, the identity verification device 100 generates and transmits an alert signal to a security device. Moreover, in the state of determining that the user is in a non-autonomous will state, if confirmed by the mental stimuli, the control unit 140 will still initiate a subsequent operational procedure.

In addition, two sets of preset passwords may be preset, one set (first string) is used for the user to operate with autonomy, and the other group (second string) is used for threatening Use when operating with autonomous will. Moreover, the two sets of preset passwords have different usage rights respectively. Specifically, the user inputs a password string by using the eye movement input for the password verification module 132 to execute the password verification program. When the password string conforms to the first string, the control unit 140 determines that the user is in an autonomous will state and opens all the usage rights of the operating program. When the password string conforms to the second string, it indicates that the user is currently in a state of non-autonomous consciousness to operate, and therefore, the control unit 140 determines that the user is in a non-autonomous will state, and opens a partial use authority of the operating program.

Moreover, it can be further set as follows. When the user selects the non-password input block by using the eye movement input, and the number of clicks exceeds the preset number of times, the user is determined to be in a non-autonomous will state. Specifically, the input interface includes a password input block and a non-password input block, and the password input block is used to provide an input of the password string. The control unit 140 determines whether the non-password point selection block is selected based on the eye movement state, and determines that the user is in the non-autonomous will state when it is determined that the number of times the non-selection block is clicked corresponds to the preset number of times.

In addition, it is also possible to set the autonomous will confirmation process using the blinking action. For example, when the number of blinks is detected to be 5 or more times (the number of blinks may be determined by the case), it is determined that the user is in a state of involuntary will. For another example, when it is detected that the blink frequency is within a warning frequency range, it is determined that the user is in a non-autonomous will state. Alternatively, it is determined that the user is in a state of involuntary will when detecting that the user continues to blink for more than a warning length (for example, 6 seconds or longer).

In addition, the identity verification method may also be performed by a remote server device, and the local device is only responsible for capturing the facial image sequence, displaying the input interface and outputting a result. Let me give another example.

4 is a block diagram of an identity verification system in accordance with an embodiment of the present invention. Referring to FIG. 4, the identity verification system 400 includes a server device 410, a local device 420, and a security device 430. The server device 410, the local device 420, and the security device 430 are electronic devices having computing capabilities and communicate via the Internet N. The security device 430 is, for example, a server of a police station or a server of a financial institution.

The server device 410 includes a control unit 411, a communication unit 412, and a storage unit 413. The control unit 411 and the storage unit 413 are similar to the control unit 140 and the storage unit 130, respectively. The storage unit 413 includes an image recognition module 414, a password verification module 415, an eyeball tracking module 416, and a database 417. For related description, reference may be made to the image recognition module 131, the password verification module 132, the eyeball tracking module 133, and the like. Database 134.

The local device 420 includes a control unit 421, a display unit 422, an image capturing unit 423, and a communication unit 424. The communication unit 412 and the communication unit 424 are, for example, wired or wireless network cards.

In this embodiment, the local device 420 is only responsible for acquiring the facial image sequence, and then transmitting the facial image sequence to the server device 410 through the communication unit 424, and the server device 410 performs the identity authentication process.

Specifically, the control unit 421 of the local device 420 first drives the image capturing unit 423 to obtain a facial image sequence of the user, and transmits the facial image sequence to the server device 410. Next, the control unit 411 of the server device 410 analyzes the facial image sequence to determine whether the biometric feature conforms to the preset feature and transmits an analysis result to the local end device 420. If the analysis result indicates that the biometric feature meets the preset feature, the control unit 421 of the local device 420 displays an input interface in the display unit 422. In addition, it may be further configured that if the analysis result indicates that the biometric feature does not conform to the preset feature, the control unit 421 of the local end device 420 will not display the input interface.

After the input interface is displayed, the control unit 411 of the server device 410 performs an eye tracking algorithm on the facial image sequence to detect the blinking action of the user, and performs the mental power confirmation through the blinking action.

The invention further provides a computer program product for use in an identity verification device. The computer program product includes a computer readable recording medium, and the computer can read at least one program instruction of the recording medium. The above program instructions are basically composed of a plurality of code segments (for example, creating an organization chart code segment, signing a form code segment, setting a code segment, and deploying a code segment). After the program command is loaded into an electronic device, the functions of the identity verification method and the identity verification device can be completed.

In summary, the user combines biometrics and eye tracking algorithms in the identity verification process to greatly improve the validity of identity verification. In addition, eye movements are performed using eye movements to make the operation more diverse. Moreover, in the case of threats, the eye movement input can be used to trigger the warning signal, thereby enabling the rescue without being detected by the culprits.

Although the present invention has been disclosed in the above embodiments, it is not intended to limit the present invention, and any one of ordinary skill in the art can make some changes and refinements without departing from the spirit and scope of the present invention. The scope of the invention is defined by the scope of the appended claims.

100‧‧‧identity verification device

110, 322‧‧‧ display unit

120‧‧‧Image capture unit

130, 313‧‧‧ storage unit

140, 411, 421‧‧‧ Control Unit

131, 414‧‧‧ image recognition module

132, 415‧‧‧ password verification module

133, 416‧‧‧ eye tracking module

134, 417‧‧ ‧ database

412, 424‧‧‧ communication unit

430‧‧‧Security equipment

N‧‧‧Internet

S205~S230‧‧‧Identification method

S305~S325‧‧‧Steps of the identity verification method of another embodiment

1 is a block diagram of an identity verification device in accordance with an embodiment of the present invention. 2 is a flow chart of an identity verification method in accordance with an embodiment of the present invention. 3 is a flow chart of an identity verification method in accordance with another embodiment of the present invention. 4 is a block diagram of an identity verification system in accordance with an embodiment of the present invention.

S205~S230‧‧‧Identification method

Claims (16)

An identity verification method includes: analyzing a facial image sequence of a user to determine whether a biometric conforms to a predetermined feature; after the biometric conforms to the preset feature, displaying an input interface; The image sequence performs an eye tracking algorithm to detect a blinking motion of the user; the blinking action is performed to perform a mental power confirmation; when the blinking motion conforms to a preset motion, the determination is confirmed by the mental intelligence; After determining the confirmation by the mental stimuli, detecting the eye movement state of the user, and based on the eye movement state, performing an autonomic confirmation procedure to determine whether the user is in an autonomous will state or a non-autonomous will state. Conduct a verification process. The method for verifying identity according to claim 1, wherein the step of determining whether the user is in the autonomous will state or the non-autonomous will state to perform the identity verification procedure further comprises: inputting a password through the input interface. a string to initiate the identity verification procedure; to initiate an operational procedure when the user is determined to be in the autonomous will state; and to generate an alert signal and initiate the operational procedure when the user is determined to be in the non-autonomous will state. The method for verifying identity according to claim 2, wherein the step of performing the autonomous will confirmation process based on the eye movement state comprises: determining whether a warning option in the input interface is triggered based on the eye movement state; When the alert option is triggered, it is determined that the user is in the non-autonomous will state. The method for verifying identity according to claim 2, wherein the step of performing the autonomous will confirmation process according to the eye movement state comprises: when the password string conforms to a first string, determining that the user is in the Autonomous will state, and open all the usage rights of the operating program; and when the password string conforms to a second string, it is determined that the user is in the non-autonomous will state, and the partial usage rights of the operating program are opened. The identity verification method of claim 2, wherein the input interface comprises a password input block and a non-password input block, the password input block is configured to provide an input of the password string, and based on the In the eye movement state, the step of performing the autonomous will confirmation process includes: determining whether the non-password point selection block is selected based on the eye movement state; and determining that the number of times the non-password input block is selected is a predetermined number of times The user is in the state of involuntary will. The method for verifying identity according to claim 2, wherein the step of inputting the password string through the input interface comprises: Determining the eye movement state of the user's eyes when looking at one of the plurality of input units in the input interface, wherein the eye movement state includes at least one of a gaze time, a pupil displacement amount, and a pupil movement direction And determining one of the input units to be output based on the blinking action. The method for verifying identity according to claim 1, wherein the biometric feature is a facial feature, an iris feature, or a white blood vessel texture feature. An identity verification device includes: a display unit for displaying an input interface; an image capture unit for capturing a facial image sequence of a user; and a control unit coupled to the display unit and the image capture a unit, wherein the control unit analyzes the facial image sequence of the user to determine whether a biometric meets a predetermined feature, and displays the input interface after the biometric meets the preset feature; the control unit Performing an eye tracking algorithm on the facial image sequence to detect a blinking motion of the user, and performing a mental power confirmation through the blinking motion, and when the blinking motion conforms to a preset motion, the control The unit determination is confirmed by the mental intelligence, wherein after the control unit determines that the eye movement state is detected by the mental power, and the eye movement state is detected based on the eye movement state, the user is determined to determine the user. It is an identity verification process in an autonomous will state or a non-autonomous will state. The identity verification device of claim 8, wherein the control unit receives a password string input through the input interface to activate the identity verification program; and when determining that the user is in the voluntary will state, The control unit initiates an operational procedure; and upon determining that the user is in a non-autonomous will state, the control unit generates a warning signal and initiates the operational procedure. The identity verification device of claim 9, wherein the control unit determines whether an alert option in the input interface is triggered based on the eye movement state, and when the alert option is triggered, determining that the user is in the non- Autonomous will state. The identity verification device of claim 9, wherein when the password string conforms to a first string, the control unit determines that the user is in the autonomous will state, and opens all usage rights of the operation program. When the password string conforms to a second string, the control unit determines that the user is in the non-autonomous will state and opens a partial usage right of the operating program. The identity verification device of claim 9, wherein the input interface comprises a password input block and a non-password input block, wherein the password input block is used to provide input of the password string, the control unit Determining whether the non-password point selection block is selected based on the eye movement state, and determining that the user is in the non-autonomous will state when determining that the number of times the non-password input block is selected is a predetermined number of times. The identity verification device of claim 9, wherein the control unit determines that the user's eyes are looking at the plurality of inputs included in the input interface. Entering the eye movement state of one of the units, and determining one of the input units to be output based on the blinking motion; wherein the eye movement state includes a gaze time, a pupil displacement amount, and a pupil movement direction At least one of them. The identity verification device of claim 8, wherein the biometric feature is a facial feature, an iris feature, or a white blood vessel texture feature. A computer program product comprising a computer readable recording medium, the computer readable recording medium recording at least one program instruction, the at least one program instruction being executed after loading an electronic device to execute the patent application scope item 1 to The steps of any of the items 7. An identity verification system includes: a local device comprising: a display unit for displaying an input interface; and an image capture unit for capturing a facial image sequence of a user; and a server device for transmitting The Internet is connected to the local device to receive the facial image sequence from the local device. The server device includes: a control unit that analyzes the facial image sequence of the user to determine whether a biometric is Matching a preset feature, and after determining that the biometric meets the preset feature, causing the local device to display the input interface to the display unit, and the control unit performs an eye tracking algorithm on the facial image sequence In order to detect a blinking motion of the user and perform a blinking motion through the blinking action The mental power confirms, and when the blinking action conforms to a preset action, the control unit determines to confirm by the mental force, wherein after the control unit determines to confirm the eye movement state of the user after confirming by the mental force, And performing an autonomic confirmation procedure based on the eye movement state, thereby determining whether the user is in an autonomous will state or an involuntary will state to perform an identity verification procedure.