KR102611467B1 - An apparatus related to user identification, authentication, encryption using biometrics technology and method for operation the same - Google Patents

Abstract

Various embodiments of the present disclosure relate to electronic devices and operating methods related to user identification, authentication, and encryption that are more improved and convenient than existing technologies by utilizing biometric recognition technology, etc.
A method of operating an electronic device includes capturing a user's eye movements, including involuntary eye movements, and identifying the user based on the captured involuntary eye movements. It may include generating a pattern, storing the unique pattern in a non-volatile storage device in a security area, and authenticating the user with the electronic device based on the stored unique pattern. Various other embodiments may also be possible.

Description

User identification, authentication and encryption device using biometric technology and method of operation thereof

The present invention generally relates to electronic devices and methods of operating the same related to user identification, authentication, and encryption. In particular, it relates to electronic devices and operating methods related to user identification, authentication, and encryption using various biometric technologies.

User access to electronic devices and databases is primarily achieved through login names and passwords. Recently, as the use of portable electronic devices such as laptop computers and smartphones has increased, the need for accurate authentication of electronic devices and users has become more important to reduce the risk of unauthorized use and misuse of data. For example, as many types of healthcare electronic devices emerge, the privacy of health data acquired on mobile healthcare electronic devices is becoming important. Additionally, as banking and payments using mobile electronic devices increase, the importance of authenticated use is increasing.

This present disclosure seeks to provide electronic devices and operating methods related to user identification, authentication, and encryption that are more improved and convenient than existing technologies by utilizing biometric recognition technology, etc.

User authentication for use of local electronic devices is often performed by a remote server. Software applications on local electronic devices often store the user's login name and password to facilitate use of the electronic device and software applications. Protecting access to your electronic devices by protecting you and your login names and passwords becomes even more important when your electronic devices are lost or stolen. In today's situation where electronic devices are being used as a means of payment, such as credit card transactions, protecting local access to user electronic devices is becoming more and more important.

Biometrics are used for user authentication to provide enhanced protection of mobile electronic devices. There are two approaches to biometrics: anatomical (e.g. physical form, such as fingerprints, iris scans, blood vessels, facial scans, DNA, etc.) and behavioral approaches (typing or keystrokes, handwritten signatures, voice or intonation). For example, aspects of the user's anatomy, such as fingerprints, are used for local authentication of the user and for access control to electronic devices.

A method of operating an electronic device according to an embodiment of the present disclosure includes collecting a motion signal from a part of the user's body from a motion sensor, filtering unwanted signals from the motion signal, and filtering the filtered motion signal. Neuro-muscular micro-motion data is extracted from the signal, unique characteristics are extracted by performing signal processing and feature extraction on the neuro-muscular micro-motion data, and neuro-mechanical analysis is performed based on the unique characteristics. Perform identifier (neuro-mechanical identifier) generation.

A method of locally authenticating an authenticated user and controlling access to an electronic device in one embodiment of the present disclosure includes using the electronic device to detect a user in response to a micro-motion signal detected from a part of the user's body. Generating a neurodynamic fingerprint for

A matching rate of the neurodynamic fingerprint is generated in response to an authenticated user calibration parameter, and user access control to the electronic device is performed in response to the matching rate.

A method of operating an electronic device according to an embodiment of the present disclosure includes capturing a user's eye movement, including an involuntary eye movement, and capturing the captured involuntary eye movement. Based on this, a unique pattern for identifying the user is generated, the unique pattern is stored in a non-volatile storage device in a security area, and the user authentication is performed with the electronic device based on the stored unique pattern.

By utilizing biometric recognition technology, etc., we provide electronic devices and operation methods related to user identification, authentication, and encryption that are more improved and convenient than existing technologies.

Figure 1 shows categories of identification methods.
Figure 2 shows a block diagram of the operating environment of electronic devices according to an embodiment.
Figure 3 shows a block diagram of the configuration of an electronic device according to an embodiment.
Figure 4 shows an example of using neural fingerprints according to an embodiment.
FIGS. 5A to 5F illustrate a flowchart or flow of operations exemplarily showing a method for generating a neurodynamic identifier according to an embodiment.
Figure 6 shows a Poincaré phase diagram for each user according to an embodiment.
Figure 7 shows CEPSTRUM analysis results of different users according to an embodiment.
Figure 8 is a flowchart illustrating a method for generating a neural eye identifier according to an embodiment.
Figure 9 shows a cross-sectional view of the eyeball of a human skull.
Figures 10A-10C show diagrams of various muscles associated with the eye causing various eye movements.
Figure 11 shows a diagram of the process for acquiring involuntary eye movements.
Figures 12A and 12B show diagrams of an enlarged portion of the retina with a graph of involuntary eye movements over the area of the cone cells in response to the user fixating an object for a period of time.
13A-13C show diagrams of various combinations of involuntary eye movements that can be used for user identification and authentication.
14A-14C show diagrams of features that can be relevantly observed in video images of the eye that can be used to detect involuntary eye movements.
Figure 15 shows a block diagram of an electrooculography system for directly generating a user's involuntary eye movement signal.
Figure 16A shows a diagram of an electronic device with a video camera that can be used to obtain images of eye movements of a user.
FIG. 16B shows a block diagram of an electronic device with the video camera shown in FIG. 16A that can be used to obtain images of eye movements of a user.
17A and 17B show diagrams of an electronic eyeglasses with a video camera that can be used to acquire images of a user's eye movements.
Figures 18A and 18B show diagrams of a VR device with a video camera that can be used to obtain images of the user's eye movements.
Figure 19A shows a diagram of an eye containing a contact lens with an emitter device that helps obtain data regarding the user's eye movements.
19B-19D show diagrams of contact lenses with various emitter devices that can be used to help obtain data regarding a user's eye movements.
Figure 19E shows a functional block diagram of an active emitter device.
FIG. 19F shows a side view of a sensing device near an eye-mounted contact lens that can be used to obtain data regarding a user's eye movements.
Figures 20a and 20b show a device for acquiring eye movements attached for access control of a building structure.
Figures 21A and 21B show an independent eye scanner for authenticating a user to the system.

In the following detailed description of embodiments of the present disclosure, numerous specific details and various examples are set forth for thorough understanding. It is clear and obvious to those skilled in the art that the present invention can be practiced without these specific details and can be practiced with various changes or modifications within the scope of the present disclosure. In certain instances, well-known methods, procedures, components (parts, parts), functions, circuits, well-known or typical details have not been described in detail so as not to unnecessarily obscure the embodiments of the present disclosure.

The terms, words, and expressions described in the present disclosure are intended to simply describe embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. Unless otherwise defined, all expressions (or terms), including technical and scientific terms, may have the same or similar meaning in context as understood by those skilled in the art in the field to which the technology pertains. In some cases, even if expressions (or terms) are defined in the present disclosure, they are not construed to limit or exclude the scope of embodiments of the present disclosure.

Embodiments associated with the present disclosure may be implemented as a device, method, server-client device and/or method, device and/or method collaboration, chipset, computer program, or any combination of the above-described, etc. . Accordingly, the embodiment may be implemented as entire hardware (including chipsets), as entire software (firmware, any form of software, etc.), or as a combination of software and hardware. Software and hardware forms can be described as modules, units (parts or components), components (parts, parts), blocks, components, members, systems, subsystems, etc. Additionally, this embodiment may be implemented in the form of program code (including computer files) usable on a computer implemented in media.

The expressions “one embodiment,” “one embodiment,” “an embodiment,” “an embodiment,” “one example,” and “example” refer to specific features described in connection with an example or embodiment of the present disclosure. , may mean structure or characteristics. Accordingly, these expressions used herein do not necessarily all refer to the same embodiment or example. Additionally, specific functions, structures or characteristics may be combined in any suitable combination and/or sub-combination in one or more embodiments or examples.

The expression “one” or “one” in the singular form may include the plural form unless the text clearly states otherwise. For example, “a sensor” may refer to one or multiple sensors.

In some cases, the expressions "first", "second", "first", "second", "first" or "second" are used to describe various elements, but these elements are not defined by these expressions. is not limited by These expressions may be used to distinguish one component from another and may be independent of the order or importance of the components. For example, a first sensor may be referred to as a second sensor, and likewise, a second sensor may be referred to as a first sensor. The first sensor and the second sensor are both sensors, but may not be the same sensor.

The expression “and/or” can include any possible combination of one or more listed items. For example, “A or B”, “at least one of A and B”, “at least one of A or B”, “one or more of A or B”, “one or more of A and B”, The expressions “A and/or B”, “at least one of A and/or B” or “one or more of A and/or B” mean “include at least one A”, “include at least one B” ", or "contains both at least one A and at least one B".

“is (have)”, “is (have)”, “may be (have)”, “includes”, “includes”, “may include”, “consists of”, “consisted of” The expression ", or "may consist of" or "includes" refers to the presence of components, features, steps, operations, functions, numerical values, components (parts, parts), members, or combinations thereof. indicates, but excludes the addition or presence of one or more components, features, steps, operations, functions, numerical values, components (parts, parts), members, or combinations thereof. no. For example, a method or device consisting of a list of components is not necessarily limited to consisting of only these components, but may include components not explicitly listed in the method or device. You can.

When a component is said to be “connected to” or “coupled to or coupled with” another component, it means that the component is directly connected to, or at least one or more of, the other components. It can be said to be connected or combined through other components above. On the other hand, when a component is said to be “directly connected” or “directly coupled” to another component, it can be said to be connected or combined without going through another component.

In the present disclosure, embodiments of various types of electronic devices and operating methods related to user identification, authentication, and encryption are described.

In one embodiment, the electronic device includes a smartphone, tablet computer, telephone, mobile phone, telephone, e-book reader, navigation device, desktop computer, laptop computer, workstation computer, server computer, camera, camcorder, Electronic pens, wireless communication equipment, access points (AP or wireless router), projectors, electronic boards, copiers, watches, glasses, head mount devices, wireless headsets/earphones, electronic clothing, various types of wearable devices, TVs, DVD players, Audio players, digital multimedia players, electronic picture frames, set-top boxes, TV boxes, game consoles, remote controllers, bank ATMs, payment system devices (including POS and card readers), refrigerators, openers, microwave ovens, air conditioners, vacuum cleaners, washing machines, Dishwashers, air purifiers, home automation control devices, smart home devices, various types of home appliances, security control devices, electronic locking/unlocking devices (including keys and locks), electronic signature receiving devices, various types of security system devices, blood pressure Measuring devices, blood sugar monitoring devices, heart rate monitoring devices, body temperature measuring devices, magnetic resonance imaging devices, computed tomography devices, various types of medical measuring devices, various types of medical devices, water meters, electricity meters, gas meters, electromagnetic wave meters, It may be a thermometer, various types of measuring devices, artificial intelligence (AI) devices, artificial intelligence speakers, artificial intelligence robots, various types of Internet of Things devices, or similar things.

The electronic device may be a combination or part of one or more of the above-described devices. In one embodiment, an electronic device may be part of a piece of furniture, building, structure, or machine (including a vehicle, automobile, airplane, or ship), or may be in the form of an embedded board, chipset, computer file, or sensor of any kind. The electronic device of the present disclosure is not limited to the above-mentioned devices and may be a new type of electronic device as technological development advances.

1 is a diagram showing the categories of identification methods.

Referring to Figure 1, various known anatomical and behavioral identification methods can be identified. Behavioral identification methods are related to the user's actions or habits. Known anatomical identification methods involve physical characteristics of the user, such as fingerprints, iris eye scans, blood vessels, facial scans, and/or DNA.

Some user movements are habitual or part of the user's movement repertoire. For example, signing a document is a motion that creates a user's behavioral habits. In general, the motion of signing using a writing instrument is a macro motion or large scale motion. Since most of these movements are movements based on the user's consciousness or intention, they can be called voluntary movements. For example, from the large movements of a signed signature, one can determine by eye whether the person signing the letter is left-handed or right-handed.

While these large motions can be useful, there are also micro motions, which are very small motions that a user creates when they sign, make another motion, or simply do nothing. These micro-motions are neurally derived or neurally based and are invisible to the eye. Therefore, it belongs to involuntary movement, not movement based on the user's consciousness or intention. These micro-motions of the user are due to the unique neuromuscular anatomy of each human and may also be referred to as neuro-derived micro-motions in the present disclosure. These micromotions are also linked to motor control processes from the individual's motor cortex to the hands. At least one sensor, signal processing algorithm, and/or filter may be used to obtain an electronic signal (“motion signal” and “micro motion signal”) including the user's neurally induced micro-motion. Of particular interest are micro-motion electronic signals, which represent the user's micro-motions within the motion signal.

The user's micro-motions are related to cortical and subcortical control of motor activity in the brain or other nervous systems of the body. Like mechanical filters, an individual's specific anatomy and musculoskeletal system can influence the user's micro-motions and contribute to motion signals containing the user's micro-motions. This contributed signal is a muscle operation signal caused by a nerve signal and can be called neuromuscular tone. Motion signals obtained from the user may reflect part of a proprioceptive control loop that includes the brain and proprioceptors present in the user's body.

When motion signals are properly analyzed for micro-motion signals representing the user's micro-motions, the resulting data can generate unique and stable physiological identifiers, especially neurological identifiers, which can be used as unencrypted signatures. This unique identifier can be called the user's neuro-mechanical fingerprint. Neurodynamic fingerprints may also be used in this disclosure by the term “NeuroFingerPrints (NFP).”

Figure 2 is a block diagram showing the operating environment of electronic devices according to an embodiment of the present disclosure.

As shown in FIG. 2, the electronic device 201 includes a processing unit 210, a sensor 220, an input/output interface 230, a display 240, a neural fingerprint (NFP) accelerator 250, and a memory 260. ), power system 270, communication interface 280, etc.

It can be understood that this is just one example of an embodiment of the present disclosure. The electronic devices 201, 202, 203, 204, and 205 may be composed of more or fewer components than the number shown, and may be composed of two or more components combined, or may be configured by setting or arranging the components. may be configured differently. The various components shown may be implemented in hardware, software, or a combination of hardware and software. Electronic devices 201, 202, 203, 204, and 205 may be connected to each other through a network 206 or a communication interface 280.

The processing unit 210 may include at least one central processing unit, and the at least one central processing unit may include one or more processing cores. In addition to the heavy processing unit, the processing unit 210 may include at least one auxiliary processor, a communication-only processor, a signal processing processor, a graphics processing processor, a low-power sensor control processor, or a special-purpose controller. In addition, it is equipped with internal volatile and non-volatile memories of various hierarchical structures to perform initial booting operations of electronic devices, operations for communication with external electronic devices, operations for downloading initial booting or loading-related programs from external electronic devices, interrupt operations, and program operations. It can perform functions such as operations to improve performance during execution. The processing unit receives commands in the form of programs through memory, communication modules, etc., decodes the received commands, performs operations according to the decoded commands, processes data, stores results, and controls related peripheral devices. It can perform identification, authentication, encryption, or functions utilizing fingerprints (NFP). The processing unit is classified into a processor, an application processor (AP), a central processing unit (CPU), a micro controller unit (MCU), a controller, etc. by those skilled in the art. They are often used interchangeably.

The sensor 220 can detect or measure the state and physical quantities of an electronic device and convert them into electrical signals. The sensor 220 includes an optical sensor, RGB sensor, IR sensor, UV sensor, fingerprint sensor, proximity sensor, compass, acceleration sensor, gyro sensor, barometer, grip sensor, magnetic sensor, iris sensor, and GSR (Galvanic Skin Response) sensor. , may include an electroencephalography (EEG) sensor, etc. The sensor 220 collects motion signals from parts of the user's body and transmits them to at least one component of the electronic device 201, including the processing unit 210 of the electronic device and the neural fingerprint (NFP) accelerator 250. Thus, identification, authentication, encryption, or functions using these related to neural fingerprint (NFP) can be performed.

The input/output interface 230 is an input interface that receives inputs including signals and/or commands from the user or the outside of the electronic device 201 and transmits them to the inside of the electronic device, and sends signals or commands to the outside of the user or the electronic device 201. It can serve as an interface that converts output including commands. For example, input buttons, LEDs, vibration motors, various serial interfaces (e.g. USB (Universal Serial Bus), UART (Universal asynchronous receiver/transmitter), HDMI (High Definition Multimedia Interface), MHL (Mobile High-definition Interface) Link), IrDA (Infra-red Data Association), etc.).

The display 240 can display various contents such as images, text, and video to the user. The display 240 may be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or a hologram output device. The display 355 may include a display driving circuit (DDI) and a display panel, and the display driving circuit transmits an image driving signal corresponding to the image information received from the processing unit 301 to the display panel to display a preset frame. Video can be displayed according to the frame rate. The display driving circuit may be implemented in the form of an IC and may include components such as a video memory capable of storing image information, an image processing unit, a display timing controller, and a multiplexer. The display 240 may include a device that performs input functions such as a touch screen, an electronic pen input panel, a fingerprint sensor, and a pressure sensor, and/or an output device such as haptic feedback. Depending on the specifications of the electronic device 201, the display 240 may not be optionally provided, and may be composed of at least one very simple light emitting diode. In order to collect a motion signal from a part of the user's body, the display 220 shows the user a position where a part of the user's body touches the electronic device 201, a state in which motion signal acquisition begins, a state in which a motion signal is being acquired, By performing the function of displaying the collection start state, such as the acquisition completion state of the motion signal, identification, authentication, encryption, or functions utilizing these related to neural fingerprint (NFP) can be performed.

The neural fingerprint accelerator 250 is used to speed up the performance of calculations that process signals obtained from parts of the user's body, or to perform identification, authentication, encryption, or functions utilizing neural fingerprints (NFP). It performs some or all of the functions to improve the performance of the overall system.

Memory 260 includes volatile memory 262 (e.g., Dynamic RAM (DRAM), Static RAM (SRAM), or Synchronous Dynamic RAM (SDRAM), etc.), non-volatile memory 264 (e.g., NOR flash) Memory, NAND flash memory, EPROM (Erasable and Programmable ROM), EEPROM (Electrically Erasable and Programmable ROM), HDD (Hard Disk Drive), SSD (Solid State Drive), SD (Secure Digital) card memory, micro SD memory, MMC (Multimedia Card, memory, etc.) may be included. The non-volatile memory 264 includes at least one bootloader, operating system 291, communication function (292) library, device driver 293, neural fingerprint (NFP) library 294, application 295, and user data 296. ), etc. can be saved. Power is applied to the volatile memory 262 and the electronic device begins to operate. The processing unit 210 performs an operation of loading programs or data stored in the non-volatile memory into the volatile memory 2620, and processes the main memory through a series of operations that interface with the processing unit 210 during the operation of the electronic device. can perform its role.

The power system 270 may perform functions such as supplying power to the electronic device 201, controlling and managing it. The power system may include a Power Management Integrated Circuit (PMIC), a battery 272, a charging IC, a fuel gauge, etc. The power system can receive alternating current or direct current power as input. It may include wired charging and wireless charging functions that can charge the supplied power to the battery 272.

The wireless communication interface 280 may include, for example, cellular communication, Wi-Fi communication, Bluetooth, GPS, RFID, NFC, etc. and may configure an RF unit for wireless communication. The RF unit may include a transceiver, PAM (Power Amp Module), frequency filter, LNA (Low Noise Amplifier), or antenna.

Figure 3 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

As shown in FIG. 3, the electronic device 300 includes a processing unit 301, a camera 350, an input/output interface 353, a haptic feedback controller 354, a display 355, and short-range wireless communication 356. , external memory slot 357, sensor 370, memory 390, power system 358, clock source 361, audio circuit 362, SIM card 363, wireless communication processor 364, RF It may include a circuit 365, a neural fingerprint (NFP) accelerator 366, etc.

It can be understood that this is just one example of an embodiment of the present disclosure. An electronic device may be composed of more or fewer components than the number shown, may be composed of two or more components combined, or may have different settings or arrangements of components. The various components shown may be implemented in hardware, software, or both hardware and software.

The processing unit 301 may include at least one central processing unit 302, and the at least one central processing unit may include one or more processing cores. In addition to the heavy processing unit, the processing unit 301 may include at least one auxiliary processor, a communication-only processor, a signal processing processor, a graphics processing processor, a low-power sensor control processor, or a special-purpose controller. The processing unit 301 is in the form of a semiconductor chip and may be implemented as a System On Chip (SoC) including various components. In one embodiment, the processing unit 301 includes a GPU (Graphics Processing Unit) 320, a digital signal processing (DSP) 321, an interrupt controller 322, a camera interface 323, and a clock controller. (324), display interface (325), sensor core (326), location controller (327), security accelerator (328), multimedia interface (329), memory controller (330), peripheral device interface (331), communication/connectivity (332), internal memory (340), etc. In addition, it is equipped with internal volatile and non-volatile memories of various hierarchical structures to perform initial booting operations of electronic devices, operations for communication with external electronic devices, operations for downloading initial booting or loading-related programs from external electronic devices, interrupt operations, It can perform functions such as operations to improve performance during program execution. The processing unit 301 receives instructions in the form of a program through the memory 390, communication/connectivity 332, or wireless communication processor 364, decodes the received instructions, and performs operations according to the decoded instructions. It is possible to perform identification, authentication, encryption, or functions utilizing neural fingerprints (NFP) by processing data, storing results, and performing operations to control associated peripheral devices. The processing unit is classified into a processor, an application processor (AP), a central processing unit (CPU), a micro controller unit (MCU), a controller, etc. by those skilled in the art. They are often used interchangeably.

The central processing unit 302 may include at least one processor core 304, 305, or 306. The central processing unit 302 may include processor cores with relatively low power consumption and high-performance processor cores with high power consumption, and a plurality of cores are grouped into a first cluster 303 and a second cluster 314. It may include at least one or more clusters such as: This structure can be said to be a technology used to improve the performance and power consumption of electronic devices by dynamically allocating cores in consideration of calculation amount and current consumption in a multi-core environment. The processor core may be equipped with technology to enhance security. ARM® processor, one of the well-known mobile processors, has installed this security technology into its processor and named it TrustZone®. For example, the first core 304, which is one physical processor core, may operate in a normal mode 307 and a security mode 308. Since the processor's registers and interrupt processing are separated, access to resources that require security (for example, peripheral devices or memory areas, etc.) can be accessed only in security mode. The monitor mode 313 serves to enable mode switching between the normal mode 307 and the security mode 308. The normal mode 307 can be converted to the security mode 308 through a special command, or the normal mode 307 can be converted to the security mode 308 due to an interrupt, etc. Applications running in the normal mode (307) and security mode (308) are separated so that they cannot affect the applications running in each mode, so applications that require high reliability are run in the security mode (308), thereby improving security. Reliability can be increased. Security can be improved by allowing some of the calculations in performing identification, authentication, encryption, or functions utilizing these related to neural fingerprint (NFP) to be operated in the security mode 308.

The camera 350 may include a lens, an optical sensor, an image processing processor (ISP), etc. for acquiring images, and can acquire still images and moving images. It may include a plurality of cameras (first camera 351, second camera 352, etc.) to provide various functions.

The input/output interface 353 is an input interface that receives inputs including signals and/or commands from the user or the outside of the electronic device 300 and transmits them to the inside of the electronic device, and transmits signals or commands to the outside of the user or the electronic device 300. It can serve as an interface that converts output including commands. For example, input buttons, LEDs, vibration motors, various serial interfaces (e.g., USB (Universal Serial Bus), UART (U Universal asynchronous receiver/transmitter), HDMI (High Definition Multimedia Interface), MHL (Mobile High- definition Link), IrDA (Infra-red Data Association), etc.).

The haptic feedback controller 354 includes, for example, a vibration motor called an actuator to provide the user with the ability to feel a specific sensation through tactile sensation by causing the user to feel vibration. You can.

The display (touch-sensitive display) 355 can display various contents such as images, text, and video to the user. The display 355 may be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or a hologram output device. The display 355 may include a display driving circuit (DDI) and a display panel, and the display driving circuit transmits an image driving signal corresponding to the image information received from the processing unit 301 to the display panel to display a preset frame. Video can be displayed according to the frame rate. The display driving circuit may be implemented in the form of an IC and may include components such as a video memory capable of storing image information, an image processing unit, a display timing controller, and a multiplexer. Depending on the specifications of the electronic device 300, the display 355 may not be optionally provided, and may be composed of at least one very simple light emitting diode. The display 355 may further include a device that performs input functions such as a touch screen, an electronic pen input panel, a fingerprint sensor, and a pressure sensor, and/or an output device such as haptic feedback. In order to collect a motion signal from a part of the user's body, the display 355 shows the user a position where a part of the user's body touches the electronic device 300, a state in which motion signal acquisition begins, a state in which a motion signal is being acquired, By performing the function of displaying the collection start state, such as the acquisition completion state of the motion signal, identification, authentication, encryption, or functions utilizing these related to neural fingerprint (NFP) can be performed.

The short-range wireless communication 356 uses, for example, Near Field Communication (NFC), Radio Frequency Identification (RFID), and MST (Near Field Communication) to perform communication with other electronic devices located close to the electronic device 300. Magetic Secure Transmission), etc.

The external memory slot 257 may include an interface for mounting a memory card (eg, SD card, Micro SD card, etc.) to expand the storage space of the electronic device 300.

The power system 358 may perform roles such as supplying power to, controlling, and managing the electronic device 300. The power system may include a Power Management Integrated Circuit (PMIC), a battery 359, a charging IC 360, a fuel gauge, etc. The power system can receive alternating current or direct current power as input. It may include wired charging and wireless charging functions that can charge the supplied power to the battery 359.

The clock source 361 may include a system clock that serves as a standard for the operation of the electronic device 300 and at least one frequency oscillator for transmitting and receiving RF signals.

The audio circuit 362 may include an audio input unit (e.g., microphone), an audio output unit (receiver, speaker, etc.), and a codec that performs conversion between audio signals and electrical signals, through which the user It can provide an interface between and electronic devices. The audio signal acquired through the audio input unit can be converted into an analog electrical signal, sampled, converted into digital data, and transmitted to other components (e.g., a processing unit) in the electronic device 300 to perform audio processing, Digital audio data received from other components within the electronic device 300 can be converted into analog electrical signals and an audio signal can be generated through an audio output unit.

The SIM card (63) is an IC card that implements a subscriber identification module for identifying subscribers in cellular communication. In most cases, the SIM card is mounted in the slot provided in the electronic device 310. Depending on the type of electronic device, it can also be implemented in the form of an embedded SIM combined inside the electronic device. Each SIM card can have its own unique number recorded and a fixed number called ICCI (Integrated Circuit Card). Identifier) and IMSI (International Mobile Subscriber Identity) information that varies for each subscriber line.

The wireless communication processor 364 may include, for example, cellular communication, Wi-Fi communication, Bluetooth, GPS, etc. Through the wireless communication processor 364, identification, authentication, encryption, or functions utilizing them related to neural fingerprint (NFP) can be performed through cooperation with at least one other electronic device (including a server) through a network.

The RF circuit 365 may include a transceiver, a power amplifier module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. By exchanging control information and user data with the wireless communication processor and processing unit, transmission and reception can be performed through radio frequencies in a wireless environment.

The neural fingerprint accelerator 366 is used to speed up the performance of calculations that process signals obtained from parts of the user's body, or to perform identification, authentication, encryption, or functions utilizing neural fingerprints (NFP). It performs some or all of the functions to improve the performance of the overall system.

The sensor 370 can detect or measure the state and physical quantities of an electronic device and convert them into electrical signals. The sensor 370 includes a compass 371, an optical sensor 372, a fingerprint sensor 373, a proximity sensor 374, a gyro sensor 375, an RGB sensor 376, a barometer 378, and a UV sensor 379. ), a grip sensor 380, a magnetic sensor 381, an acceleration sensor 382, an iris sensor 383, etc. The sensor 370 may further include an IR sensor, a Galvanic Skin Response (GSR) sensor, an Electroencephalography (EEG) sensor, etc., which are not shown in the drawing.

The sensor 370 collects motion signals from parts of the user's body and transmits them to at least one component of the electronic device 300, including the processing unit 301 of the electronic device and the neural fingerprint (NFP) accelerator 366. Thus, identification, authentication, encryption, or functions utilizing them related to neural fingerprint (NFP) can be performed.

The memory 390 includes volatile memory 391 (e.g., Dynamic RAM (DRAM), Static RAM (SRAM), or Synchronous Dynamic RAM (SDRAM), etc.), non-volatile memory 392 (e.g., NOR flash) Memory, NAND flash memory, EPROM (Erasable and Programmable ROM), EEPROM (Electrically Erasable and Programmable ROM), HDD (Hard Disk Drive), SSD (Solid State Drive), SD (Secure Digital) card memory, Micro SD memory, MMC (Multimedia Media Card, memory, etc.) may be included. The non-volatile memory 392 includes at least one bootloader, operating system 393, communication function library 394, device driver 395, neural fingerprint (NFP) library 396, application 397, and user data 398. ), etc. can be saved. The volatile memory 391 starts operating when power is applied, and the processing unit 301, etc., performs an operation of loading programs or data stored in non-volatile memory, thereby interfacing with the processing unit 301 during the operation of the electronic device. It can perform the role of main memory through a series of operations.

The electronic device 300 may acquire a signal from a part of the user's body through the sensor 370, and transmit the acquired signal to at least one of the processing unit 301, the neural fingerprint accelerator 366, and the memory 390. Through mutual operation, identification, authentication, encryption, or functions related to neural fingerprint (NFP) can be performed. Identification, authentication, encryption, or functions related to neural fingerprint (NFP) can be performed independently by the electronic device 300, and through cooperation with at least one other electronic device (including a server) through a network. It can be done.

Figure 4 is a diagram showing an example of using neural fingerprint (NFP) according to an embodiment.

Referring to FIG. 4, when a user holds an electronic device with his or her hand, a motion signal for the movement of the person's hand is acquired by the sensor of the electronic device and converted into an electrical signal, and nerve-induced micro-motion (neural-induced micro-motion) is generated from the converted signal. Neuro-derived micro motion or neuromuscular tone can be extracted.

If the extracted neurally induced micro-motion or neuromuscular tone is properly analyzed, a unique and stable physiological identifier, especially a neurological identifier, can be generated, which can be called a neural fingerprint (NFP) identifier unique to each user. In other words, the neural fingerprint can be recognized by holding the user's electronic device, and through this, a comparison is made with the user's neural fingerprint already stored in the security area of the electronic device to determine whether the user is an authorized user. The case can be judged.

In detail, motion signals are collected from parts of the user's body from a motion sensor, unwanted signals are filtered from the motion signals, and neuro-muscular micro-motion is generated from the filtered motion signals. ) Data can be extracted, unique characteristics can be extracted by performing signal processing and feature extraction on the neuromuscular micro-motion data, and neuro-mechanical identifier generation can be performed based on the unique characteristics. .

Additionally, the electronic device may be used to generate a neurodynamic fingerprint for the user in response to micro-motion signals detected from parts of the user's body, to locally authenticate the authenticated user and control access to the electronic device. A matching rate of the neurodynamic fingerprint may be generated in response to the user calibration parameter, and user access control to the electronic device may be performed in response to the matching rate. These operations include credit card payment, collection and processing of medical information from medical devices, authentication or encryption methods such as authentication keys of security chipsets, authentication or encryption such as user login in a cloud environment, authentication or encryption by being mounted on wearable devices, and various other methods. It can be used in a variety of fields, such as unlocking locks (door locks, screen unlocks, car keys, and unlocking various devices).

5A to 5F are flowcharts or flow diagrams illustrating a method for generating a neurodynamic identifier according to an exemplary embodiment. Figure 6 is a graph showing a Poincaré phase diagram for each user according to an embodiment. Figure 7 is a graph showing Cepstrum analysis results of different users according to an embodiment. To facilitate understanding, FIGS. 5A to 5F will be described assuming the electronic device shown in FIG. 3.

Referring to FIG. 5A, the electronic device 300 can collect motion signals from parts of the user's body from a motion sensor (510). The motion sensor may include a sensor capable of detecting the user's movement or vibration. For example, the motion sensor may be a compass 371, a gyro sensor 375, an acceleration sensor 382, a camera 350, an optical sensor, or a touch sensor of a touch-sensitive display 355. Electronic devices can detect movement and vibration that occur in parts of the user's body that are in contact, convert them into electrical signals through motion sensors, and convert them into digital signals that can be processed through sampling and quantization. When the camera is driven using the motion sensor, eye movement signals can be collected. The motion sensor may collect signals by sampling at a predetermined sampling frequency over a predetermined time range (eg, 5, 10, 20, 30 seconds, etc.).

The sampling frequency was, for example, between 3 and 30 Hz, taking into account the frequency range of 3 Hz to 30 Hz, which is the frequency range in which the frequencies of neuromuscular micromotions resulting from the unique neuromuscular anatomy of humans, either nerve-derived or nerve-based, are predominantly observed. It can be 60Hz, 200Hz, 250Hz, 330Hz, etc., which is twice as much. The mainly observed frequency range can be observed by determining a different frequency range depending on the application using the neural fingerprint, and signals of other frequency ranges can also be used to improve accuracy or performance. The motion sensor may further perform operations to remove noise or improve signal quality to increase the quality of the digital signal. When the electronic device 300 is implemented as a portable device, the issue of power consumption may become important. To this end, the electronic device 300 can operate in a sleep mode, and the electronic device 300 can receive the collected and digitally converted motion signal to the processing unit 301 through the sensor core 326. The sensor core 326 may further perform operations to remove noise or improve signal quality in order to increase the quality of the digital signal received from the motion sensor. The processing unit may frequently enter a sleep mode to increase the efficiency of power consumption of the electronic device 300. In general, in sleep mode, various methods can be applied, such as cutting off the power of some components in the electronic device 300, switching to a low power mode, or lowering the frequency of the operating clock to minimize power consumption. When the processing unit 301 enters the sleep mode, the efficiency of power consumption can be said to increase, but since there may be a delay in the mutual response between the user and the electronic device, an auxiliary processor such as the sensor core 326 It may be provided inside the processing unit or in the electronic device. Even if the processing unit 301 enters the sleep mode, the sensor core 326 observes signal detection from the sensors 370 and determines that continuous processing by the processing unit 301 is required. An operation may be performed to wake up the processing unit 301 from the sleep mode through a bus or an interrupt signal. Neural fingerprint (NFP)-related identification, authentication, encryption, or functions utilizing these can be said to be operations that require security. In this case, the operation of controlling data collection of the motion sensor may be performed by, for example, switching the first core 304 in the processor unit 301 to the security mode 308. A signal transmitted through a bus or interrupt by the motion sensor or the sensor core 326 can be input to the monitor mode 313 to switch the first core 304 to the security mode 308. The core that has entered the security mode 308 can access or control system resources of the electronic device that are accessible only in the preset security mode.

The electronic device 300 may extract neuromuscular micro-motion data from the motion signal collected from the motion sensor (520). The operation of extracting signals from motion signals is either processed by the central processing unit 302 of the processing unit 301, or in cooperation with GPU 320, digital signal processing (DSP) 321, neural fingerprint (NFP) accelerator, etc. Signals can be extracted.

To extract micro motion data from the collected motion signals, filtering can be performed to remove unnecessary signals (522). Unwanted signals may include, for example, noise, macro motion signals, distortion due to gravity, etc. In signals obtained from a motion sensor, signals for a certain period of time (e.g., up to about 1 to 2 seconds) at the beginning of signal collection or at the end of signal collection may contain a large amount of signals of the user's macro motion and may contain a large amount of signals of the user's macro motion. There may be some shaking, so you can throw it all away. Since power noise may occur in signals collected when an electronic device is charging, the signal can be filtered taking into account the characteristics caused by power noise. The frequency of neuromuscular micromotions, which are either nerve-derived or nerve-based and result from the unique neuromuscular anatomy of humans, are primarily observed in the range of 3 Hz to 30 Hz. Therefore, signals in the range of 3Hz to 30Hz or 4Hz to 30Hz can be extracted from the collected motion signals by performing a signal processing algorithm. Depending on the characteristics of the unwanted signal to be removed, the cutoff frequency of the band filter of the signal processing algorithm can be changed. For example, in one embodiment, signals in the range of 4Hz to 30Hz may be extracted, and in another embodiment, signals in the range of 8Hz to 30Hz may be extracted. In another embodiment, signals in the range of 4Hz to 12Hz or 8Hz to 12Hz may be extracted.

In one embodiment, macro motion (large scale movements of the user's body, arm movements, walking, running, jogging, hand movements, etc.) is separated from the small signal amplitude of the micro motion to remove it from the collected motion signals. A signal analysis method that classifies signals can be used. For example, the interpretation method in "Time Series Classification Using Gaussian Mixture Models of Reconstructed Phase Spaces" (by Richard J. Povinelli et al., IEEE Transactions on Knowledge and Data Engineering, Vol. 16, No. 6, June 2004); Or, for separation of signals due to voluntary movement, see "Estimation of Physiological Tremor from Accelerometers for Real-Time Applications" (by Kalyana C. Veluvolu et al., Sensors 2011, vol. 11, pages 3020-3036, attached hereto in the appendix )'s analysis method can be used to remove macro motion signals and extract micro motion signals.

The electronic device 300 may extract unique characteristics from the extracted neuromuscular micro-motion data (530). Poincaré's phase information, CEPSTRUM analysis, analysis using the Lyapunov exponent of chaos analytics, analysis based on Reconstructed Phase Space (RPS), and various signal processing technologies. Unique characteristic values can be composed of at least one linear combination of components. As shown in Figure 6, looking at the Poincaré phase diagram for each user, each user (610, 620, 630, 640) has a unique and different signal pattern, and the center of mass of the pattern for each user (612, 622, 632) , 642) can also be seen to be unique and different for each user. At this time, signal processing and feature extraction can be performed to extract unique characteristics for each user from the micro motion data (532). In order to extract hidden patterns within each user's micro-motion, they can be detected through signal spectrum analysis, signal inverse spectrum analysis, etc. For example, the well-known CEPSTRUM analysis can detect patterns of repeating cycles and frequency intervals. In FIG. 7, looking at the CEPSTRUM analysis results of different users in one embodiment, five CEPSTRUM amplitude peak values (P1, P2, P3, P4, P5) extracted from user 1 (710) ) and the five CEPSTRUM Amplitude peak values (P1, P2, P3, P4, P5) extracted from user 2 (720) are different from each other, and the five cuprency extracted from user 1 (710) are different from each other. Quefrency (F1, F2, F3, F4, F5) and the five Quefrency (F1, F2, F3, F4, F5) extracted from user 2 (720) are unique to each user with different values and intervals. Since it can be seen that it is showing certain characteristics, these results can be included in the unique pattern information for each user.

The electronic device 300 may generate a neuromechanical identifier from the neuromuscular micro-motion data (540). In one embodiment, a set of result values from the operation of extracting (530) unique characteristics from the extracted neuromuscular micro-motion data is generated as a neuromechanical identifier in the form of a data structure with multidimensional values and stored in non-volatile memory. Save. The neurodynamic identifier may be referred to as a neural fingerprint unique to each user.

In one embodiment, when performing the operations of the neurodynamic identifier generation method 510, 520, 530, and 540, the electronic device 300 uses a cluster of high-performance processor cores (e.g., the first cluster 303). If is a cluster of high-performance cores, the first cluster 303 may be assigned to ) and performed.

In one embodiment, when performing the operations of the neurodynamic identifier generation method (510, 520, 530, 540), the operations are performed in the security mode of the core in the processor unit 301 (e.g., the operations are performed in the first core ( When assigned to 304), it can be switched to the security mode (308) and operated. The core that has entered the security mode 308 can access or control system resources of the electronic device that are accessible only in the preset security mode and perform the operations of the neurodynamic identifier generation method 510, 520, 530, and 540. .

In one embodiment, when performing the operations of the neuromechanical identifier generation method 510, 520, 530, and 540, when the electronic device 300 is in a sleep mode when the operations begin, the sensor core 326 By detecting signals from the sensors 370 and waking up the processing unit 301 from the sleep mode through a bus or an interrupt signal, the operation of the neurodynamic identifier generation method 510, 520, 530, and 540 is performed. It can be done.

In one embodiment, when performing the operations of the neuromechanical identifier generation method (510, 520, 530, 540), the operations collect eye movement signals by driving a camera or optical sensor with a motion sensor to generate a neuromechanical identifier (e.g. For example, a neural eye identifier) can be generated.

Referring to FIG. 5B, the electronic device 300 may sense a signal (551) from a sensor inside the electronic device 300, a signal from outside the electronic device 300, or from a short distance. The external signal generator 550 may be a part of the body that generates muscle movement induced through nerves due to the action of the human brain, or may be an object that generates a signal located in close proximity to the electronic device 300. .

The sensor can sense a signal 551 through contact or non-contact at close range from the human body, and may be comprised of part or all of the sensor 370 of FIG. 3 or at least one of the sensors constituting the sensor 370. It can be implemented by consisting of a combination of the above. It may also include sensors not described in sensor 370. The sensor converts the measured signal into an electrical signal, performs sampling and quantization of the signal at a preset sampling frequency using digital technology, and converts it into a digital signal that is easy to digitally process. The sampling frequency at this time is 3 Hz to 30 Hz, taking into account the frequency range in which the frequencies of neuromuscular micromotions, which are derived from nerves or are neuromuscular based on the unique neuromuscular anatomy of humans, are mainly observed, for example, a frequency of 30 Hz. It can be more than double 60Hz, 200Hz, 250Hz, 330Hz, etc. The mainly observed frequency range can be observed by determining a different frequency range depending on the application using the neural fingerprint, and signals of other frequency ranges can also be used to improve accuracy or performance. The motion sensor may further perform operations to remove noise or improve signal quality to increase the quality of the digital signal. When the electronic device 300 is implemented as a portable device, the issue of power consumption may become important. To this end, the electronic device 300 can operate in a sleep mode, and the electronic device 300 can receive the collected and digitally converted motion signal to the processing unit 301 through the sensor core 326. The sensor core 326 may further perform operations to remove noise or improve signal quality in order to increase the quality of the digital signal received from the motion sensor. The processing unit may frequently enter a sleep mode to increase the efficiency of power consumption of the electronic device 300. In general, in sleep mode, various methods can be applied, such as turning off the power of some components in the electronic device 300, switching to a low power mode, or lowering the frequency of the operating clock to minimize power consumption. When the processing unit 301 enters the sleep mode, the efficiency of power consumption can be said to increase, but since there may be a delay in the mutual response between the user and the electronic device, an auxiliary processor such as the sensor core 326 It may be provided inside the processing unit or in the electronic device. Even if the processing unit 301 enters the sleep mode, the sensor core 326 observes signal detection from the sensors 370 and determines that continuous processing by the processing unit 301 is required. An operation may be performed to wake up the processing unit 301 from the sleep mode through a bus or an interrupt signal. Neural fingerprint (NFP)-related identification, authentication, encryption, or functions utilizing these can be said to be operations that require security. In this case, the operation of controlling data collection of the motion sensor may be performed by, for example, switching the first core 304 in the processor unit 301 to the security mode 308. A signal transmitted through a bus or interrupt by the motion sensor or the sensor core 326 can be input to the monitor mode 313 to switch the first core 304 to the security mode 308. The core that has entered the security mode 308 can access or control system resources of the electronic device that are accessible only in the preset security mode.

In the pre-processing 552 process, a pre-processing process is performed on the signal acquired by sensing 551 from the sensor or the data stored in the storage device 557 to increase the performance and efficiency of processing at a later stage. It is possible to remove signals in a certain section at the beginning and end of signal acquisition, suppress noise components, or suppress components of features that are deemed unnecessary for processing. Additionally, unnecessarily overlapping components or components in cases where the correlation between signals is high may be unnecessary, so signal components can be suppressed. Additionally, based on the application in which the signal is utilized, compression can be performed into a low-dimensional subspace through digital filters or dimensionality reduction techniques. Unwanted signals may include, for example, noise, macro motion signals, distortion due to gravity, etc. In the signal acquired from the sensor, the signal for a certain period of time (e.g., up to about 1 to 2 seconds) at the beginning of signal collection or at the end of signal collection may contain a large amount of the signal of the user's macro motion and the shaking of the electronic device. There may be some, so you can discard them all. Since power noise may occur in signals collected when an electronic device is charging, the signal can be filtered taking into account the characteristics caused by power noise. The frequency of neuromuscular micromotions, which are either nerve-derived or nerve-based and result from the unique neuromuscular anatomy of humans, are primarily observed in the range of 3 Hz to 30 Hz. Therefore, signals in the range of 3Hz to 30Hz or 4Hz to 30Hz can be extracted from the collected motion signals by performing a signal processing algorithm. Depending on the characteristics of the unwanted signal to be removed, the cutoff frequency of the band filter of the signal processing algorithm can be changed. For example, in one embodiment, signals in the range of 4Hz to 30Hz may be extracted, and in another embodiment, signals in the range of 8Hz to 30Hz may be extracted. In another embodiment, signals in the range of 4Hz to 12Hz or 8Hz to 12Hz may be extracted. In one embodiment, in order to remove macro motion (large scale movements of the user's body, arm movement, walking, running, jogging, hand movements, etc.) from the collected signals, the micro motion is separated from the small signal amplitude. Signal analysis methods can be used to classify. In the preprocessing 552 process, signals related to neuromuscular micro-motion can be extracted from signals collected from the sensor. The operation of extracting signals related to neuromuscular micro-motion from signals acquired from sensors is all processed by the central processing unit 302 of the processing unit 301, GPU 320, or digital signal processing (DSP) 321. ), the signal can be extracted in cooperation with the neural fingerprint (NFP) accelerator.

In the feature extraction (553) process, a process of extracting unique meaningful characteristics from signal components transmitted through the preprocessing (552) process is performed. These characteristics can have values in the form of a measurable multidimensional vector-type data structure with mathematical properties that can clearly distinguish user characteristics, and are also called feature vectors for convenience. Poincaré's phase information, CEPSTRUM analysis, analysis using the Lyapunov exponent of chaos analytics, analysis based on Reconstructed Phase Space (RPS), and various signal processing technologies. A feature vector can be composed of at least one linear combination of components. As shown in Figure 6, looking at the Poincaré phase diagram for each user, each user (610, 620, 630, 640) has a unique and different signal pattern, and the center of mass of the pattern for each user (612, 622, 632) , 642) can also be seen to be unique and different for each user. In order to extract hidden patterns within each user's micro-motion, they can be detected through signal spectrum analysis, signal inverse spectrum analysis, etc. For example, the well-known CEPSTRUM analysis can detect patterns of repeating cycles and frequency intervals. In FIG. 7, looking at the CEPSTRU analysis results of different users in one embodiment, five CEPSTRUM amplitude peak values (P1, P2, P3, P4, P5) extracted from user 1 (710) and the five cepstrum amplitude (CEPSTRUM width) peak values (P1, P2, P3, P4, P5) extracted from user 2 (720) are different from each other, and the five cepstrum amplitude (CEPSTRUM width) peak values (P1, P2, P3, P4, P5) extracted from user 1 (710) are different. ) (F1, F2, F3, F4, F5) and the five Quefrencies (F1, F2, F3, F4, F5) extracted from User 2 (720), each with different values and intervals unique to each user. It can be seen that it shows the characteristics, so these results can be included in the feature vector, which is unique pattern information for each user. The feature vector is used as training data for the next step of learning (553). Only for training data. The performance of the learning algorithm is not excellent, and test data is needed to determine whether it generalizes well to general data. The storage device 557 also has test feature vectors prepared in advance to be used as test data that have been developed and verified in advance, so the next Control of the flow can be transferred to the learning stage (553).

In the learning 554 step, the extracted feature vector and a pre-prepared test feature vector are used as input to select a model that can best express the feature vector and predict the model. These learning steps can generally be broadly classified into supervised learning, unsupervised learning, reinforcement learning, etc., and more specifically, k-Nearest Neighbors, Linear Regression, Logistic Regression, Support Vector Machine (SVM), Decision Tree, Random Forestes, Neural Network, k-means Means), Hierarchical Cluster Analysis (HCA), Extraction Maximization, Principal Component Analysis (PCA), Kernel PCA, Locally-Linear Embedding (LLE), There are algorithms such as t-SNE (t-distributed Stochastic Neighbor Embedding), Apriori, Eclat, etc. In the learning 554 step, a combination of at least one learning algorithm including the various algorithms above Model prediction is performed by performing learning through. A single model prediction algorithm can be used, or an ensemble learning method that can achieve better performance through a combination of various algorithms can be performed. The results of each algorithm The final model can be predicted by imposing a final score with preset weights applied. The extracted feature vectors, the models selected through learning, and at least one combination of the models' parameters are stored in a storage device to distinguish neural data for each user. Can be used as a fingerprint.

In the evaluation step 555, the model selected from the training feature vector can be evaluated. At this time, the test feature vector can be used to estimate the generalization error to determine the level of performance for feature vectors that have not yet been used. In some cases, all or part of the test feature vector may be used in the learning stage for model selection. In this case, the test feature vectors may be stored in advance in a storage device in the form of a plurality of sets. If the performance meets the required level, the model and model parameters selected in the learning stage can be used to predict new future data.

In the decision 556 step, a decision can be made about the new data by predicting a model for the newly sensed data using the model or model parameters selected in the learning step. The steps in FIG. 5B may be performed in a different order depending on the case. When learning is completed through the process of Figure 5b and the selected models and model parameter data for the neural fingerprint are prepared, a signal is sensed from a device such as a sensor (551), preprocessing is performed (552), and the preprocessed signal is A feature vector can be extracted from the data (553), and a decision (575) step can be performed through a pre-selected prediction model. If prediction models have already been learned and selected from the neural fingerprint of an authorized user of the electronic device 300, and parameters for the prediction models are determined, there is a match as to whether it is a learned model in the decision step 556. When the match percentage exceeds a certain number, it becomes possible to determine whether it is the neural fingerprint of an authorized user. If the match percentage of the data associated with the newly sensed neural fingerprint is lower than a certain value, it can be determined whether it is the neural fingerprint of an authorized user. The result of the decision 556 step can be reported through the display, or the result of authentication can be notified to the application to block the use of unwanted electronic devices.

In one embodiment, when performing operations 551, 552, 553, 554, 555, and 556 of the neurodynamic identifier generation method, in all or part of each operation, the electronic device 300 uses a cluster of high-performance processor cores ( For example, if the first cluster 303 is a cluster of high-performance cores, the first cluster 303 may be assigned to and performed.

In one embodiment, when performing operations 551, 552, 553, 554, 555, and 556 of the neurodynamic identifier generation method, in all or part of each operation, the security mode of the core within the processor unit 301 (e.g. For example, when the above operations are assigned to the first core 304, the operation may be switched to the security mode 308). The core that has entered the security mode 308 can access or control system resources of the electronic device that are accessible only in the preset security mode and perform the operation of generating the neurodynamic identifier.

In one embodiment, when performing the operations 551, 552, 553, 554, 555, and 556 of the method for generating a neurodynamic identifier, in all or part of each operation, when performance of the operations begins, the electronic device 300 When in a sleep mode, the sensor core 326 detects signals from the sensors 370 and wakes up from the sleep mode of the processing unit 301 through a bus or an interrupt signal, so that the neurodynamic The operation of creating an identifier can be performed.

In one embodiment, when performing operations 551, 552, 553, 554, 555, and 556 of the neurodynamic identifier generation method, in all or part of each operation, the sensing 551 operates a camera or optical sensor. Eye movement signals may be collected to generate a neuromechanical identifier (e.g., neuro-ocular identifier).

Referring to FIG. 5C, the electronic device processes the electrical signal detected from a part of the user's body from a device such as a sensor of the electronic device 300 by sampling and quantizing the signal at a preset sampling frequency during a preset period. Generate a digital signal that can be used (570), remove data from a certain section of the first or last part of the signal, suppress signal components due to noise, gravity, and intentional actions (571), and use various methods. Or, through at least one combination of algorithms, a preset measurable feature vector containing mathematical properties representing characteristics that can be associated with the user's neuromuscular function and distinguishing the user is extracted (572), and from the extracted feature vector Select a combination of one or more learning algorithms, select prediction models using the ensemble method, extract parameters of the selected prediction models (573), select through pre-prepared test feature vectors, and evaluate the performance of the selected prediction models (574) to obtain a final result. Decide on a model. At least one combination of the extracted feature vectors, the models selected through learning, and the parameters of the models can be stored in a storage device and used as a neural fingerprint that can be distinguished for each user. The step of FIG. 5C may be performed by omitting some steps in some cases. When learning is completed through the process of Figure 5c and the selected models and model parameter data are prepared, a signal is sensed from a device such as a sensor (570), preprocessing is performed (571), and A feature vector can be extracted (572), and a prediction or decision (575) can be made through a pre-selected prediction model. In the prediction or decision step 575, a decision can be made about the new data by predicting a model for newly sensed data using the model or model parameters selected in the learning step. If prediction models have already been selected from the neural fingerprint of an authorized user of the electronic device 300 and parameters for the prediction models have been determined, whether it is a learned model is determined in the prediction or decision step 575. When the match percentage exceeds a certain number, it becomes possible to determine whether it is the neural fingerprint of an authorized user. If the match percentage of the data associated with the newly sensed neural fingerprint is lower than a certain value, it can be determined whether it is the neural fingerprint of an authorized user. The results of the prediction or decision 575 step can be reported through the display, or the results of authentication can be notified to the application to block the use of unwanted electronic devices.

The neural fingerprint generated by the method of FIGS. 5A to 5C is used for user authentication of electronic devices, user access control, data encryption or decryption, and detecting changes in the user's health. It can be utilized. If a sensor that senses neural fingerprints is used to sense involuntary eye movements through an optical sensor such as a camera, neural fingerprints can be extracted through eye movements and used for user identification, authentication, and encryption using neural fingerprints. there is.

Referring to FIG. 5D, a neural fingerprint or a neuro-mechanical fingerprint is generated (576) by the method of FIGS. 5A to 5C, and the match rate of the neuro-mechanical fingerprint corresponding to the signal obtained from the user is calculated. Generate (577) and control access to the user's electronic device in response to the match rate (578). If the match rate has a value higher than the access level, identify the user as an authenticated user, and control the access to the user's electronic device in response to the match rate (578). Access to the electronic device is permitted (579), and if the match rate is lower than the preset level, training and learning are performed again for the authenticated user to select prediction models and update the parameters of the selected models ( 580) You can. By dividing the match rate into several sections, you can distinguish the stage at which training and learning must be re-performed for each section. If the match rate falls within the first specific section, the electronic device can voluntarily re-perform training and learning without displaying it to the user, thereby updating values such as parameters related to the neural fingerprint. If the match rate falls within the second specific section, the user may be displayed or notified through a display or input/output device that training and learning need to be re-performed. At this time, the settings of the first specific section and the second specific section can be set in advance through various combinations.

Referring to FIG. 5E, data is detected from a part of the user's body using the method of FIGS. 5A to 5C using a sensor of an electronic device attached to a door of a structure such as a building or a machine, or a part of a structure such as a building or a machine. Generates a neural fingerprint or neuro-mechanical fingerprint (581) and obtains an operation signal from a part of the user's body from a sensor of an electronic device attached to a door or part of a structure such as a building or machine ( 582), the signal obtained from the user is differentiated between the signal for the neurodynamic fingerprint and the user's intentional gesture movement (583), and the match rate of the neuromechanical fingerprint and whether the intentional gesture operation is a preset operation is performed. In judgment 584, if the intentional gesture action is a preset action and the match rate of the neurodynamic fingerprint has a value higher than the access level, the user is identified as an authenticated user and the door is opened or locked according to the gesture action. You can perform (580). At this time, if the preset action is a locking gesture, locking can be performed, and if the preset action is an opening action, opening can be performed. If the match rate of the neurodynamic fingerprint is lower than the access level, access may not be granted.

Referring to FIG. 5F, a neural fingerprint or neuro-mechanical fingerprint is generated 586 from a part of the user's body using the sensor of the electronic device by the method of FIGS. 5A to 5C, and the neuro-mechanical fingerprint is generated (586). Generate an encryption key using a combination of all or part of the value (587), perform encryption on the data of the electronic device using the encryption key and store it in a storage device (588), and provide access to the encrypted data. Access 589 can be performed using the value of the authenticated user's neurodynamic fingerprint.

Figure 8 is a flow chart illustrating a method for generating a neural eye identifier according to an embodiment. To facilitate understanding, FIG. 8 will be described assuming the electronic device shown in FIG. 3.

As shown in FIG. 8, the electronic device 300 acquires an image of the user's eye movement using an optical sensor (e.g., RGB sensor 376, iris sensor 383, camera 350, etc.) and The motion signal can be extracted and converted into a digital signal that can be processed through sampling and quantization. In order to collect the eye movements, signals may be collected by sampling at a predetermined sampling frequency over a predetermined time range (eg, 5, 10, 20, 30 seconds, etc.). The sampling frequency is, for example, twice the 30 Hz frequency, taking into account 3 Hz to 30 Hz, the frequency range in which the frequency of involuntary eye movements, either neurally derived or neurally based and due to the unique neuromuscular anatomy of humans, is predominantly observed. It can be 60Hz, 200Hz, 250Hz, 330Hz, etc. The mainly observed frequency range can be observed by determining a different frequency range depending on the application, and signals of other frequency ranges can also be used for accuracy or performance improvement. The optical sensor may further perform operations to remove noise or improve signal quality to increase the quality of the digital signal. When the electronic device 300 is implemented as a portable device, the issue of power consumption may become important. To this end, the electronic device can operate in a sleep mode, and the electronic device 300 can receive the collected and digitally converted eye movement signal to the processing unit 301 through the sensor core 326. The sensor core 326 may further perform operations to remove noise or improve signal quality in order to increase the quality of the digital signal received from the optical sensor. The processing unit may frequently enter a sleep mode to increase the efficiency of power consumption of the electronic device 300. In general, in sleep mode, various methods can be applied, such as cutting off the power of some components in the electronic device 300, switching to a low power mode, or lowering the frequency of the operating clock to minimize power consumption. When the processing unit 301 enters the sleep mode, the efficiency of power consumption can be said to increase, but since there may be a delay in the mutual response between the user and the electronic device, an auxiliary processor such as the sensor core 326 It may be provided inside the processing unit or in the electronic device. Even if the processing unit 301 enters the sleep mode, the sensor core 326 observes signal detection from the sensors 370 and determines that continuous processing by the processing unit 301 is required. An operation may be performed to wake up the processing unit 301 from the sleep mode through a bus or an interrupt signal. Identification, authentication, encryption, or functions utilizing neural fingerprints (NFPs) or neural eye fingerprints (NEPs) can be considered secure operations. In this case, the operation of controlling data collection of the optical sensor may be performed by, for example, switching the first core 304 in the processor unit 301 to the security mode 308. A signal transmitted through a bus or interrupt by the motion sensor or the sensor core 326 can be input to the monitor mode 313 to switch the first core 304 to the security mode 308. The core that has entered the security mode 308 can access or control system resources of the electronic device that are accessible only in the preset security mode.

The electronic device 300 may extract involuntary eye movement data from the eye movement signal collected from the optical sensor (820). The operation of extracting signals from eye movement signals is all processed by the central processing unit 302 of the processing unit 301, or by GPU 320, digital signal processing (DSP) 321, neural fingerprint (NFP) accelerator, etc. Signals can be extracted collaboratively. To extract unconscious eye movement data from the collected eye movement signals, filtering to remove unnecessary signals can be performed (822). Unwanted signals may include, for example, noise, conscious eye movement signals, macro motion signals, distortion due to gravity, etc. In signals obtained from optical sensors, the signal for a certain period of time (e.g., up to about 1 to 2 seconds) at the beginning or end of signal collection may contain a large amount of signals of the user's conscious eye movement or macro motion. It can be discarded altogether as there may be shaking of the electronic device. Since power noise may occur in signals collected when an electronic device is charging, the signal can be filtered taking into account the characteristics caused by power noise. The frequency of involuntary eye movements, which are either neurally derived or neurally based and due to the unique neuromuscular anatomy of humans, is mainly observed in the range of 3 Hz to 30 Hz. Therefore, signals in the range of 3Hz to 30Hz or 4Hz to 30Hz can be extracted from the collected eye movement signals by performing a signal processing algorithm. Depending on the characteristics of the unwanted signal to be removed, the cutoff frequency of the band filter of the signal processing algorithm can be changed. For example, in one embodiment, signals in the range of 4Hz to 30Hz may be extracted, and in another embodiment, signals in the range of 8Hz to 30Hz may be extracted. In another embodiment, signals in the range of 4Hz to 12Hz or 8Hz to 12Hz may be extracted.

In one embodiment, a signal analysis method that separates and classifies small signal amplitudes of unconscious eye movements may be used to remove conscious eye movement signals from the collected eye movement signals. For example, the interpretation method in "Time Series Classification Using Gaussian Mixture Models of Reconstructed Phase Spaces" (by Richard J. Povinelli et al., IEEE Transactions on Knowledge and Data Engineering, Vol. 16, No. 6, June 2004); Or, for separation of signals due to voluntary movement, see "Estimation of Physiological Tremor from Accelerometers for Real-Time Applications" (by Kalyana C. Veluvolu et al., Sensors 2011, vol. 11, pages 3020-3036, attached hereto in the appendix )'s interpretation method can be used to remove conscious eye movement signals and extract unconscious eye movement signals.

The electronic device may extract unique characteristics from the extracted involuntary eye movement data (830).

The electronic device may generate a neural eye identifier from the involuntary eye movement data (840). In one embodiment, a set of result values from the operation of extracting (830) unique characteristics from the extracted involuntary eye movement data is generated as a neural eye identifier in the form of a data structure with multidimensional values and stored in non-volatile memory. do.

In one embodiment, when performing the operations of the neural eye identifier generation method 810, 820, 830, and 840, the electronic device 300 uses a cluster of high-performance processor cores (e.g., the first cluster 303). If is a cluster of high-performance cores, the first cluster 303 may be assigned to ) and performed.

In one embodiment, when performing the operations of the neural eye identifier generation method (810, 820, 830, 840), the operations are performed in a security mode of a core in the processor unit 301 (e.g., the operations are performed in the first core ( When assigned to 304), it can be switched to the security mode (308) and operated. The core that has entered the security mode 308 can access or control system resources of the electronic device that are accessible only in the preset security mode and perform the operations of the neural eye identifier generation method (810, 820, 830, 840). .

In one embodiment, when performing the operation of the neural eye identifier generation method (810, 820, 830, 840), when performing the operation, when the electronic device 300 is in a sleep mode, the sensor core At 326, a signal from the sensors 370 is detected and the processing unit 301 is woken up from the sleep mode through a bus or an interrupt signal, and the neural eye identifier generation method (810, 820, 830, 840) operations can be performed.

In one embodiment, the method of generating a neural eye identifier is generated in the same way as the method of generating a neural fingerprint or neuro-mechanical fingerprint using the sensing portion of the method of FIGS. 5A to 5C using an optical sensor or camera. You can.

Figure 9 shows a cross-sectional view of the eyeball located within the human skull.

9, a cross-sectional view of a human eyeball 900 within a skull 902 is shown. The human eye 900 is an imperfect glob that can be moved within the skull 902 by a plurality of muscles. The act or process of changing the position of a person's eye 900 within the skull 902 is referred to as eye movement or eye motion.

The eye 900 has a retina 910, a pupil 912, an iris 914, and a fovea 916 that interact to acquire a color image for processing by the brain. ) and a lens 918. The cornea 922 of the eye directs images to the retina 910 through the pupil 912, iris 914, and lens 918. Pupil 912 changes its diameter to adjust the amount of light received by retina 910. The eye's retina 910 includes two types of photoreceptors: rod cells and cone cells. There are about 120 million cone cells and 6 to 7 million rod cells in the retina. Rod cells are concentrated in the rod-free area of the retina, called the central retina or macula (center of the retina), providing maximum vision and color. Cone cells are smaller and more densely packed than elsewhere in the retina 910. Nerve 926 is a cable of nerve fibers coupled to the eye and transmits electrical signals from the rods and cones of the retina to the brain. There are no rods or cones at the point where the optic nerve leaves the eye through the retina. Thus, the optic nerve forms a “blind spot” in the retina.

10A-10C are diagrams showing various muscles connected to the eye that cause various eye movements.

In Figure 10A, the left rectus muscle 1001L and the right rectus muscle 1001R move the eye 900 horizontally from side to side as shown by arrows 1011.

In FIG. 10B, the superior rectus 1001T on the upper line and the inferior rectus 1001B on the lower line move eye 9100 upward and downward as shown by arrow 212.

In Figure 10C, the superior oblique muscle 1002S and the inferior oblique muscle 1002I cause eye 900 to roll and move as shown by curved arrow 1013. These muscles can cause voluntary eye movements according to the human will, and involuntary eye movements that the human is not even aware of can occur. Other muscles around the eye 900 can also contribute to voluntary and involuntary eye movements. Muscles are under the control of the body's nervous system, including the brain.

Involuntary eye movements, unlike voluntary eye movements, are often considered pathological when observed by an ophthalmologist. However, it is more often observed that normal, physiological, subtle involuntary movements occur when the eyes are fixed on an object. These tiny, involuntary eye movements are a normal physiological function of the body to prevent fatigue of rods and cones at focal points on the retinal surface within the eye. Involuntary eye movements have generally been considered disturbances or analyzed for research purposes.

The retina 910 of the human eye 900 may be scanned for a variety of purposes. Retinal scanners are known that obtain two-dimensional maps of the retinal anatomy of the eye. Retinal scanners are not intended to measure involuntary eye movements.

Various eye tracking systems have been used to detect conscious eye movements for various purposes. For example, a gaming virtual reality headset can track conscious eye movements during gameplay of a video game. As another example, heads-up displays in military systems can track conscious eye movements for military purposes. However, the eye tracking system was designed to evaluate the foveal field of view and the subject's focal attention, and involuntary eye movements were either disruptive or were used only for research purposes, so involuntary eye movements were not measured.

Retinal sensitivity to light (photons) is defined by genetic factors, including eye movements, eye muscles, and the integration of the entire system by specific brain wiring. Very small involuntary eye movements of the eyeball 900 are a normal physiological function of the human body. Any neurologically coordinated process will create characteristics that are unique to an individual and thus potentially useful in identifying one individual from another. Accordingly, involuntary eye movements of eye 900 are unique to an individual and can be used for user identification and user authentication.

Figure 11 is a diagram of a fixation processor for acquiring involuntary eye movements of the eye. To facilitate understanding, the description will be made along with a cross-sectional view of the eye shown in FIG. 9.

Referring to FIG. 11, small or subtle involuntary eye movements of the eyeball 900 can be obtained through the process of fixing the eyes on an image of an object. The object image may be a retinal image 1102 on the retina 910 of the eye 900. The object image may be generated by the object 1104 on the display device 1110. The user gazes or fixates on target 1104 on display device 1110 for a period of time (eg, 10 to 15 seconds). Involuntary movements are too small and difficult to capture when observed directly with the naked eye. Video camera 1112 acquires a series of images of the movement of eye 900 during the fixation process.

To prevent acquiring conscious eye movements during the fixation process, a camera 1112 can be connected to the eye 900. However, this is impractical for authentication purposes. Conscious eye movements can be substantially filtered out from the captured eye movement data to generate unconscious eye movement data.

Additionally, although fine involuntary eye movements (also referred to in this disclosure as involuntary micro-motions of the eye) are generally acquired during the fixation process, large-scale conscious eye movements are also achieved when the eyes move to see. Regardless of the focal point of the part of interest, it can be obtained. Video cameras can be used to acquire image sequences of small-scale or large-scale eye movements, including subtle involuntary eye movements. Subtle involuntary eye movements can be extracted from acquired image sequences, including both involuntary and conscious movements.

Figures 12A and 12B are diagrams of an enlarged portion of the retina with graphs of involuntary eye movements over the area of the cone cells in response to the user fixating a target for a period of time. Figure 15 is a block diagram of an electrooculography system for directly generating a user's involuntary eye movement signal. FIG. 16B is a block diagram of an electronic device equipped with the video camera shown in FIG. 16A that can be used to obtain an image of a user's eye movements. The electronic device 1600 of FIG. 16B may be the electronic device 200 of FIG. 2 or the electronic device 300 of FIG. 3, and may be a component of the electronic device 200 of FIG. 2 and the electronic device 300 of FIG. 3. It may include a combination of some or all of them, or may include components not shown in the present disclosure. To facilitate understanding, the electronic device 1600 will be described together with FIGS. 12A and 12B and the system of FIG. 15 using the same.

In Figure 12A, cone-shaped area 1200 is shown as an enlarged portion of the central fovea of retina 910. Figure 12a also shows a graph of involuntary eye movements in the cone-shaped area corresponding to a user's fixation on an object for approximately 10 seconds. FIG. 12A shows a plurality of cones within about 5 microns in diameter of retina 910.

Involuntary eye movements exist regardless of whether the user fixates on a stationary object. Subtle involuntary eye movements, including various types of eye movements, can be used to identify users.

12A shows small involuntary eye movements, including saccades (also known as fine saccades) (1201A-1201F), curved drifts (1203A-1202E), and zigzag tremors (1203A-1203E). Shows various types.

Figure 12B shows an enlarged view of a saccade 1201F, a curved drift 1202E, and a zigzag shaped tremor 1203E. The zigzag-shaped tremor 1203E was superimposed on the curve-shaped drift 1202E.

Drift eye movements (1202A-1202E) are like random traces without any precise target when the eyes are not focused on anything in particular. These eye movements change direction with a frequency of about 20 to 40 Hz and are characterized by small amplitude. Trembling eye movements are characterized by very small amplitudes (for example, an angle of 0.2 to 2-3 degrees) and high frequencies (for example, from 40 Hz to 150 Hz, 90 Hz is a typical value). The saccadic eye movements are characterized by amplitudes between angles of 15-20 degrees, rapid movements (200-500 degrees per second), and relatively low frequencies (e.g., 0.1 Hz to 1-5 Hz).

Subtle involuntary eye movements are a normal physiological function of the body to prevent fatigue of the rods and cones in focus on the retinal surface within the eye. The image of an object or object on the retina of the eye constantly moves due to involuntary eye movements. Drift eye movements (1202A-1202E) cause the image to slowly drift outward from the central center. Drift eye movements end at the beginning of saccadic eye movements. Saccadic eye movements (1201A-1201F) bring the image back toward the fovea. Tremor eye movements 1203A-1203E superimposed on drift eye movements 1202A-1202E have amplitudes that span multiple cones to prevent fatigue of a single cone when gazing or fixating an object.

These small, involuntary eye movements are thought to prevent retinal fatigue. Retinal fatigue is the depletion of several important biochemical substances essential for photon acquisition by retinal cells. Small involuntary eye movements cannot be detected by the human eye. However, as shown in FIGS. 11 and 16B, involuntary eye movements can be acquired and detected/watched/recognized using a retina scanner and a video camera with a speed sufficient to record eye movements.

Involuntary eye motions or movements can be measured by various means or methods, such as using infrared light-emitting diodes (LEDs), pupil scanning methods, or even refractometers, with appropriate time or frequency and displacement resolution to acquire involuntary eye motions or movements. By modifying it, it can be detected.

Involuntary eye movements can be selectively detected using an electroculogram (EOG) with sufficient sensitivity to generate an electroculogram (EOG) representing involuntary eye movements. The raw EOG signal originates from the dipole between the eye's cornea and retina.

Electrical signals occur in the oculomotor muscles when the eyes cause eye movements. These electrical signals generated from the oculomotor muscles can be detected as representing eye movements, similar to the electromyography (EMG) of the eye.

Referring to Figure 15, safety recording involves detecting electrical signals by measuring the potential difference (voltage) between the retina (ground) and the cornea of the eye. Typically, electrodes are placed around the eye to measure voltage up, down, left, and right relative to one or more ground electrodes. The difference between left and right voltage measurements can be made to generate a signal representing horizontal eye movements. The difference between upward and downward voltage measurements can be made to generate a signal indicative of vertical eye movement. Signals can be amplified and filtered before signal processing occurs. The analog signal is converted to a digital signal so that digital signal processing can analyze the EOG signal for patterns in the user's involuntary eye movements.

Involuntary eye movement patterns are used as a way to perform physiological biometrics of an individual because they are associated with that individual's specific neuro-muscular-retinal anatomy. Obtaining microscopic movements of the eye's involuntary eye movements can extract patterns that are unique to an individual and can be used to authenticate an individual's identity. By analyzing the extracted patterns, features that provide a unique identifier for each individual can be extracted. In one embodiment, pupils can be identified and tracked using high-speed, high-resolution cameras. The images can be processed by a processor running image processing software to extract features from involuntary eye movements that uniquely identify the user.

Involuntary eye movements can be used as a standalone authentication method or in conjunction with a retinal scanner to acquire images of the eye's retinal anatomy. Involuntary eye movements can also be used as an authentication method in conjunction with an iris scanner that acquires images of the eye's iris anatomy. Involuntary eye movements can also be used as an authentication method in conjunction with retina and/or iris scanners that take images of the eye anatomy.

Involuntary eye movements and the retina and/or iris can be used to provide two- or three-factor authentication. Retinal and iris scanning techniques do not involve a neurological component. Retinal scanning technology simply maps the anatomy of the retina to perform authentication based on two-dimensional (2D) scanned images. Involuntary eye movements can add biological parameters to current eye scanning techniques.

US Patent Publication No. 2014/0331315, filed by Birk et al., discloses a method for deriving biometric parameters from eye movements for use in an authentication protocol. Combining conventional authentication methods, such as eye movements based on a repertoire of actions that actively focus the eyes, such as "a password or other information known to the user" and "visually grasping a piece of information embedded in the display," as described in the abstract, there is. Birk's behavioral repertoire may be a path of eye movements or a characteristic of the user's eye movements that sequentially places a series of numbers/letters that match a password or code the user knows.

Birk recognizes that there are two types of eye movements: conscious and unconscious. However, Birk describes using unique, active, conscious eye movements as input to recognition techniques. Birk essentially uses a defined grid, which the user's eyes must move around in a specific order to be recognized. Birk taps into the user's repertoire of specific behaviors (habits).

In contrast to Birk, embodiments of the present disclosure utilize only normal physiological involuntary eye movements controlled by the brainstem and cerebral cortex. The involuntary eye movements achieved by embodiments are not based on what the user does, but rather on what the user is.

U.S. Patent No. 6,785,406, issued August 31, 2004 to Mikio Kamada (hereinafter referred to as Kamada), collects data from users using eye movements and iris contraction, rather than images or other body parts. It describes an iris authentication device that makes it possible. The eye movements used in Kamada utilize movements associated with the vestibular control of the eyes to ensure eye-head coordinates when looking at an object while the head is moving. Kamada also uses the optokinetic nystagmus eye movement, which is a large saccadic movement in which an object moves out of the field of view and brings the center of the retina back to the center.

The eye movements described by Kamada are not related to retinal fatigue and user-specific fine movements. The eye movements described by Kamada are reflex movements. Kamada does not use eye movements to identify users and verifies whether the data obtained is from real users. The eye movements described by Kamada are different for each user and are reflexive, so they cannot be used to identify the user, so they are like the knee reflex and appear in everyone like the knee reflex. In Kamada, different degrees of detachment may be recognized, but not sufficiently to distinguish individuals to provide identity.

U.S. Patent No. 8,899,748, issued to Brandon Lousi Migdal on December 12, 2014, describes a method for detecting ocular nystagmus movements associated with vestibular control (equilibrium) in an individual. However, vestibular nystagmus eye movements can be vertical or horizontal eye movements and are initiated by a common position reflex. Moreover, vestibular nystagmus eye movements lack the fine granularity of involuntary eye movements useful for differentiating individuals according to embodiments of the present disclosure.

U.S. Patent No. 9,195,890 (Bergen), issued November 24, 2015 to James R. Bergen, discloses a biometric identification method based on iris image comparison. Bergen's comparison is based on unique anatomical features extracted from the iris of the eye. The characteristics of the iris are described in various detail by Bergen. To align and match the aperture to the stored iris pattern, Bergen must compensate for tilt/position and motion, as well as correcting other image distortions. The eye movements mentioned in Bergen are large and undesirable because they interfere with the collection of quality data in Bergen's iris recognition system. This can be said to be contrary to the embodiment of the present disclosure.

US Patent No. 7,336,806, issued to Schonberg et al. (hereinafter Schonberg), discloses iris-based recognition of a user where ring circumference is the key. Schonberg considers other features, including eye movements, as noise that should be removed.

U.S. Patent No. 7,665,845, issued to Kiderman et al. (Kiderman), describes a video occulo graphics (VOG) system based on lightweight goggles. Kiderman's goggles were designed so that the weight of the measuring device would not interfere with clinical measurements. However, Kiderman does not disclose the use of involuntary eye movements, which is the subject of interest in embodiments of the present disclosure. Moreover, there is no explicit need for spatial resolution with respect to embodiments of the present disclosure.

Electrooculography can directly generate involuntary eye movement signals. When tracking eye movements using acquired video images, visible features of the eye in the video image are used. Pattern recognition can be used to detect visible features in each image of a video image.

14A-14C are diagrams of features visible in video images of the eye that can be used to detect involuntary eye movements.

Referring to Figure 14A, the visible features of the image of the eye may be the iris 914, the pupil 912, or the fovea 916. Edge detection can be used to track involuntary eye movements from video images of the eye 900 that can be obtained while the user fixates an object. Edge detection may be performed from the location of the perimeter 614 of the iris 1414, the perimeter 1412 of the pupil 912, or the fovea 916.

As shown in Figure 14B, the perimeter 1414 of the iris 914 is well defined and consistent, making it useful for tracking involuntary eye movements. The perimeter 1412 of the pupil 912 may, optionally, be used to track involuntary eye movements. However, the size and position of the pupil change depending on the light intensity.

As shown in Figure 14C, fovea 916 can optionally be used to track involuntary eye movements. However, tracking the fovea 916 typically requires a light source shining through the lens 918 to illuminate the retina 910 of the eye 900.

13A-13C are diagrams illustrating various combinations of involuntary eye movements that may be used for user identification and authentication. Figure 15 is a block diagram of an electrooculography system for directly generating a user's involuntary eye movement signal. To facilitate understanding, FIGS. 13A to 13C and the system of FIG. 15 using them will be described together.

In Figure 13A, three types of involuntary eye movements, including saccadic trajectories, drifts, and tremors, can be used to determine each user's unique identifier. In one embodiment, features are extracted from all three involuntary eye movements to uniquely identify each user. The system extracts the same features over and over again for each user.

In Figure 13B, two involuntary eye movements, such as saccade trajectory and drift, can be used to determine each user's unique identifier. In another embodiment, features are extracted from saccade trajectories and drifts to uniquely identify each user. The system extracts the same features over and over again for each user.

In Figure 13C, a single involuntary eye movement, such as a tremor component, can be used to determine each user's unique identifier. In another embodiment, features are extracted from the tremor component of eye movements to uniquely identify each user. The system extracts the same features over and over again for each user.

Referring to FIG. 15 , an example of a safety recording system 1500 that directly generates an unconscious eye movement signal of a user 1599 is shown. A plurality of electrodes 1501A-1501E are located in the eye area around the user's head/face to obtain the voltage of each eye during eye movement. The electrodes may be part of a hood or facial receiving device to connect the electrodes to the surface of the user's head/face. Electrodes 1501A-1501B acquire the up and down movement of the eyeball 900. Electrodes 1501C-1501D acquire left and right movements of the eye 900. One or more electrodes 1501E provide a ground or zero voltage reference to each electrode 1501 - 1501D.

Electrodes 1501A-1501D measure involuntary eye movements occurring during the fixation process up and down, left and right, and voltage relative to one or more ground electrodes 1501E.

The upper and lower voltages of electrodes 1501A-1501B with or without filtering are connected to the negative and positive inputs of differential amplifier 1504A to amplify and calculate the difference between the upper and lower voltages. This forms a vertical eye movement signal. The vertical eye movement signal may be coupled to an analog filter 1506A to further enhance the vertical eye movement signal by removing noise and other unwanted signals. The filtered vertical eye movement signal, i.e., analog signal, may be connected to an A/D converter 1508A to convert the analog form to a digital form signal. The digitally filtered vertical eye movement signal may be coupled to a first parallel digital input of digital signal processor 1510. In another embodiment, a signal processor may be manufactured to support mixed analog and digital signals. Accordingly, a signal processor 1510 may be included.

Similarly, the left and right voltages of the filtered electrodes 1501C-1501D, with or without filtering, are connected to the negative and positive inputs of differential amplifier 1504B to amplify and calculate the difference between the left and right voltages. This forms a horizontal eye movement signal. The horizontal eye movement signal may be coupled to an analog filter 1506B to further enhance the horizontal eye movement signal by removing noise and other unwanted signals. The filtered horizontal eye movement signal, i.e., analog signal, is connected to an A/D converter 1508B to convert the analog form to a digital form signal. The digitally filtered horizontal eye movement signal may be coupled to a second parallel digital input of digital signal processor 1510.

Digital signal processor 1510 performs digital signal processing using both the digitally filtered horizontal eye movement signal and the digitally filtered vertical eye movement signal to analyze, extract features, and uniquely identify the user's involuntary eye movements. Create a pattern. In this case, unique identifying patterns of involuntary eye movements are obtained directly from the user's head/face without using a video camera or scanner and analyzing the images.

Figure 16A is a diagram of an electronic device with a video camera that can be used to obtain images of eye movements of a user. The electronic device 1600 of FIG. 16B may be the electronic device 200 of FIG. 2 or the electronic device 300 of FIG. 3, and may be a component of the electronic device 200 of FIG. 2 and the electronic device 300 of FIG. 3. It may include a combination of some or all of them, or may include components not shown in the present disclosure. FIG. 16B is a block diagram of an electronic device 1600 equipped with the video camera shown in FIG. 16A that can be used to obtain an image of a user's eye movements.

16A and 16B, electronic device 1600 with a video camera is used to obtain images of eye movements of user 1699 during a fixation process.

In FIG. 16B, the electronic device 1600 includes a processor, microcomputer, or display device 1607 connected to a microprocessor 1601 and a video camera 1612. The electronic device 1600 further includes a memory 1602 that stores program commands and user-related data. Electronic device 1600 includes one or more (radio frequency transmitter/receiver) radios 1609 and one or more wired connectors 1622, 1624 to provide communication between electronic device 1600 and other devices (radio frequency transmitter/receiver). ) (For example, it includes a USB port 1622 and/or a network interface port. The electronic device 1600 is a touch screen display device ( 1607). Software control may be displayed on the touch screen display device 1607 to allow the user to control the electronic device. The electronic device 1600 may be configured to allow the user to further control the electronic device. One or more hardware buttons 1604 may optionally be included.

To acquire eye movements, processor 1601 generates a fixation target 1650 that is displayed by display device 1607. The user may be asked to fixate their gaze on a target 1650 while a video camera 1612, under the control of processor 1601, acquires a time-series sequence of images, video, and the user's eyes. A video camera captures the user's involuntary eye movements of one or both eyes on a frame-by-frame basis. 3D accelerometer data is acquired along with the video to determine eye movement by eliminating the physical movement of the camera 1612 and electronics 1600.

Processor 1601 may provide or include signal processor functionality. In either case, the processor executes instructions in pattern recognition software and signal processing software to analyze the video obtained for involuntary eye movements of one or both eyes of the user. Pattern recognition software can be used to identify the eye and iris and pupil in the video.

The acquired video is analyzed for whether the user blinked in a time-series sequence of images and whether sufficient sequence is captured to detect eye movements. Otherwise, the user is prompted by the user interface to repeat the fixation process.

Once sufficient image sequences are acquired, further analysis is performed to determine eye movements from the images. Reference points of the eye, such as the iris or pupil, are used to determine involuntary movements in video acquired during fixation or gaze. The 3D accelerometer data is used to subtract the physical movement of the video acquired camera 16812 from the raw eye movement data to form realistic eye movement data.

The eye movement data is further analyzed to extract the types of involuntary eye movement data used to authenticate and/or uniquely identify the user.

The user is initialized with electronic device 1600 to store an initial data set of initially acquired involuntary eye movement data. From the initial data set, features linked to the time series of involuntary eye movements are extracted and classified in relation to the user.

Features extracted from the acquired data set of newly acquired involuntary eye movement data are compared with features extracted and stored from the initially acquired involuntary movement data to identify the user. A match rate can be calculated to determine whether the user is authorized to use the electronic device.

Newly acquired involuntary eye movement data is compared to initially acquired involuntary movement data to determine a consensus result. If the match is within the match percentage, the user is identified and authorized to use the device. If the match result is outside the match percentage, the user will not be informed and will not have permission to use the device.

Involuntary eye movements (e.g., tremors) can occur with a maximum frequency of approximately 150 hertz (Hz). The cone shape has a diameter of 0.5 to 4 micrometers.

To capture involuntary eye movements, a suitable recording device of the system, such as a video camera 1612 connected to processor 1601 of electronics 1600, may have a minimum frame rate (frames per second) of at least 300 Hz-450 Hz. range or optimally operates at a frame rate of 1000Hz or higher.

Cameras and processors in existing electronic devices can be reconfigured by software applications or drivers to run faster than a typical setup. Normal (large-scale saccades) guarantees about 300 degrees per second and allows the eyes to realign within 1/3 of a second. Involuntary microsaccades range in amplitude from a few degrees per second to less than 0.2 degrees. Accordingly, the spatial resolution of the recording device, which can be resolved within the range of 0.1-0.2 degrees, may be optimal for obtaining the desired involuntary eye movements.

Although one type of electronic device 1600 is shown in FIGS. 8A and 8B to obtain the desired involuntary eye movements, other types of electronics may be used to capture involuntary eye movements.

17A-17C show diagrams and targets of an electronic glasses with a video camera that can be used to acquire images of user eye movements.

17A and 17B, electronic glasses 1700 that can be used to obtain images of eye movements of a user are shown. From acquired images of eye movements, small-scale involuntary eye movements can be extracted. Electronic glasses 1700 may be temporarily worn by a user to identify, authenticate, and authenticate the user to a system or device.

The electronic glasses 1700 include an eye glass frame 1702 with a left lens 1704L and a right lens 1704R mounted on a pair of eye wires. In one embodiment, eyeglass frame 1702 includes a bridge 1707, a left section 1708L, a right section 1708R, a nose pad, a nose pad arm, and a pair of eye wires. These electronic glasses 1700 may be glasses clips that can be worn with prescription glasses. Lenses (1704L-1704R) may or may not be prescription lenses.

A small target 1706 may be formed in the upper right corner of the left lens 1704L to point the user's eye toward the video camera. Optionally, a target 1706 can be formed in the upper left corner of right lens 1704R to direct the user's eye toward the video camera. Target 1706 can be formed by shining target light onto the surface of the lens, by printing the target image on the surface of the lens, by etching the target image into the lens. or by other known means of applying the image onto a transparent surface.

Referring to Figure 17C, target 1706 is a short depth target with a central opening or hole 1707. Target 1706 on lens 1704L or 1704R may be too close for the user to focus on. However, the user can focus through the central aperture 907 of target 1706. Target 1706 with a central aperture 1707 acts like an optical tether so that the pupil is in line with the video camera to better capture eye movements.

Referring to FIG. 17B, the electronic glasses 1700 include a video camera 1710, a processor 1712, a memory device 1714, a wireless transmitter/receiver 1716, and a power supply (for example, a power supply) attached to the glasses frame 1702. For example, it may include a battery) (1720). Electronic glasses 1700 may further include an optional light emitting diode (LED) 1718 mounted on frame 1702 or nose pad arm 1722.

The video camera 1710 has a lens with a target, for example, a left lens 1704L with a target 1706, slightly inclined in this direction. Therefore, you can be more aligned with your eyes when focusing on a target. Video camera 1710 and processor 1712 operate together at frame rates ranging from 300 to 1000 frames per second to capture involuntary eye movements at frequencies up to 150 Hz.

Radio 1716 may be used by the processor to communicate with a computer or server to authenticate a user to the computer or server. Memory 1714 may be used to store initialization data containing characteristics of initial involuntary eye movements extracted from acquired eye movements.

The electronic glasses 1700 can be temporarily worn by the user to authenticate the user to the system. Optionally, electronic goggles can be used.

18A and 18B are diagrams of a Virtual Reality (VR) device with a video camera that can be used to obtain images of a user's eye movements.

18A and 18B, an electronic virtual reality headset or goggles 1800 is shown that includes a video camera that can be used to capture eye movements of a user's eyes. The electronic virtual reality headset includes a frame 1802 and a head strap 1803 to maintain the headset attached to the user's head. Frame 1802 includes upper, lower, left, and right flexible curtains or blinders 1804 configured to accommodate the face around the user's eyes to provide a hooded area 1899 to block external light. Lower curtain or blinder 1804B includes a nose opening 1805 to accommodate the user's nose.

In Figure 18B, headset 1800 further includes a video camera 1810 coupled to frame 1802, a left display device 1812L, and a right display device 1812R. The headset 1800 further includes a processor 1811 and a memory 1814 connected together. Video camera 1810 and left and right display devices 1812L and 1812R are connected to processor 1811. The left display device 1812L and the right display device 1812R can provide stereo three-dimensional images perceived at various depths to the user. Video cameras 1810 may be coupled to frames 1802 within hood area 1899 at different locations to capture eye movements. Video camera 1810 may be positioned on the right side as shown to capture eye movements to avoid interference with video images displayed by left and right display devices 1812L, 1812R. In another embodiment, a pair of video cameras 1810 may be positioned on opposite sides to capture left and right eye movements such that involuntary eye micro-movements from one or both eyes can be used to authenticate the user.

Stereo three-dimensional targets including left target 1806L and right target 1806R may be generated by processor 1811 and displayed on left display device 1812L and right display device 1812R, respectively. Left target 1806L and right target 1806R can make the target look distant to focus the eyes in the distance and can fix the eyes to better capture involuntary eye movements with the video camera 1810.

Processor 1811 can be wired to other systems by cable 1850 and plug 1852. Optionally, a wireless transmitter/receiver 1816 may be coupled to the processor 1811 such that it can be wirelessly connected to another system to utilize the authentication capabilities of the processor and headset/goggles headset/goggles 1800.

Figure 19A is a diagram of an eye containing a contact lens with an emitter device that helps obtain data regarding the user's eye movements. 19B-19D are diagrams of contact lenses with various emitter devices that can be used to help obtain data regarding a user's eye movements. Figure 19E is a functional block diagram of an active emitter device. Figure 19f is a side view of a sensing device near an eye-mounted contact lens that can be used to obtain data regarding a user's eye movements.

Referring to FIG. 19F , eye movements can be obtained in another manner by using a contact lens 1900 mounted on one or both eyes 900 of the user over lens 918. Contact lens 1900 may include one or more emitters 1902 connected (e.g., embedded or printed) with lens material 1104.

Emitter 1902 may be an active device, such as a light emitting diode (LED) or a wireless beacon associated with a driving circuit (e.g., radio transmitter/receiver, diode driver), or a reflector, retro reflector. (retro-reflector), or may be a passive device such as a mirror.

In the case of active emitter devices, one or more sensors are used that directly obtain eye movements over a period of time by receiving emitted light or wireless signals to determine the eye's position from one viewpoint to the next. Power can be wirelessly routed from a base antenna around the eye to an antenna connected to an active emitter device in the contact lens. A wireless signal may also be coupled between the base antenna and the antenna connected to the active integrated circuit device. A three-dimensional motion sensor may be included as part of the integrated circuit to capture eye movements, including involuntary eye micro-movements.

For passive emitter devices, light from a light source is directed to the passive emitter device, which activates it to reflect the light back to one or more photodiode sensors around the eye. A combination of active and passive emitter devices can be used in the same contact lens to capture eye movements in either or both methods.

Figures 19B-19D show one emitter (1902A), two emitters (1902B-11\902C) and four emitters (1902D-1902G) embedded in a contact lens (1900A-1900C), respectively.

In FIG. 19B, active emitter 1902A is shown connected to two or more antenna lines 1905A-1905B at the peripheral edge of contact lens 1900A. Active emitter 1902A includes an integrated circuit 1906 having a processor/controller and other circuitry coupled externally thereto or integrated internally on the integrated circuit. Optionally, emitter 1902A may be an active emitter.

In Figure 19C, a pair of passive emitters 1902B-1902C are shown disposed around a portion of the peripheral edge of contact lens 1900B. Optionally, emitters 1902B-1902C may be active emitters with two or more antenna feeds 1905A-1905B in a portion near the peripheral edge of contact lens 1900B.

19D, a pair of active emitters 1902D-1902E and a pair of passive emitters 1902F-1902G are shown near the peripheral edge of contact lens 1900C. Optionally, all emitters (1902D-1902G) can be active or passive emitters; Only the other can be passive. The other can be activated passively.

Referring to Figure 19E, additional details of an example active emitter 1902 are shown. Active emitter 1902A includes an integrated circuit 1906 having a processor/controller 1910 and other circuitry coupled externally thereto or integrated internally on the integrated circuit coupled to the processor/controller 1910.

Integrated circuit 1906 may receive power to power conversion circuit 1912 coupled to processor 1910 via antennas 1905A-1905B. Radio frequency energy from the oscillating radio frequency (RF) signal generates more antenna feeds 1905A-1905B by a nearby base antenna. Power conversion circuit 1912 may rectify and regulate AC RF signals into DC power for other circuits within IC 1906 as well as circuits connected to IC 1906.

With a light emitting diode (LED) 1908 connected to the integrated circuit 1906, an internal diode driver 1918 is coupled between the processor 1910 and the light emitting diode (LED) 1908. The power conversion processor may generate a signal to activate the diode driver 1918. The processor uses power to activate diode driver 19118 to power light emitting diode 1908 to emit light from the user's eyes.

Alternatively or additionally, integrated circuit 1906 may have a 3D motion sensor 1917 coupled to the processor that directly senses eye movements. With power, a processor coupled to wireless transmitter/receiver (transceiver) 1909 may transmit and receive wireless signals to wireless transceiver 1919 via antenna lines 1905A-1195B. A basic unit with a wireless transceiver can collect and further process eye movement data.

19F, for an active emitter, a base unit 1999 is shown comprising a frame 1921, a lens 1922, and the poles of base antennas 1924A-1924B wrapped around the lens 1922. do. Base antennas 1924A-1924B are connected to a base radio receiver/transmitter (not shown) and to a base processor (not shown) that processes the acquired eye movement signals.

With the base antennas 1124A-1124B near the antenna lines 1905A-1905B of the contact lens, they are directed to transmit radio frequency power/energy/radio frequency signals together between the base unit 1199 and the contact lens 1900C. Can be inductively coupled to.

When power is applied, eye movements acquired by motion sensor 1917 can be transmitted from contact lens 1900C to base unit 1999 by the respective wireless transceivers.

For a passive emitter, the base unit 1999 (additionally or alternatively) includes one or more photo diode sensors 1934A-1934B connected near the edge of the lens 1912. Base unit 1999 may further include a light source 1932, such as a display device, to direct light toward one or more passive emitters 1902F, 1902G. Light reflected from one or more passive emitters 1902F, 1902G is captured by one or more photo diode sensors 1934A-1934B to determine eye position and movement over time. Wires 1914 from photo diode sensors 1934A-1134B are connected to a base processor (not shown) to process the acquired eye movement signals.

The base unit 1999 may be in the form of glasses, VR goggles, a stand-alone eye scanner, or a wall-mounted eye scanner.

Electronic devices may be worn by the user on or near the user's eyes, as in the case of glasses, goggles, or contact lenses, but electronic devices for capturing eye movements may be worn by the user on or near the eyes of a video camera, sensor, or antenna. It can be processed by a structure or system that involves involuntary eye movements while positioning the eye.

20A to 20B show a device for acquiring eye movements attached for access control of a building structure.

FIG. 20A shows an eye movement acquisition device 2001 attached to a building structure 2000 to control access to one or more doors 2002 in response to a user's involuntary eye micro-movements. Eye movement acquisition device 2001 is coupled to Access system 2003 to control access to one or more doors 2002 of structure 2000. Access system 2003 may perform unlocking control of one or more doors 2002 in response to involuntary eye micro-movements of an authorized user.

Figure 20b shows an enlarged view of the eye movement acquisition device 2001 receiving the user's facial area around the left or right eye. As previously discussed with reference to FIG. 18B, eye movement acquisition device 2001 includes a video camera 1810, a display device 1812R, and a memory 1814 coupled to the processor, as well as some similar structures and functions of processor 1811. may include. Processor 1811 may be wired to access system 2003 by cable and plug. Optionally, processor 1811 may be wirelessly coupled to access system 2003 by a wireless device 1816 coupled to processor 1811.

Figures 21A and 21B show an independent eye scanner for authenticating a user to the system.

FIG. 21A shows a standalone eye scanner 2101 connected to system 2102 by cable connection or wirelessly with respective wireless transmitters/receivers. System 2102 may be connected to server 2106. The server may be remote and accessible via a wide area network 2104, such as the Internet. Standalone eye scanner 2101 may be used to non-invasively authenticate a user to system 2102 and server 2106 in response to eye micro-movements of one or both eyes of the user.

FIG. 21B shows an enlarged view of a stand-alone eye scanner 2101 that accommodates the eye region of a user's face. As previously discussed with reference to FIG. 18B, eye movement acquisition device 2101 includes a processor 1811 as well as one or more video cameras 1810, a left display device 1812L, a right display device 1812R, and a processor 1812 connected to the processor. Includes memory 1814. Processor 1811 may be wired to system 2102 by cable 1850 and plug 1852. Left and/or right targets 1806L, 1806R are similarly created on the left and/or right display devices 1812L, 1812R so that the stand-alone eye scanner 2101 can respond to eye micro-movements of one or both eyes of the user. Users can be authenticated non-invasively.

Claims (20)

An electronic glasses device utilizing biometric recognition technology,
at least one display;
A frame 1802 coupled to the display;
A fixing part 1803 that is coupled to the frame and can be worn on a part of the user's body;
a movement information acquisition device for capturing at least one movement information of a user including neuro-muscular tone in a part of the user's body at a preset sampling frequency;
a processor electrically connected to the movement information acquisition device; and
A storage device electrically connected to the processor and configured to store information associated with a unique characteristic value for identifying a user,
The storage device, when executed, the processor,
Obtain at least one first movement information of the user including neuromuscular tone at a preset sampling frequency during a first preset time range using the movement information acquisition device,
Remove signal information for a second preset time range from among the at least one first motion information of the user obtained, from information in the initial part of acquiring the first motion information,
From the at least one first movement information from which the signal information for a second preset time range has been removed, suppress the signal component associated with the conscious movement of the neuromuscular tone, thereby producing the first involuntary movement of the neuromuscular tone. Obtain non-voluntary movement information,
extract at least one first unique characteristic value associated with the neuromuscular tone based on the obtained first involuntary movement information of the neuromuscular tone, including a mathematical property that is measurable in advance and can identify the user; ,
Forming the at least one first unique characteristic value into data in the form of a multidimensional vector,
Select at least one prediction model based on the at least one first unique characteristic value and extract first parameters of the at least one prediction model,
Performing performance evaluation of the at least one prediction model based on unique characteristic values prepared in advance
storing commands
Electronic glasses device utilizing biometric recognition technology.
According to claim 1,
The storage device, when executed, the processor,
Additional storing instructions for generating a first neuro-mechanical fingerprint based on the first unique characteristic value
Electronic glasses device utilizing biometric recognition technology.
According to claim 1,
The processor has at least one power mode
Electronic glasses device using biometric technology
According to claim 1,
The storage device, when executed, the processor,
Additionally storing a command to store the parameters of the prediction model
Electronic glasses device utilizing biometric recognition technology.
According to claim 3,
When running, the processor:
It is additionally equipped with a command to enter low-power mode based on detection of input from the user for a preset period of time.
Electronic glasses device utilizing biometric recognition technology.
According to claim 2,
The storage device, when executed, the processor,
Obtain at least one second movement information of the user including neuromuscular tone at a preset sampling frequency during a third preset time range using the movement information acquisition device,
remove signal information during a fourth preset time range from the at least one second motion information of the user obtained;
Obtaining second involuntary movement information of neuromuscular tone by suppressing signal components associated with conscious movement of neuromuscular tone from at least one second movement information from which signal information during a fourth preset time range has been removed. Let it be done,
Based on the obtained second involuntary movement information of the neuromuscular tone, extract at least one second unique characteristic value associated with the neuromuscular tone, including a mathematical property that is pre-measureable and can identify the user. do,
extract a second parameter of a prediction model based on the at least one second unique characteristic value,
generate a second neuro-mechanical fingerprint based on the at least one second unique characteristic value,
Save additional commands
Electronic glasses device utilizing biometric recognition technology.
According to claim 6,
The storage device, when executed, the processor,
Further storing instructions for determining a matching percentage between information associated with the first neuro-mechanical fingerprint and information associated with the second neuro-mechanical fingerprint.
Electronic glasses device utilizing biometric recognition technology.
According to claim 7,
The storage device, when executed, the processor,
In response to the case where the match rate is equal to or higher than a preset access match rate, additionally storing a command to determine whether the user is a pre-authenticated user.
Electronic glasses device utilizing biometric recognition technology.
According to claim 8,
The storage device, when executed, the processor,
Additional storage of commands that allow access to the electronic device to the authenticated user.
Electronic glasses device utilizing biometric recognition technology.
According to claim 2,
The storage device, when executed, the processor,
Additional storing instructions for generating encryption data with an encryption algorithm using an encryption key based on the first neuro-mechanical fingerprint.
Electronic glasses device utilizing biometric recognition technology.
According to claim 10,
The storage device, when executed, the processor,
acquire at least one second movement information of the user including neuromuscular tone at a preset sampling frequency during a third preset time range using the movement information acquisition device;
remove signal information during a fourth preset time range from the at least one second motion information of the user obtained;
Obtaining second involuntary movement information of neuromuscular tone by suppressing signal components associated with conscious movement of neuromuscular tone from at least one second movement information from which signal information during a fourth preset time range has been removed. Let it be done,
Based on the obtained second involuntary movement information of neuromuscular tone, extract at least one second unique characteristic value associated with the neuromuscular tone and comprising a pre-measurable and user-identifiable mathematical property; ,
extract a second parameter of a prediction model based on the at least one second unique characteristic value,
generating a second neuro-mechanical fingerprint based on the second unique characteristic value,
Save additional commands
Electronic glasses device utilizing biometric recognition technology.
According to claim 11,
The storage device, when executed, the processor,
Additional storing instructions for determining a match rate between information associated with the first neuro-mechanical fingerprint and information associated with the second neuro-mechanical fingerprint.
Electronic glasses device utilizing biometric recognition technology.
According to claim 12,
The storage device, when executed, the processor,
If the match rate is equal to or higher than a preset access match rate, additionally storing instructions for decrypting the encrypted data using the first neuro-mechanical fingerprint.
Electronic glasses device utilizing biometric recognition technology.
According to claim 13,
The storage device, when executed, the processor,
Additional storage of commands for transmitting the encrypted data to a remote server
Electronic glasses device utilizing biometric recognition technology.
According to claim 14,
The storage device, when executed, the processor,
Additional storage of commands for receiving the encrypted data from a remote server
Electronic glasses device using biometric technology
According to claim 8,
Further comprising an access device that controls locking or unlocking the door,
The storage device, when executed, the processor,
In response to whether the user is the authenticated user, additionally storing a command to control the locking or unlocking of the door
Electronic glasses device utilizing biometric recognition technology.
According to claim 1,
Further comprising at least one lens 1704 coupled to the frame
Electronic glasses device utilizing biometric recognition technology.
According to claim 1,
Further comprising a wireless communication device (280, 332, 1816, 1919) electrically connected to the processor.
Electronic glasses device utilizing biometric recognition technology.
According to claim 18,
Further comprising at least one RF circuit 365 and a wireless device antenna 1905 electrically connected to the wireless communication device.
Electronic glasses device utilizing biometric recognition technology.
According to claim 1,
A power supply unit that supplies power to the electronic glasses device; and
Further comprising a battery electrically connected to the power supply unit.
Electronic glasses device utilizing biometric recognition technology.