KR20200093392A - An apparatus related to user identification, authentication, encryption using biometrics technology and method for operation the same - Google Patents

Abstract

Various embodiments of the present disclosure relate to an electronic device and which is more improved than the existing technology by utilizing a biometrics technology and related to more convenient user identification, authentication, and encryption and an operation method thereof. The operation method of the electronic device includes the steps of: capturing user eye movement, including involuntary eye movements; creating a unique pattern for identifying a user based on the captured involuntary eye movements; storing the unique pattern in a nonvolatile storage device in a security area; and authenticating the user with the electronic device based on the stored unique pattern. Various other embodiments may also be possible.

Description

User identification, authentication and encryption device using biometrics technology and its operation method{AN APPARATUS RELATED TO USER IDENTIFICATION, AUTHENTICATION, ENCRYPTION USING BIOMETRICS TECHNOLOGY AND METHOD FOR OPERATION THE SAME}

The present invention relates generally to electronic devices related to user identification, authentication, and encryption and methods of operation thereof. In particular, the present invention relates to an electronic device related to user identification, authentication, and encryption using various biometric technologies and a method of operating the same.

User access to the electronic device and database is primarily through a login name and password. Recently, as the use of portable electronic devices such as laptop computers and smart phones has increased, the need for accurate authentication of electronic devices and users has become more important to reduce the risk of unauthorized use and illegal data use. For example, with the emergence of many types of healthcare electronic devices, the privacy of health data acquired in mobile healthcare electronic devices has become important. In addition, as banking and payments using mobile electronic devices increase, the importance of authenticated use increases.

The present disclosure intends to provide an improved and convenient electronic device related to user identification, authentication, and encryption, and a method of operating the same, using biometric technology and the like.

User authentication for the use of local electronics is often performed by remote servers. The software application of the local electronic device often stores the user's login name and password to facilitate the use of the electronic device and software application. Protecting users and their login names and passwords to protect their access to electronic devices becomes even more important when electronic devices are lost or stolen. In the present situation where electronic devices are used as payment means, such as credit card transactions, it is becoming more and more important to protect local access of user electronic devices.

Biometrics are used for user authentication to provide more enhanced protection of mobile electronic devices. Biometrics have an anatomical approach (e.g., physical form, such as fingerprints, iris scans, blood vessels, facial scans, DNA, etc.) and an behavioral approach (typing or keystroke, handwritten signature, voice or intonation). For example, the user's anatomy, such as a fingerprint, is used for local authentication of the user and access control of the electronic device.

The operation method of the electronic device according to an embodiment of the present disclosure collects a motion signal from a part of the user's body from a motion sensor, filters unwanted signals from the motion signal, and filters the filtered motion Neuro-muscular micro-motion data is extracted from the signal, unique characteristics are extracted by performing signal processing and feature extraction of the neuromuscular micro-motion data, and neurodynamics are based on the unique characteristics. Generating an identifier (neuro-mechanical identifier).

A method of locally authenticating an authenticated user and controlling access to an electronic device according to an embodiment of the present disclosure includes using the electronic device to respond to a user's body motion in response to a detected micro-motion signal. For generating neuromechanical fingerprints,

A matching ratio of the neurodynamic fingerprint is generated in response to an authenticated user calibration parameter, and user access control to the electronic device is performed in response to the matching ratio.

An operation method of an electronic device according to an embodiment of the present disclosure, captures a user's eye movement including an involuntary eye movement, and captures the captured unconscious eye movement A unique pattern for identifying the user is generated based on the basis, the unique pattern is stored in a non-volatile storage device in a secure area, and the user authentication is performed with the electronic device based on the stored unique pattern.

It provides an improved and convenient user identification, authentication, and electronic device related to the encryption device and its operation method by utilizing biometric technology.

1 shows a category of the identification method.
2 is a block diagram of an operating environment of electronic devices according to an embodiment.
3 is a block diagram of a configuration of an electronic device according to an embodiment.
4 is a diagram illustrating an example of using a nerve fingerprint according to an embodiment.
5A to 5F are flowcharts or flows of operations illustrating an example of a method for generating a neurodynamic identifier according to an embodiment.
6 illustrates a phase diagram of Poincare for each user according to an embodiment.
7 illustrates the results of analysis of different users' capstrum (CEPSTRUM) according to an embodiment.
8 is a flowchart exemplarily showing a method for generating a neural eye identifier according to an embodiment.
9 shows a cross-sectional view of the eyeball of a human skull.
10A-10C show diagrams of various muscles connected to the eye that cause various eye movements.
11 shows a diagram of a process for obtaining unconscious eye movement.
12A and 12B show a diagram of an enlarged portion of the retina with a graph of unconscious eye movements over a region of the conical cell band in response to holding the object for a certain period of time by the user.
13A-13C show diagrams of various combinations of unconscious eye movements that can be used for user identification and authentication.
14A-14C show diagrams of features that can be observed vascularly in a video image of the eye that can be used to detect unconscious eye movement.
15 shows a block diagram of an electrooculography system for directly generating a user's unconscious eye movement signal.
16A shows a diagram of an electronic device with a video camera that can be used to obtain a user's eye movement image.
16B shows a block diagram of an electronic device with the video camera shown in FIG. 16A that can be used to obtain a user's eye movement image.
17A and 17B show diagrams of electronic glasses with video cameras that can be used to acquire images of user eye movements.
18A and 18B show diagrams of VR devices with video cameras that can be used to obtain images of the user's eye movements.
19A shows a diagram of an eye including a contact lens with an emitter device to assist in obtaining data about the user's eye movement.
19B-19D show diagrams of contact lenses with various emitter devices that can be used to help users obtain data regarding eye movements.
19E shows a functional block diagram of an active emitter device.
19F shows a side view of a sensing device near a contact lens mounted on the eye that can be used to obtain data regarding the user's eye movement.
20A and 20B show an apparatus for acquiring eye movements attached for access control of a building structure.
21A and 21B show independent eye scanners for authenticating a user to the system.

In the following detailed description of embodiments of the present disclosure, many specific details and various examples are developed for sufficient understanding. It is clear and clear to those skilled in the art to which this technology pertains can be practiced without these specific details and with various changes or modifications within the scope of the present disclosure. In certain instances, well-known methods, procedures, components (parts, parts), functions, circuits, well-known or typical details have not been described in detail in order not to unnecessarily obscure the embodiments of the present disclosure.

The terms, words, and expressions described in the present disclosure are merely for describing embodiments of the present disclosure and are not intended to limit the scope of the present disclosure. Unless defined otherwise, all expressions (or terms), including technical and scientific terms, may have the same or similar meaning in context to those of ordinary skill in the art. In some cases, although the expression (or term) is defined in the present disclosure, it is not interpreted to limit or exclude the scope of the embodiments of the present disclosure.

Embodiments associated with the present disclosure may be implemented with devices, methods, server-client devices and/or methods, collaboration of devices and/or methods, chipsets, computer programs, or any combination of those described above. . Accordingly, the embodiment may be implemented with all hardware (including a chipset), all software (firmware, any form of software, etc.) or may be implemented with a combination of software and hardware. Software and hardware types can be described as modules, units (units or units), components (parts, parts), blocks, components, members, systems, and subsystems. Also, the present embodiment may be implemented in the form of program code (including computer files) usable in a computer implemented in media.

The expressions “one embodiment”, “one embodiment”, “an embodiment”, “any embodiment”, “one example”, and “example” are specific features described in connection with the examples or embodiments of the present disclosure. , Can mean a structure or characteristic. Thus, all of these expressions used herein are not necessarily referring to the same embodiment or example. Further, a particular function, structure, or characteristic may be combined in any suitable combination and/or subcombination in one or more embodiments or examples.

The expression “one” or “one” in the singular form may include the plural form unless expressly stated differently in the text. For example, “one sensor” may refer to one or more sensors.

In some cases, expressions such as “first”, “second”, “first”, “second”, “first”, or “second”, etc. have been used to describe various components, but these components are not used It is not limited by. This expression may be used to distinguish one component from another component and may be independent of the order or importance of components. For example, the first sensor may be referred to as a second sensor, and likewise, the second sensor may be referred to as a first sensor. The first sensor and the second sensor are both sensors, but may not be the same sensor.

The expression “and/or” can include any possible combination of one or more of the listed items. For example, "A or B", "at least one of A and B", "at least one of A or B", "one or more of A or B", "one or more of A and B", The expressions "A and/or B", "at least one of A and/or B" or "one or more of A and/or B" include "at least one A", "at least one B" ", or "including at least one A and at least one B."

"Have (have)", "have (have)", "can be (have)", "includes", "includes", "can contain", "consists", "consists of The expression ", or "can be constructed" or "includes" is a component, feature, step, action, function, numerical value, component (part, part), member, or combination thereof (presence) , But excluding one or more additions or presences of components, features, steps, actions, functions, numerical values, components (parts, parts), members or combinations thereof no. For example, a method or device composed of a list of components is not necessarily limited to being composed of only these components, but may include components not explicitly listed in the method or device. Can.

When a component is said to be "connected to" or "coupled to or coupled with" another component, a component is directly connected to another component, or at least one or more It can be said that it is connected or combined through another component. On the other hand, when a component is said to be "directly connected" or "directly coupled" with another component, it can be said that it is connected or combined without passing through another component.

In the present disclosure, embodiments of various types of electronic devices related to user identification, authentication, and encryption and methods of operation thereof are described.

In one embodiment, the electronic device is a smart phone, tablet computer, telephone, mobile phone, telephone, e-book reader, navigation device, desktop computer, laptop computer, workstation computer server computer, camera, camcorder, Electronic pen, wireless communication equipment, access point (AP or wireless router), projector, electronic board, copier, watch, glasses, head mount device, wireless headset/earphone, electronic clothes, various types of wearable devices, TV, DVD player, Audio player, digital multimedia player, electronic picture frame, set top box, TV box, game machine, remote controller, bank ATM, payment system device (including POS, card reader), refrigerator, open, microwave, air conditioner, vacuum cleaner, washing machine, Dishwasher, air cleaner, home automation control device, smart home device, various kinds of home appliances, security control device, electronic lock/open device (including key and lock), electronic signature receiver, various types of security system device, blood pressure Measurement device, blood sugar monitoring device, heart rate monitoring device, body temperature measurement device, magnetic resonance imaging device, computed tomography device, various types of medical measurement devices, various types of medical devices, water meters, electric meters, gas meters, electromagnetic wave meters, It may be a thermometer, various types of measuring devices, artificial intelligence (AI) devices, artificial intelligence speakers, artificial intelligence robots, various types of IoT devices, or the like.

The electronic device may be a combination or part of one or more of the aforementioned devices. In one embodiment, the electronic device may be part of furniture, buildings, structures, or machinery (including vehicles, cars, airplanes, or ships) or may be in the form of embedded boards, chipsets, computer files, or some sort of sensor. The electronic device of the present disclosure is not limited to the aforementioned devices, and may be a new type of electronic device as technology development progresses.

1 is a view showing a category of the identification method.

Referring to FIG. 1, various known anatomical and behavioral identification methods can be identified. The behavioral identification method is related to the user's behavior or habits. Known anatomical identification methods relate to the user's physical characteristics such as fingerprints, iris eye scans, blood vessels, facial scans and/or DNA.

Some user motions are customary or part of the user motion repertoire. For example, signing a document is a motion that creates a user's behavioral habits. In general, the motion of signing using a writing instrument is macro motion or large scale motion. Since most of these movements are movements according to the user's consciousness or intention, it may be referred to as a voluntary movement. For example, from the large movement of a signed signature, one can visually determine whether the person who signed the signature is left-handed or right-handed.

This might be useful, but there is Micro Motion, a very small action that the user makes when signing, making another action, or simply not. These micro-motions are derived from nerves or are nerve-based and invisible. Therefore, it belongs to the involuntary movement, not the movement according to the user's consciousness or intention. This micro-motion of the user is due to the unique neuromuscular anatomy of each human, and may also be referred to as a neuro-derived micro motion in the present disclosure. This micro-motion is also linked to the motor control process from the individual motor cortex to the hand. At least one sensor, signal processing algorithm and/or filter may be used to obtain an electronic signal (“motion signal” and “micro motion signal”) that includes the user's nerve-inducing micro-motion. Of particular interest are micro-motion electronic signals that represent the user's micro-motion within the motion signal.

The user's micro-motion is related to the cerebral cortex and subcortical control of motor or other human nervous system motor activity. Like a mechanical filter, a person's particular anatomical musculoskeletal system can affect a user's micro-motion and contribute to a motion signal that includes the user's micro-motion. The signal thus contributed may be referred to as a neuro muscular tone as a signal of muscle movement by a nerve signal. The motion signal obtained from the user may reflect a part of a unique accommodating control loop including a brain and a unique receptor present in the user's human body.

When the motion signal is properly analyzed for the micro-motion signal representing the user's micro-motion, the resulting data can generate a unique and stable physiological identifier, especially a neurological identifier, which can be used as an unencrypted signature. This unique identifier can be referred to as the user's Neuro-Mechanical Fingerprint. Neuromechanical fingerprints can also be used in this disclosure with the term "NeuroFingerPrints (NFP)".

2 is a block diagram showing an operating environment of electronic devices according to an embodiment of the present disclosure.

As shown in FIG. 2, the electronic device 201 includes a processing unit 210, a sensor 220, an input/output interface 230, a display 240, a nerve fingerprint (NFP) accelerator 250, and a memory 260 ), a power system 270, a communication interface 280, and the like.

It can be understood that this is only one example of an embodiment of the present disclosure. The electronic devices 201, 202, 203, 204, and 205 may be composed of more or fewer components than the number shown, and two or more components are combined or set or disposed of the components May be configured differently. The various components shown may be implemented in hardware, software, or a combination of hardware and software. The electronic devices 201, 202, 203, 204, and 205 may be connected to each other through a network 206 or a communication interface 280.

The processing unit 210 may include at least one central processing unit, and the at least one central processing unit may include one or more processing cores. The processing unit 210 may include at least one coprocessor, a communication processor, a signal processing processor, a graphic processing processor, a low-power sensor control processor, or a special-purpose controller in addition to the heavy-duty processing device. In addition, various layers of internal volatile and non-volatile memories have an initial booting operation for electronic devices, an operation for communication with an external electronic device, an operation for downloading an initial booting or loading program from an external electronic device, an interrupt operation, and a program. Functions such as operations for improving performance during execution can be performed. The processing unit receives instructions in the form of a program through a memory, a communication module, etc., decodes the received instructions, and performs operations according to the decoded instructions, processing data, storing results, and controlling associated peripherals. It can perform identification, authentication, encryption, or functions utilizing them associated with a fingerprint (NFP). The processing unit may be a processor, an application processor (AP), a central processing unit (CPU), a micro controller unit (MCU), a controller, etc. Often used interchangeably.

The sensor 220 may detect or measure the state, physical quantity, etc. of the electronic device, and convert it into an electrical signal. The sensor 220 is an optical sensor, RGB sensor, IR sensor, UV sensor, fingerprint sensor, proximity sensor, compass, acceleration sensor, gyro sensor, barometer, grip sensor, magnetic sensor, iris sensor, GSR (Galvanic Skin Response) sensor , EEG (electroencephalography) sensor, and the like. The sensor 220 collects motion signals from parts of the user's body and delivers them to at least one component of the electronic device 201, including the processing unit 210 of the electronic device, the NFP accelerator 250, and the like. Therefore, it can perform identification, authentication, encryption, or a function utilizing them associated with a NFP.

The input/output interface 230 receives an input interface including a signal and/or command from the outside of the user or the electronic device 201 and transmits it to the inside of the electronic device, and a signal to the outside of the user or the electronic device 201 or It can act as an interface that transforms into the form of output including commands. For example, input button, LED, vibration motor, various serial interfaces (e.g., Universal Serial Bus (USB), Universal asynchronous receiver/transmitter (UART), High Definition Multimedia Interface (HDMI), Mobile High-definition MHL Link), IrDA (Infra-red Data Association, etc.).

The display 240 may display various contents such as images, texts, and videos to the user. The display 240 may be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or a hologram output device. The display 355 may include a display driving circuit (DDI) and a display panel, and the display driving circuit transmits an image driving signal corresponding to the image information received from the processing unit 301 to the display panel to set a predetermined frame. The image can be displayed according to the frame rate. The display driving circuit may be implemented in the form of an IC, and may include components such as a video memory, image processing unit, display timing controller, and multiplexer capable of storing image information. The display 240 may include an input device such as a touch screen, an electronic pen input panel, a fingerprint sensor, a pressure sensor, and/or an output device such as haptic feedback. Depending on the specifications of the electronic device 201, the display 240 may not be selectively provided, and may be formed of at least one light emitting diode in a very simple form. The display 220 is a position in which a user touches a part of the user's body to the electronic device 201 to collect a motion signal from a part of the user's body, a start state of acquisition of a motion signal, a state of acquisition of a motion signal, By performing a function of displaying a collection start state, such as an acquisition completion state of a motion signal, it is possible to perform identification, authentication, encryption, or a function utilizing them associated with a neural fingerprint (NFP).

The neural fingerprint accelerator 250 is used to increase the speed of performing an operation processing a signal obtained from a user's body part, or to perform identification, authentication, encryption, or a function utilizing them associated with the neural fingerprint (NFP). Performs a part or all of the functions to increase the performance of the overall system.

The memory 260 includes volatile memory 262 (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), non-volatile memory 264 (eg, NOR flash Memory, NAND flash memory, Erasable and Programmable ROM (EPROM), Electrically Erasable and Programmable ROM (EPMROM), Hard Disk Drive (HDD), Solid State Drive (SSD), Secure Digital (SD) card memory, Micro SD memory, MMC (Multimedia Card) memory, or the like. The non-volatile memory 264 includes at least one boot loader, an operating system 291, a communication function 292 library, a device driver 293, a nerve fingerprint (NFP) library 294, an application 295, and user data 296. ) And so on. The volatile memory 262 is supplied with power to the electronic device and starts to operate. The processing unit 210 performs an operation of loading programs or data stored in the non-volatile memory into the volatile memory 2620, and through the series of operations interfacing with the processing unit 210 during operation of the electronic device, the main memory Can play a role.

The power system 270 may perform functions such as supplying power to the electronic device 201, controlling and managing the power, and the like. The power system may include a PMIC (Power Management Integrated Circuit), a battery 272, a charging IC, and a fuel gauge. The power system can receive AC or DC power as an input. Wired charging and wireless charging may be provided to charge the supplied power to the battery 272.

The wireless communication interface 280 may include, for example, cellular communication, Wi-Fi communication, Bluetooth, GPS, RFID, NFC, etc., and configure an RF unit for wireless communication. The RF unit may include a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna.

3 is a block diagram showing the configuration of an electronic device according to an embodiment of the present disclosure.

As shown in FIG. 3, the electronic device 300 includes a processing unit 301, a camera 350, an input/output interface 353, a haptic feedback controller 354, a display 355, and short-range wireless communication 356. , External memory slot 357, sensor 370, memory 390, power system 358, clock source 361, audio circuit 362, SIM card 363, wireless communication processor 364, RF Circuit 365, a neural fingerprint (NFP) accelerator 366, and the like.

It can be understood that this is only one example of an embodiment of the present disclosure. The electronic device may be composed of more or fewer components than the illustrated number, and two or more components may be combined, or the configuration or arrangement of the components may be configured differently. The various components shown can be implemented in hardware, software, or both hardware and software.

The processing unit 301 may include at least one central processing unit 302 and the at least one central processing unit may include one or more processing cores. The processing unit 301 may include at least one coprocessor, a communication processor, a signal processing processor, a graphic processing processor, a low-power sensor control processor, or a special-purpose controller in addition to the heavy-duty processing device. The processing unit 301 may be implemented as a system on chip (SoC) including various components in the form of a semiconductor chip. In one embodiment, the processing unit 301 is a GPU (Graphics Processing Unit) 320, a digital signal processing (DSP, Digital Signal Processing) 321, an interrupt controller 322, a camera interface 323, a clock controller 324, display interface 325, sensor core 326, position controller 327, security accelerator 328, multimedia interface 329, memory controller 330, peripheral interface 331, communication/connectivity 332, an internal memory 340, and the like. In addition, the internal volatile and non-volatile memories of various hierarchical structures have an initial boot operation of an electronic device, an operation for communication with an external electronic device, an operation for downloading an initial boot or loading-related program from an external electronic device, an interrupt operation, Functions such as operations for improving performance during program execution can be performed. The processing unit 301 receives a program-type instruction through a memory 390, a communication/connectivity 332 or a wireless communication processor 364, decodes the received instruction, and operates according to the decoded instruction, Through processing data, storing results, and performing operations to control associated peripheral devices, it is possible to perform identification, authentication, encryption, or functions utilizing them associated with a neural fingerprint (NFP). The processing unit may be a processor, an application processor (AP), a central processing unit (CPU), a micro controller unit (MCU), a controller, etc. Often used interchangeably.

The central processing unit 302 may include at least one or more processor cores 304, 305, and 306. The central processing unit 302 may include processor cores with relatively low power consumption and processor cores with high power consumption, but may combine high-performance processor cores, and bundle a plurality of cores to form a first cluster 303 and a second cluster 314. It may include at least one or more clusters. This structure can be said to be a technique that is used to improve the performance and power consumption of electronic devices by dynamically allocating cores in consideration of computational power and current consumption in a multicore environment. The processor core may be applied with technology for enhancing security. The ARM® processor, one of the well-known mobile processors, has this security technology built into its processor and is called TrustZone®. For example, the first core 304, which is one physical processor core, may be divided into a normal mode 307 and a secure mode 308 to operate. The processor register and interrupt processing are separated so that access to resources (eg, peripherals or memory areas) that require security can be made accessible only in the secure mode. The monitor mode 313 serves to enable mode switching between the normal mode 307 and the security mode 308. In the normal mode 307, a special command may be used to switch to the secure mode 308, or an interrupt or the like may be used to switch from the normal mode 307 to the secure mode 308. The applications performed in the normal mode 307 and the secure mode 308 are separated so as not to affect the applications performed in the respective modes, thereby allowing the application requiring high reliability to be executed in the secure mode 308 to secure the application. Reliability can be increased. Security can be enhanced by allowing some of the operations in performing identification, authentication, encryption, or functions associated with the NFP, to operate in the security mode 308.

The camera 350 may include a lens for obtaining an image, an optical sensor, an image processing processor (ISP), and the like, and may acquire still images and videos. In order to provide various functions, a plurality of cameras (first camera 351, second camera 352, etc.) may be included.

The input/output interface 353 receives an input interface including a signal and/or command from the outside of the user or the electronic device 300 and transmits it to the inside of the electronic device, and a signal to the outside of the user or the electronic device 300 or It can act as an interface that transforms into the form of output including commands. For example, input button, LED, vibration motor, various serial interfaces (e.g., Universal Serial Bus (USB), U Universal asynchronous receiver/transmitter (UART), High Definition Multimedia Interface (HDMI), Mobile High-MHL definition Link), IrDA (Infra-red Data Association, etc.).

The haptic feedback controller 354 provides a tactile sense to the user by including a vibration motor called an actuator, for example, to provide a user with a function to sense a specific sense through tactile sense. Can.

The display (touch-sensitive display) 355 may display various contents such as images, texts, and videos to the user. The display 355 may be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, or a hologram output device. The display 355 may include a display driving circuit (DDI) and a display panel, and the display driving circuit transmits an image driving signal corresponding to the image information received from the processing unit 301 to the display panel to set a predetermined frame. The image can be displayed according to the frame rate. The display driving circuit may be implemented in the form of an IC, and may include components such as a video memory, image processing unit, display timing controller, and multiplexer capable of storing image information. Depending on the specifications of the electronic device 300, the display 355 may not be selectively provided, and may be formed of at least one light emitting diode in a very simple form. The display 355 may further include an input device such as a touch screen, an electronic pen input panel, a fingerprint sensor, a pressure sensor, and/or an output device such as haptic feedback. The display 355 is a position in which the user touches a portion of the user's body to the electronic device 300 to collect a motion signal from a part of the user's body, the start state of the acquisition of the motion signal, the state of the acquisition of the motion signal, By performing a function of displaying a collection start state, such as an acquisition completion state of a motion signal, it is possible to perform identification, authentication, encryption, or a function utilizing them associated with a neural fingerprint (NFP).

The short-range wireless communication 356 is, for example, to perform communication with other electronic devices that are close to the electronic device 300, for example, NFC (Near Field Communication), RFID (Radio Frequency Identification), MST ( Magetic Secure Transmission).

The external memory slot 257 may include an interface on which a memory card (eg, SD card, Micro SD card, etc.) can be mounted to expand the storage space of the electronic device 300.

The power system 358 may perform functions such as supplying power to the electronic device 300, controlling and managing the power, and the like. The power system may include a PMIC (Power Management Integrated Circuit), a battery 359, a charging IC 360, and a fuel gauge. The power system can receive AC or DC power as an input. Wired charging and wireless charging may be provided to charge the supplied power to the battery 359.

The clock source 361 may include at least one frequency of a system clock, a frequency oscillator for transmitting and receiving RF signals, which is a standard for the operation of the electronic device 300.

The audio circuit 362 may include an audio input unit (for example, a microphone) and an audio output unit (receiver, speaker, etc.), and a codec for performing conversion between an audio signal and an electrical signal, through which the user And an electronic device. The audio signal obtained through the audio input unit is converted into an analog electrical signal, sampled, and digitalized to be transferred to other components (for example, a processing unit) in the electronic device 300 to perform audio processing. The digital audio data received from other components in the electronic device 300 may be converted into an analog electrical signal to generate an audio signal through the audio output unit.

The SIM card 63 is an IC card that implements a subscriber identification module to identify a subscriber in cellular communication. In most cases, a SIM card is mounted in a slot provided in the electronic device 310. Depending on the type of electronic device, it can also be implemented in the form of an embedded SIM in a form that is combined inside the electronic device, and each unique number can be recorded on the SIM card, and the fixed number is ICCI (Integrated Circuit Card). Identifier) and IMSI (International Mobile Subscriber Identity) information that is different for each subscriber line.

The wireless communication processor 364 may include, for example, cellular communication, Wi-Fi communication, Bluetooth, GPS, and the like. Through the wireless communication processor 364, identification, authentication, encryption, or functions utilizing them associated with a neural fingerprint (NFP) may be performed through cooperation with at least one other electronic device (including a server) through a network.

The RF circuit 365 may include a transceiver, a power amp module (PAM), a frequency filter, a low noise amplifier (LNA), or an antenna. By exchanging control information and user data with the wireless communication processor and processing unit, it is possible to perform transmission and reception through radio frequency in a wireless environment.

The neural fingerprint accelerator 366 increases the speed of performing an operation processing a signal obtained from a user's body part, or performs an operation for performing identification, authentication, encryption, or a function utilizing them associated with the neural fingerprint (NFP) Performs a part or all of the functions to increase the performance of the overall system.

The sensor 370 may detect or measure the state, physical quantity, etc. of the electronic device, and convert it into an electrical signal. The sensor 370 is a compass 371, optical sensor 372, fingerprint sensor 373, proximity sensor 374, gyro sensor 375, RGB sensor 376, barometer 378, UV sensor 379 ), a grip sensor 380, a magnetic sensor 381, an acceleration sensor 382, an iris sensor 383, and the like. The sensor 370 may further include an IR sensor, a galvanic skin response (GSR) sensor, an electroencephalography (EEG) sensor, and the like, which are not shown in the drawings.

The sensor 370 collects a motion signal from a part of the user's body and transmits it to at least one component of the electronic device 300 including a processing unit 301 of an electronic device, a NFP accelerator 366, and the like. Therefore, it can perform identification, authentication, encryption, or a function utilizing them associated with a NFP.

The memory 390 includes volatile memory 391 (eg, dynamic RAM (DRAM), static RAM (SRAM), or synchronous dynamic RAM (SDRAM)), nonvolatile memory 392 (eg, NOR flash) Memory, NAND flash memory, Erasable and Programmable ROM (EPROM), Electrically Erasable and Programmable ROM (EEPROM), Hard Disk Drive (HDD), Solid State Drive (SSD), Secure Digital (SD) card memory, Micro SD memory, MMC (Multimedia Media Card) memory, or the like. The non-volatile memory 392 includes at least one boot loader, an operating system 393, a communication function 394 library, a device driver 395, a nerve fingerprint (NFP) library 396, an application 397, and user data 398. ) And so on. The volatile memory 391 starts to operate when power is applied, and performs an operation in which the processing unit 301 loads programs or data stored in the non-volatile interface to interface with the processing unit 301 during operation of the electronic device. The main memory can be performed through a series of operations.

The electronic device 300 may acquire a signal from a part of the user's body through the sensor 370, and the obtained signal may be obtained from at least one of a processing unit 301, a neurofinger accelerator 366, and a memory 390. Through interaction, it is possible to perform identification, authentication, encryption, or a function utilizing them associated with a neural fingerprint (NFP). Identification, authentication, encryption, or functions utilizing them associated with the NFP, can be independently performed by the electronic device 300 and through cooperation with at least one other electronic device (including a server) through a network. It can be done.

4 is a diagram illustrating an example of using a neurofingerprint (NFP) according to an embodiment.

Referring to FIG. 4, when a user is holding an electronic device by hand, a motion signal for movement of a person's hand is acquired by a sensor of the electronic device, converted into an electrical signal, and a nerve-inducing micro-motion from the converted signal ( Neuro-derived micro motion or neuro muscular tone can be extracted.

When the extracted neural-inducing micro-motion or neuromuscular tone is properly analyzed, an original and stable physiological identifier, in particular, a neurological identifier, can be generated, which can be referred to as a unique NFP identifier for each user. That is, it is possible to recognize a nerve fingerprint by gripping the user's electronic device. Through this, a comparison with the user's nerve fingerprint stored in the security area of the electronic device is performed to determine whether or not the user is an authorized user. You can judge the case.

Specifically, a motion signal is collected from a part of the user's body from a motion sensor, an unwanted signal is filtered from the motion signal, and a neuro-muscular micro-motion is obtained from the filtered motion signal. ) Extracting data, extracting unique characteristics by performing signal processing and feature extraction on the neuromuscular micro-motion data, and generating a neuro-mechanical identifier based on the unique characteristics .

Further, in order to locally authenticate the authenticated user and control access to the electronic device, the electronic device is used to generate a neurodynamic fingerprint for the user in response to a micro-motion signal detected in the user's body part, and to authenticate A match ratio of the neurodynamic fingerprint may be generated in response to the user calibration parameter, and user access control to the electronic device may be performed in response to the match ratio. These operations include credit card payment, medical information collection and processing of medical devices, authentication or encryption means such as an authentication key for a security chip set, authentication or encryption such as user login in a cloud environment, and authentication or encryption by being mounted on a wearable device. It can be used in a variety of fields such as unlocking locks (door locks, unlocking screens, unlocking keys on cars, unlocking various devices).

5A to 5F are flow charts or flows of operations illustrating an example of a method for generating a neurodynamic identifier according to an embodiment. 6 is a graph showing a phase diagram of Poincare for each user according to an embodiment. 7 is a graph showing results of analysis of capstrums of different users according to an embodiment. For ease of understanding, FIGS. 5A to 5F will be described on the assumption of the electronic device shown in FIG. 3.

Referring to FIG. 5A, the electronic device 300 may collect a motion signal from a part of the user's body from a motion sensor (510). The motion sensor may include a sensor that can detect a user's movement or vibration. For example, the motion sensor may be a compass 371, a gyro sensor 375, an acceleration sensor 382, a camera 350, an optical sensor, a touch sensor of the tactical display 355, and the like. The electronic device detects movements and vibrations occurring in a part in contact with a user's body part, converts it into an electrical signal through a motion sensor, and performs conversion through sampling and quantization into a digital signal capable of signal processing. When the camera is driven by the motion sensor, an eye movement signal may be collected. The motion sensor may collect a signal by sampling at a predetermined sampling frequency over a predetermined time range (eg, 5, 10, 20, 30 seconds, etc.).

The sampling frequency is, for example, considering the frequency range of 3 Hz to 30 Hz, which is a frequency range in which neuromuscular micro-motion frequencies derived from nerves or due to nerve-based human neuromuscular anatomy are mainly observed, for example, two frequencies of 30 Hz. It may be 60 Hz, 200 Hz, 250 Hz, 330 Hz or more. The frequency range that is mainly observed can be determined by observing the frequency range differently depending on the application using the neural fingerprint, and signals of other frequency ranges necessary for accuracy or performance improvement can be used. The motion sensor may further perform an operation of removing noise or improving signal quality in order to increase the quality of the digital signal. When the electronic device 300 is implemented as a portable device, a problem of power consumption may become important. To this end, the electronic device 300 may operate in a sleep mode, and the electronic device 300 may receive the collected and digitally converted motion signal to the processing unit 301 through the sensor core 326. The sensor core 326 may further perform an operation of removing noise or improving the quality of the signal in order to increase the quality of the digital signal received from the motion sensor. The processing unit may operate to enter the sleep mode often to increase the efficiency of power consumption of the electronic device 300. In general, in the sleep mode, various methods such as cutting off the power of some components in the electronic device 300, switching to a low power mode, or lowering the frequency of the operating clock may be applied to minimize power consumption. It can be said that the efficiency of power consumption increases when the processing unit 301 enters the sleep mode, but since a delay may occur in terms of mutual response between a user and an electromagnetic period, an auxiliary processor such as the sensor core 326 May be provided in the processing unit or in the electronic device. Even if the processing unit 301 enters the sleep mode, the sensor core 326 observes the signal detection from the sensors 370, and it is determined that the processing of the processing unit 301 is always required. An operation may be performed to wake up from the sleep mode of the processing unit 301 through a bus or an interrupt signal. Identification, authentication, encryption, or a function utilizing them associated with a NFP is a security-required operation. In this case, the operation of controlling the data collection of the motion sensor may be operated by switching the first core 304 in the processor unit 301 to the security mode 308, for example. The signal transmitted through the bus or interrupt by the motion sensor or the sensor core 326 may be input to the monitor mode 313 to switch the first core 304 to the security mode 308. The core entering the security mode 308 may access or control system resources of the electronic device accessible only in the preset security mode.

The electronic device 300 may extract neuromuscular micro-motion data from the motion signal collected from the motion sensor (520). The operation of extracting a signal from the motion signal is processed by the central processing unit 302 of the processing unit 301, or in cooperation with a GPU 320, a digital signal processing (DSP) 321, a neural fingerprint (NFP) accelerator, etc. The signal can be extracted.

Filtering to remove unnecessary signals may be performed to extract micro-motion data from the collected motion signals (522). Unnecessary signals may include, for example, noise, macro motion signals, and distortion due to gravity. In the signal obtained from the motion sensor, the signal for a certain period (for example, about 1 to 2 seconds) at the beginning of the signal collection or at the end of the signal collection may include a lot of macro motion signals of the user and There may be shake, so you can throw it all away. When the electronic device is being charged, power noise may occur in the collected signal, so that the signal can be filtered in consideration of characteristics caused by power noise. The frequency of neuromuscular micro-motion derived from nerves or due to nerve-based human-specific neuromuscular anatomy is mainly observed in the range of 3 Hz to 30 Hz. Therefore, a signal ranging from 3 Hz to 30 Hz or 4 Hz to 30 Hz can be extracted from the collected motion signal by performing a signal processing algorithm. Depending on the characteristics of the unwanted signal to be removed, it is possible to change the cutoff frequency of the band filter of the signal processing algorithm. For example, in one embodiment, a signal ranging from 4 Hz to 30 Hz may be extracted, and in another embodiment, a signal ranging from 8 Hz to 30 Hz may be extracted. In another embodiment, a signal in a range of 4 Hz to 12 Hz or 8 Hz to 12 Hz may be extracted.

In one embodiment, the motion is separated from the small signal amplitude of the micro-motion to remove the macro motion (large movement of the user's body, movement of the arm or walking, running, jogging, hand movement, etc.) from the collected motion signal. The signal analysis method to classify can be used. For example, an interpretation method of "Time Series Classification Using Gaussian Mixture Models of Reconstructed Phase Spaces" (by Richard J. Povinelli et al., IEEE Transactions on Knowledge and Data Engineering, Vol. 16, No. 6, June 2004), Or "Estimation of Physiological Tremor from Accelerometers for Real-Time Applications" (by Kalyana C. Veluvolu et al., Sensors 2011, vol. 11, pages 3020-3036, attached hereto in the appendix ), the macro-motion signal can be removed and the micro-motion signal can be extracted.

The electronic device 300 may extract unique characteristics from the extracted neuromuscular micro-motion data (530). Through Poincare's phase information, CEPSTRUM analysis, Chaos analytics Lyapunov exponent analysis, Reconstructed Phase Space (RPS) based analysis and various signal processing technologies Unique property values can also be constructed with at least one linear combination of components. As shown in FIG. 6, when looking at the phase diagram of Poincare for each user, each user (610, 620, 630, 640) has a unique and different signal pattern, and the center of mass of each user pattern (612, 622, 632) , 642) is also unique and different for each user. At this time, signal processing and feature extraction may be performed to extract unique characteristics for each user from the micro-motion data (532). In order to extract the hidden pattern in each user's micro-motion, it can be detected through spectral analysis of the signal and inverse spectrum analysis of the signal. For example, it is possible to detect patterns and frequency intervals of repetitive cycles through a well-known CEPSTRUM analysis. In FIG. 7, when looking at the analysis results of different users' capstrum (CEPSTRUM), the peak values (P1, P2, P3, P4, P5) of the 5 capstrum amplitudes extracted from user 1 (710) ) And 5 capstrum amplitude peak values (P1, P2, P3, P4, and P5) extracted from user 2 (720) are different from each other, and 5 cuprics extracted from user 1 (710) Quefrency) (F1, F2, F3, F4, F5) and 5 quencities extracted from user 2 (720) (F1, F2, F3, F4, F5). It can be seen that one characteristic is being shown, and such a result can be included in the unique pattern information for each user.

The electronic device 300 may generate a neurodynamic identifier from the neuromuscular micro-motion data (540). In an embodiment, a set of result values for the operation of extracting 530 unique characteristics from the extracted neuromuscular micro-motion data is generated as a neurodynamic identifier in the form of a multi-dimensional data structure and stored in a non-volatile memory. To save. The neurodynamic identifier may be referred to as a unique fingerprint for each user.

In one embodiment, when performing the operations of the neurodynamic identifier generation method (510, 520, 530, 540), the operations, the electronic device 300 is a cluster of high-performance processor core (for example, the first cluster 303) When is a cluster of high performance cores, the first cluster 303 may be allocated and performed.

In one embodiment, when performing the operation of the method for generating the neurodynamic identifier (510, 520, 530, 540), the operations, the security mode of the core in the processor unit 301 (for example, the operation is the first core ( 304), it may operate by being switched to the security mode 308). The core entering the security mode 308 may access or control system resources of the electronic device accessible only in a preset security mode to perform the operations of the neurodynamic identifier generation methods 510, 520, 530, 540. .

In one embodiment, when the operation of the neurodynamic identifier generation method (510, 520, 530, 540) when the execution of the operation starts, when the electronic device 300 is in the sleep mode, the sensor core 326 By detecting the signal from the sensors 370 and to wake up from the sleep mode of the processing unit 301 through a bus or interrupt signal, the operation of the neurodynamic identifier generation method (510, 520, 530, 540) It can be done.

In one embodiment, when performing the operations of the methods for generating neurodynamic identifiers (510, 520, 530, 540), the operations are performed by driving a camera or an optical sensor with a motion sensor to collect eye movement signals to collect a neurodynamic identifier (eg For example, it is possible to generate a neural eye identifier).

Referring to FIG. 5B, the electronic device 300 may sense 551 a signal from a contact or a short distance from the outside of the electronic device 300 from a sensor inside the electronic device 300. The external signal source 550 may be a part of the body that generates muscle movement induced through the nerve due to the action of the human brain, or an object that generates a signal located at a short distance to the electronic device 300. .

The sensor may sense 551 a signal through contact or short-range non-contact from a human body, and may be configured as part or all of the sensor 370 of FIG. 3 or at least one of the sensors constituting the sensor 370 It can be implemented in a combination of the above. It may also include sensors not described in sensor 370. The sensor converts the signal to be measured into an electrical signal and utilizes digital technology to perform sampling and quantization of the signal at a preset sampling frequency to convert it into a digital signal that is easy for digital processing. In this case, the sampling frequency is, for example, considering a frequency range of 3 Hz to 30 Hz, which is a frequency range in which neuromuscular micro-motion frequencies derived from nerves or due to nerve-based human neuromuscular anatomy are mainly observed, for example, at a frequency of 30 Hz. It can be 60 Hz, 200 Hz, 250 Hz, 330 Hz, or more. The frequency range that is mainly observed can be determined by observing the frequency range differently depending on the application using the neural fingerprint, and signals of other frequency ranges necessary for accuracy or performance improvement can be used. The motion sensor may further perform an operation of removing noise or improving signal quality in order to increase the quality of the digital signal. When the electronic device 300 is implemented as a portable device, a problem of power consumption may become important. To this end, the electronic device 300 may operate in a sleep mode, and the electronic device 300 may receive the collected and digitally converted motion signal to the processing unit 301 through the sensor core 326. The sensor core 326 may further perform an operation of removing noise or improving the quality of the signal in order to increase the quality of the digital signal received from the motion sensor. The processing unit may operate to enter the sleep mode often to increase the efficiency of power consumption of the electronic device 300. In general, in the sleep mode, various methods such as cutting off power of some components in the electronic device 300, switching to a low power mode, or lowering the frequency of an operating clock may be applied to minimize power consumption. It can be said that the efficiency of power consumption increases when the processing unit 301 enters the sleep mode, but since a delay may occur in terms of mutual response between a user and an electromagnetic period, an auxiliary processor such as the sensor core 326 May be provided in the processing unit or in the electronic device. Even if the processing unit 301 enters the sleep mode, the sensor core 326 observes the signal detection from the sensors 370, and it is determined that the processing of the processing unit 301 is always required. An operation may be performed to wake up from the sleep mode of the processing unit 301 through a bus or an interrupt signal. Identification, authentication, encryption, or a function utilizing them associated with a NFP is a security-required operation. In this case, the operation of controlling the data collection of the motion sensor may be operated by switching the first core 304 in the processor unit 301 to the security mode 308, for example. The signal transmitted through the bus or interrupt by the motion sensor or the sensor core 326 may be input to the monitor mode 313 to switch the first core 304 to the security mode 308. The core entering the security mode 308 may access or control system resources of the electronic device accessible only in the preset security mode.

In the pre-processing 552, a pre-processing process is performed in order to increase the performance and efficiency of the processing in the subsequent step, the signal obtained by sensing 551 from the sensor or the data stored in the storage device 557. It is possible to remove signals in a certain section of the initial and last portions of signal acquisition, suppress noise components, or suppress components of features that are considered unnecessary for processing. In addition, unnecessary components and components in the case of high correlation between signals may be unnecessary, so that signal components can be suppressed. In addition, based on the application in which the signal is utilized, compression may be performed in a low-dimensional subspace through a digital filter or a dimensional reduction technique. Unnecessary signals may include, for example, noise, macro motion signals, and distortion due to gravity. In the signal obtained from the sensor, the signal for a certain period (for example, about 1 to 2 seconds) at the beginning of the signal collection or at the end of the signal collection may include a user's macro motion signal very much and the shaking of the electronic device There may be, so you can throw them all away. When the electronic device is being charged, power noise may occur in the collected signal, so that the signal can be filtered in consideration of characteristics caused by power noise. The frequency of neuromuscular micro-motion, derived from nerves or due to nerve-based human-specific neuromuscular anatomy, is mainly observed in the range of 3 Hz to 30 Hz. Therefore, a signal ranging from 3 Hz to 30 Hz or 4 Hz to 30 Hz can be extracted from the collected motion signal by performing a signal processing algorithm. Depending on the characteristics of the unwanted signal to be removed, it is possible to change the cutoff frequency of the band filter of the signal processing algorithm. For example, in one embodiment, a signal ranging from 4 Hz to 30 Hz may be extracted, and in another embodiment, a signal ranging from 8 Hz to 30 Hz may be extracted. In another embodiment, a signal in a range of 4 Hz to 12 Hz or 8 Hz to 12 Hz may be extracted. In one embodiment, it is separated from the small signal amplitude of the micro-motion to remove the macro motion (large movement of the user's body, movement of the arm or walking, running, jogging, hand movement, etc.) from the collected signal. A classification signal analysis method can be used. In the pre-processing 552, signals related to neuromuscular micro-motion may be extracted from signals collected from sensors. The operation of extracting the signal related to the neuromuscular micro-motion from the signal obtained from the sensor is all processed by the central processing unit 302 of the processing unit 301, or the GPU 320, digital signal processing (DSP) (321) ), a neurofingerprint (NFP) accelerator, etc., to extract the signal.

In the process of feature extraction 553, a process of extracting unique characteristics having meaning from a signal component transmitted through a process of pre-processing 552 is performed. These characteristics can have a value in the form of a data structure in the form of a measurable multidimensional vector with mathematical properties that can clearly distinguish the user's characteristics, and are sometimes referred to as a feature vector for convenience. Through Poincare's phase information, CEPSTRUM analysis, Chaos analytics Lyapunov exponent analysis, Reconstructed Phase Space (RPS) based analysis and various signal processing technologies The feature vector may also be composed of at least one linear combination of components. As shown in FIG. 6, when looking at the phase diagram of Poincare for each user, each user (610, 620, 630, 640) has a unique and different signal pattern, and the center of mass of each user pattern (612, 622, 632) , 642) is also unique and different for each user. In order to extract the hidden pattern in each user's micro-motion, it can be detected through spectral analysis of the signal and inverse spectrum analysis of the signal. For example, it is possible to detect patterns and frequency intervals of repetitive cycles through a well-known CEPSTRUM analysis. In FIG. 7, the capstrum of different users in one embodiment (CEPSTRU analysis results show that 5 capstrum amplitudes extracted from user 1 710 (CEPSTRUM Amplitude) peak values (P1, P2, P3, P4, P5) And 5 capstrum amplitude (CEPSTRUM width) peak values (P1, P2, P3, P4, and P5) extracted from user 2 (720) are different from each other, and 5 cuprances extracted from user 1 (710) ) (F1, F2, F3, F4, F5) and 5 quadrants extracted from user 2 (720) (F1, F2, F3, F4, F5). It can be seen that the characteristics are shown, and these results can be included in the feature vector, which is unique pattern information for each user, and the feature vector is used as training data for learning 553 in the next step. The learning algorithm performance is not excellent, and test data is needed to check whether the general data is well generalized.The storage device 557 also has test feature vectors prepared in advance to be used as test data that have been developed and verified in advance. Control of the flow may be passed to the step of learning 553, which is a step.

In the learning step 554, the extracted feature vector and the previously prepared test feature vector are used as inputs to select a model that can best represent the feature vector and predict the model. These learning stages can be generally classified into supervised learning, unsupervised learning, reinforcement learning, and more specifically, k-Neareast Neighbors. Linear Regression, Logistic Regression, Support Vector Machine (SVM), Decision Tree, Random Forestes, Neural Network, k-Means (k- Means), Hierarchical Cluster Analysis (HCA), Maximum Maximization, Principle Component Analysis (PCA), Kernel PCA (Kernal PCA), Locally-Linear Embedding (LLE), There are algorithms such as t-distributed stochastic neighbor embedding (t-SNE), apriori, eclat, etc. In the learning step 554, a combination of at least one learning algorithm including the various algorithms is included. The model prediction is performed by learning through the use of a single model prediction algorithm, or an ensemble learning method that can have better performance through a combination of various algorithms. The final model can be predicted by imposing a final score applied with a preset weight, and at least one combination of selected model vectors and parameters of the models through learning is stored in a storage device, and a nerve that can be distinguished for each user Can be used as a fingerprint.

In the evaluation step 555, evaluation of a model selected from the training feature vectors may be performed. In this case, a test feature vector can be used to estimate the generalization error of the performance level of the feature vector that has not been used yet. In some cases, all or part of the test feature vector may be used in the training step for model selection. In this case, a plurality of sets of test feature vectors may be stored in advance and stored in a storage device. If the performance satisfies the required level, the model and model parameters selected in the learning stage can be used to predict new future data in the future.

In the determination step 556, a model for the newly sensed data may be predicted using the model or model parameters selected in the learning stage to make a decision for the new data. The steps of FIG. 5B may be performed in a different order of steps depending on the case. When learning is completed through the process of FIG. 5B and selected models and model parameter data for a nerve fingerprint are prepared, a signal is sensed from a device such as a sensor (551), pre-processing (552), and pre-processed signal. It is possible to extract the feature vector from the data (553) and perform the determination (575) through a preselected prediction model. If the predicted models are selected by learning the nerve fingerprint for the authorized user of the electronic device 300 and the parameters for the predicted models are determined, a match in step 556 is whether the model is trained or not. The percentage (Match Percentage) becomes more than a certain value, and it is possible to determine whether the authorized user is a nervous fingerprint. If the data associated with the newly sensed nerve fingerprint is lower than a certain value, it is possible to determine whether or not it is the nerve fingerprint of the authorized user. The result of the decision 556 may be reported on the display or the result of the authentication may be notified to the application to prevent the use of unwanted electronic devices.

In one embodiment, when performing the operations (551, 552, 553, 554, 555, 556) of a method for generating a neurodynamic identifier, in all or part of each operation, the electronic device 300 may display a cluster of high-performance processor cores ( For example, when the first cluster 303 is a cluster of high-performance cores, the first cluster 303 may be allocated and performed.

In one embodiment, when performing the operations (551, 552, 553, 554, 555, 556) of a method for generating a neurodynamic identifier, in all or part of each operation, a security mode of a core in the processor unit 301 (eg For example, when the above operations are assigned to the first core 304, the operation may be performed by switching to the security mode 308). The core entering the security mode 308 may perform the operation of generating the neurodynamic identifier by accessing or controlling system resources of the electronic device accessible only in a preset security mode.

In one embodiment, when performing the operations (551, 552, 553, 554, 555, 556) of a method for generating a neurodynamic identifier, when the execution of the operations starts in all or part of each operation, the electronic device 300 When is in the sleep mode, the sensor core 326 detects signals from the sensors 370 and wakes up from the sleep mode of the processing unit 301 through a bus or interrupt signal, thereby enabling the neurodynamics. The operation of generating an identifier may be performed.

In one embodiment, when performing the operations (551, 552, 553, 554, 555, 556) of the method for generating a neurodynamic identifier, in all or part of each operation, the sensing 551 drives the camera or optical sensor A neurodynamic identifier (eg, a neuroocular identifier) may be generated by collecting eye movement signals.

Referring to FIG. 5C, the electronic device processes the electrical signal detected from a part of the user's body from a device such as a sensor of the electronic device 300 at a preset sampling frequency during a predetermined section, and quantizes the signal. A digital signal that can be generated (570), removes data of a certain section of the first or last part of the signal, suppresses signal components due to noise, gravity, and intentional motion (571), and various methods Or, through a combination of at least one or more algorithms, extract a preset measurable feature vector, including mathematical properties representing features that can distinguish the user from the user's neuromuscular function (572), and extract from the extracted feature vector By selecting a combination of one or more learning algorithms, selecting predictive models using an ensemble method, extracting parameters of the selected prediction models (573), and evaluating the performance of the selected and selected predictive models through a test feature vector prepared in advance (574). Determine the model. The extracted feature vectors and at least one combination of the selected models and parameters of the models through learning may be stored in a storage device and used as a nerve fingerprint that can be distinguished for each user. The step of FIG. 5C may be performed in some cases, omitting some steps. When learning is completed through the process of FIG. 5C and selected models and model parameter data are prepared, a signal is sensed from a device such as a sensor (570), pre-processing is performed (571), and the pre-processed signal data is used. Feature vectors are extracted (572), and prediction or determination (575) can be performed through a preselected prediction model. In the predicting or determining step 575, a model for the newly sensed data may be predicted using the model or model parameters selected in the learning step to make a decision about the new data. If the prediction models are selected by learning the nerve fingerprint for the authorized user of the electronic device 300, and the parameters for the prediction models are determined, whether the model is learned in the prediction or determination step 575. Match Percentage becomes more than a certain value, and it is possible to determine whether the authorized user is a neurofinger. If the data associated with the newly sensed nerve fingerprint is lower than a certain value, it is possible to determine whether or not it is the nerve fingerprint of the authorized user. The result of the prediction or decision 575 may be reported on the display or the result of the authentication may be notified to the application, thereby preventing the use of unwanted electronic devices.

The neurofingerprint generated by the method of FIGS. 5A to 5C is used for user authentication of electronic devices, access control of users, encryption or decryption of data, and detection of changes in user's health. Can be utilized. When a sensor that senses a nerve fingerprint is used by sensing an unconscious movement of the eye through an optical sensor such as a camera, it can be used for user identification, authentication, encryption, etc. using the fingerprint by extracting the nerve fingerprint through the movement of the eye. have.

Referring to FIG. 5D, the method of FIGS. 5A to 5C generates a neurofingerprint or a neuro-mechanical fingerprint (576), and determines the matching ratio of neuromechanical fingerprints corresponding to signals obtained from a user. Create (577), and controls the access to the user's electronic device in response to the match rate (578), if the match rate has a value above the access level, identifies the user as an authenticated user, and of the authenticated user Permit access to the electronic device (579), and if the matching ratio has a value less than a preset level, perform training and learning for the authenticated user again to select prediction models and update parameters of the selected models ( 580). By dividing the coincidence ratio into several sections, it is possible to distinguish the steps in which training and learning must be re-executed for each section. If the matching ratio belongs to the first specific section, the electronic device can voluntarily retrain and learn and update values such as parameters related to the nerve fingerprint without displaying to the user. When the matching ratio belongs to the second specific section, the user may indicate or notify the user through the display or the input/output device that training and learning need to be performed again. At this time, the first specific section and the second specific section may be set in advance by various combinations.

Referring to FIG. 5E, a method of FIGS. 5A to 5C is applied from a user's body part using a sensor of an electronic device attached to a door of a structure such as a building or machine or a part of a structure such as a building or machine. It generates 581 neuro-fingerprints or neuro-mechanical fingerprints (581) and acquires motion signals from parts of the user's body from sensors of electronic devices attached to parts of structures such as doors or buildings (machines) 582), and the signal obtained from the user performs a distinction between the signal for neurodynamic fingerprint and the user's intentional gesture action (583), and whether the coincidence rate of the neurodynamic fingerprint and the intentional gesture action are preset actions If a determination 584 is made, and the intentional gesture action is a preset action, and the coincidence rate of the neuromechanical fingerprint has a value higher than the access level, the user is identified as an authenticated user and the door is opened or locked according to the gesture action. It can be performed (580). At this time, if the preset operation is a lock gesture, locking is performed, and if the preset operation is an open operation, an open operation may be performed. If the match rate of neuromechanical fingerprints is lower than the access level, access may not be permitted.

Referring to FIG. 5F, a sensor of an electronic device is used to generate a neurofingerprint or a neuro-mechanical fingerprint from a user's body part by the method of FIGS. 5A to 5C (586), and An encryption key is generated (587) using a combination of all or part of the values, and encryption of the data of the electronic device is performed using the encryption key to store (588) the storage device and access to the encrypted data. The access 589 may be performed using the value of the neurodynamic fingerprint of the authenticated user.

8 is a flowchart exemplarily showing a method for generating a neural eye identifier according to an embodiment. For ease of understanding, FIG. 8 will be described by assuming the electronic device shown in FIG. 3.

As illustrated in FIG. 8, the electronic device 300 acquires an image of the movement of the user's eyeball with an optical sensor (eg, an RGB sensor 376, an iris sensor 383, a camera 350, etc.) and the eyeball It is possible to extract the motion signal of and convert it into a digital signal that can be processed through sampling and quantization. In order to collect the eye movement, a signal may be collected by sampling at a predetermined sampling frequency over a predetermined time range (eg, 5, 10, 20, 30 seconds, etc.). The sampling frequency is, for example, twice the frequency of 30 Hz, taking into account the frequency range of 3 Hz to 30 Hz, which is a frequency range in which unconscious eye movements are mainly observed derived from nerves or caused by nerve-specific human neuromuscular anatomy. It may be more than 60Hz, 200Hz, 250Hz, 330Hz, and the like. The frequency range that is mainly observed can be observed by setting the frequency range differently depending on the application. Also, signals in other frequency ranges required for accuracy or performance improvement may be used. The optical sensor may further perform an operation of removing noise or improving signal quality in order to increase the quality of the digital signal. When the electronic device 300 is implemented as a portable device, a problem of power consumption may become important. To this end, the electronic device may operate in a sleep mode, and the electronic device 300 may receive the collected and digitally converted eye movement signal to the processing unit 301 through the sensor core 326. The sensor core 326 may further perform an operation of removing noise or improving the quality of the signal in order to increase the quality of the digital signal received from the optical sensor. The processing unit may operate to enter the sleep mode often to increase the efficiency of power consumption of the electronic device 300. In general, in the sleep mode, various methods such as cutting off the power of some components in the electronic device 300, switching to a low power mode, or lowering the frequency of the operating clock may be applied to minimize power consumption. It can be said that the efficiency of power consumption increases when the processing unit 301 enters the sleep mode, but since a delay may occur in terms of mutual response between a user and an electromagnetic period, an auxiliary processor such as the sensor core 326 May be provided in the processing unit or in the electronic device. Even if the processing unit 301 enters the sleep mode, the sensor core 326 observes the signal detection from the sensors 370, and it is determined that the processing of the processing unit 301 is always required. An operation may be performed to wake up from the sleep mode of the processing unit 301 through a bus or an interrupt signal. Identification, authentication, encryption, or functions utilizing them associated with a neural fingerprint (NFP) or a neural eye fingerprint (NEP) can be said to be an operation requiring security. In this case, the operation of controlling the data collection of the optical sensor may be performed by switching the first core 304 in the processor unit 301 to the security mode 308, for example. The signal transmitted through the bus or interrupt by the motion sensor or the sensor core 326 may be input to the monitor mode 313 to switch the first core 304 to the security mode 308. The core entering the security mode 308 may access or control system resources of the electronic device accessible only in the preset security mode.

The electronic device 300 may extract unconscious eye movement data from the eye movement signal collected from the optical sensor (820 ). The operation of extracting the signal from the eye movement signal is processed by the central processing unit 302 of the processing unit 301, or the GPU 320, the digital signal processing (DSP) 321, the neural fingerprint (NFP) accelerator, etc. The signal can be extracted in cooperation. Filtering may be performed to remove unnecessary signals to extract unconscious eye movement data from the collected eye movement signals (822). Unnecessary signals may include, for example, noise, conscious eye movement signals, macro motion signals, and distortion due to gravity. In the signal obtained from the optical sensor, the signal for a certain period of time (for example, about 1 to 2 seconds) at the beginning of the signal collection or at the end of the signal collection includes a lot of conscious eye movement signals or macro motion signals of the user. It may be, and there may be shaking of the electronic device. When the electronic device is being charged, power noise may occur in the collected signal, so that the signal can be filtered in consideration of characteristics caused by power noise. The frequency of unconscious ocular movements derived from nerves or due to nerve-specific human neuromuscular anatomy is mainly observed in the range of 3 Hz to 30 Hz. Therefore, a signal ranging from 3 Hz to 30 Hz or 4 Hz to 30 Hz can be extracted from the collected eye movement signal by performing a signal processing algorithm. Depending on the characteristics of the unwanted signal to be removed, it is possible to change the cutoff frequency of the band filter of the signal processing algorithm. For example, in one embodiment, a signal ranging from 4 Hz to 30 Hz may be extracted, and in another embodiment, a signal ranging from 8 Hz to 30 Hz may be extracted. In another embodiment, a signal in a range of 4 Hz to 12 Hz or 8 Hz to 12 Hz may be extracted.

In one embodiment, in order to remove the conscious eye movement signal from the collected eye movement signal, a signal analysis method that classifies and separates the small signal amplitude of the unconscious eye movement may be used. For example, an interpretation method of "Time Series Classification Using Gaussian Mixture Models of Reconstructed Phase Spaces" (by Richard J. Povinelli et al., IEEE Transactions on Knowledge and Data Engineering, Vol. 16, No. 6, June 2004), Or "Estimation of Physiological Tremor from Accelerometers for Real-Time Applications" (by Kalyana C. Veluvolu et al., Sensors 2011, vol. 11, pages 3020-3036, attached hereto in the appendix ) Can be used to remove the conscious eye movement signal and extract the unconscious eye movement signal.

The electronic device may extract a unique characteristic from the extracted unconscious eye movement data (830).

The electronic device may generate a neural eye identifier from the unconscious eye movement data (840 ). In one embodiment, a set of result values for the operation of extracting 830 a unique characteristic from the extracted unconscious eye movement data is generated as a data structure having a multi-dimensional value as a neuro-ocular identifier and stored in a non-volatile memory. do.

In one embodiment, when performing the operations of the method for generating an ocular eye identifier (810, 820, 830, 840), the operations, the electronic device 300 is a cluster of high-performance processor core (for example, the first cluster 303) When is a cluster of high performance cores, the first cluster 303 may be allocated and performed.

In one embodiment, when performing the operation of the method 810, 820, 830, 840 of generating an eyeball identifier, the operations, the security mode of the core in the processor unit 301 (e.g., the operation is the first core ( 304), it may operate by being switched to the security mode 308). The core that has entered the security mode 308 may access or control system resources of the electronic device accessible only in a preset security mode to perform the operation of the method 810, 820, 830, 840 of the neural eye identifier generation. .

In one embodiment, when performing the operation of the method of generating the ocular eye identifier (810, 820, 830, 840), when the execution of the operation starts when the operation is performed, when the electronic device 300 is in the sleep mode, the sensor core At 326, the signals from the sensors 370 are sensed and awakened from the sleep mode of the processing unit 301 through a bus or interrupt signal, thereby generating the neuron identifiers 810, 820, 830, 840).

In one embodiment, the method for generating a neuron eyeball identifier is the same as the method for generating a nerve fingerprint or a neuro-mechanical fingerprint using a sensor or an optical sensor among the methods of FIGS. 5A to 5C. Can.

9 shows a cross-sectional view of an eye located within a human skull.

Referring to FIG. 9, a cross-sectional view of a person's eyeball 900 within the skull 902 is shown. The human eye 900 is an imperfect glob that can be moved within the skull 902 by multiple muscles. The action or process of changing the position of a person's eyeball 900 within the skull 902 is referred to as eye movement or eye motion.

The eyeball 900 interacts with the brain to obtain a color image for processing by the brain, retina 910, pupil 912, iris 914, fovea 916 ) And lens 918. The cornea 922 of the eye is imaged to the retina 910 through the pupil 912, the iris 914, and the lens 918. The pupil 912 changes its diameter to control the amount of light received by the retina 910. The retina 910 of the eye includes two types of photoreceptors, rod cells and cone cells. The retina contains about 120 million cone cells and 6 to 7 million rod cells. The rods are concentrated in the region without the rods of the retina called the center of the retina or the macula (centreum), providing maximum vision and color. Cone cells are smaller and denser than elsewhere in the retina 910. The nerve 926 is a cable of nerve fibers coupled to the eyeball, and transmits electrical signals from the rod and cone cells of the retina to the brain. There are no rods and cones at the point where the optic nerve leaves the eye through the retina. Thus, the optic nerve forms a "square zone" in the retina.

10A to 10C are diagrams showing various muscles connected to the eye, causing various eye movements of the eye.

In FIG. 10A, the left rectus 1001L and the right rectus 1001R cause the eye 900 to move horizontally from side to side as shown by arrow 1011.

In FIG. 10B, the upper rectus muscle 1001T on the upper line and the lower rectus muscle 1001B move the eyeball 9100 upward and downward as shown by arrow 212.

In FIG. 10C, the upper inclined muscle 1002S and the lower inclined muscle 1002I cause the eye 900 to roll and move as shown by the curved arrow 1013. These muscles can cause spontaneous eye movements according to human will, and unconscious eye movements that humans do not even know about. Other muscles around the eye 900 may also contribute to spontaneous and unconscious eye movement. Muscles are under the control of the body's nervous system, including the brain.

Unconscious eye movements are often considered pathological when observed by an ophthalmologist, unlike spontaneous eye movements. However, it is often more often observed that normal, physiological and microscopic unconscious movement occurs when the eye is fixed to an object. This very small unconscious eye movement is the normal physiological function of the body to prevent fatigue of rod and cone cells at the focal point of the retinal surface in the eye. Unconscious eye movements are generally considered disturbing or have been analyzed for research purposes.

The retina 910 of the human eye 900 can be scanned for various purposes. Retinal scanners for acquiring a two-dimensional map of the retina of the eye are known. The retina scanner is not intended to measure unconscious eye movements.

Various eye tracking systems have been used to detect conscious eye movements for various purposes. For example, a virtual reality headset for gaming may track conscious eye movements in video game gameplay. As another example, the heads-up display of the military system can track conscious eye movements for military purposes. However, the eye tracking system is for evaluating the field of view from the center and focal attention of the subject. Since the unconscious eye movement was obstructed or used for research purposes only, the unconscious eye movement was not measured.

Retinal sensitivity to light (photons) is defined by genetic factors by integrating the entire system by ocular motion (eye movement), ocular muscles, and specific brain wiring. The very small unconscious eye movement of the eye 900 is a normal physiological function of the human body. Any neurologically coordinating process will create characteristics that are unique to the individual, and thus can potentially be useful in identifying one individual from another. Thus, the unconscious eye movement of the eye 900 is unique to the individual and can be used for user identification and user authentication.

11 is a diagram of a fixed processor for obtaining unconscious eye movement of the eye. To help understanding, it will be described with a cross-sectional view of the eye shown in FIG. 9.

Referring to FIG. 11, the small or fine unconscious eye movement of the eye 900 may be obtained by a process of fixing an eye to an image of a target object. The target image may be a retinal image 1102 on the retina 910 of the eye 900. The target image may be generated by the target 1104 on the display device 1110. The user gazes or holds the target 1104 on the display device 1110 for a period of time (eg, 10 seconds to 15 seconds). Unconscious movements are too small and difficult to capture when viewed directly with the naked eye. The video camera 1112 acquires a series of images of the movement of the eyeball 900 during the fixation process.

To prevent conscious eye movements during the fixation process, the camera 1112 may be connected to the eyeball 900. However, this is impractical for authentication purposes. The conscious eye movement may be substantially filtered from the captured eye movement data to generate unconscious eye movement data.

In addition, fine subconscious eye movement (also referred to as micromotion of the subconscious eye in the present disclosure) is generally obtained during the fixation process, but also through large conscious eye movement even when the eye is moving to see, It can be obtained regardless of the focus point of interest. Video cameras can be used to acquire a sequence of images of large-scale eye movements, including small-scale or fine unconscious eye movements. Fine subconscious eye movements can be extracted from the acquired image sequence, including both subconscious and conscious movements.

12A and 12B are diagrams of an enlarged portion of the retina with a graph of unconscious eye movement across the region of the conical cell band in response to a user holding the target for a period of time. 15 is a block diagram of an electrooculography system for directly generating a user's unconscious eye movement signal. 16B is a block diagram of an electronic device with the video camera shown in FIG. 16A that can be used to obtain a user's eye movement image. The electronic device 1600 of FIG. 16B may be the electronic device 200 of FIG. 2 or the electronic device 300 of FIG. 3, and the electronic device 200 of FIG. 2 and the components of the electronic device 300 of FIG. 3 It may include some or all combinations, or may include components not shown in the present disclosure. To facilitate understanding, the electronic device 1600 will be described with reference to FIGS. 12A and 12B and the system of FIG. 15 using the same.

In FIG. 12A, the conical region 1200 is shown as an enlarged portion of the central center of the retina 910. 12A also shows a graph of unconscious eye movements in the conical region corresponding to the user securing to the object for about 10 seconds. 12A shows a plurality of cones within about 5 micron diameter of the retina 910.

Unconscious eye movements exist whether or not the user moves to a stationary object. Fine subconscious eye movements, including various kinds of eye movements, can be used to identify users.

In FIG. 12A, small unconscious eye movements, including saccade (also called fine intermittent movement) (1201A-1201F), curved drift (1203A-1202E), and zigzag-shaped tremor (1203A-1203E). Various types are shown.

FIG. 12B shows an enlarged view of the intermittent motion 1201F, the curve-shaped drift 1202E, and the zigzag-shaped tremor 1203E. The zigzag-shaped tremor 1203E was superimposed on the curved-shaped drift 1202E.

Drift eye movements 1202A-1202E are like random traces with no precise targets, especially when the eye is not focusing on anything. This eye movement changes the direction with a frequency of about 20 to 40 Hz and has a small amplitude characteristic. The trembling eye movement is characterized by very small amplitudes (eg, angles of 0.2 to 2-3 degrees) and high frequencies (eg, 40 Hz to 150 Hz, 90 Hz typical values). The ocular movement of the intermittent movement is characterized by an amplitude between 15-20 degrees angles, a fast movement (200-500 degrees/sec) and a relatively low frequency (eg, 0.1 Hz to 1-5 Hz).

Fine subconscious eye movement is the normal physiological function of the body to prevent fatigue of the rod and cone at the focal point on the retinal surface within the eye. The image of an object or object in the retina of the eye is constantly moving due to unconscious eye movement. The drift eye movement 1202A-1202E causes the image to drift slowly from the center of the center to the outside. The drift eye movement ends at the start of the intermittent eye movement. The ocular movement of the intermittent movement (1201A-1201F) brings the image back toward the center. The trembling eye movement 1203A-1203E superimposed on the drift eye movement 1202A-1202E has an amplitude across a plurality of cones to prevent fatigue of a single cone when staring or fixing an object.

This small unconscious eye movement is thought to prevent retinal fatigue. Retinal fatigue is the depletion of several important biochemicals that are essential for acquiring photons by retinal cells. Small unconscious eye movements cannot be detected by the naked eye. However, as shown in FIGS. 11 and 16B, an unconscious eye movement may be acquired and sensed/viewed/recognized by a retina scanner and a video camera at a speed sufficient to record eye movement.

Unconscious eye motion or movement can be performed by various means or methods, such as using infrared light emitting diodes (LEDs), pupil scanning methods, and even refractometers, to obtain the appropriate time or frequency and displacement resolution to obtain unconscious eye motion or movement. By modifying it, it can be detected.

Unconscious eye movements can be selectively detected using a safety record with sufficient sensitivity to generate an electroculogram (EOG) representing unconscious eye movements. Primitive EOG signals occur in the dipole between the ocular cornea and the retina.

Electrical signals are generated by the ocular motor muscles when the eye causes eye movement. These electrical signals generated by ocular motor muscles can be detected to indicate eye movements similar to the electromyography (EMG) of the eye.

Referring to FIG. 15, the safety record includes sensing an electrical signal by measuring a potential difference (voltage) between the retina (ground) and the cornea of the eye. Typically, electrodes are placed around the eye to measure the voltages above, below, left and right against one or more ground electrodes. The difference between left and right voltage measurements can be made to generate a signal that indicates horizontal eye movement. The difference between the up and down voltage measurements can be made to generate a signal indicative of vertical eye movement. The signal can be amplified and filtered before signal processing is performed. The analog signal is converted into a digital signal so that digital signal processing can analyze the EOG signal for the pattern in the user's unconscious eye movement.

The unconscious eye movement pattern is used as a method of performing an individual's physiological biometric recognition because it is associated with a particular neuro-muscle-retinal anatomy of the individual. Acquiring the fine motion of the unconscious eye movement of the eye can extract patterns unique to the individual and can be used to authenticate the identity of the individual. By analyzing the extracted pattern, it is possible to extract features providing a unique identifier for each individual. In one embodiment, the pupil can be identified and tracked using a high-speed, high-resolution camera. The image can be processed by a processor running image processing software to extract features from unconscious eye movements that uniquely identify the user.

Unconscious eye movements can be used with proprietary authentication methods or retinal scanners that acquire images of the retina anatomy of the eye. Unconscious eye movements can also be used as an authentication method with an iris scanner that acquires an iris anatomy image of the eye. Unconscious eye movements can also be used as an authentication method in conjunction with retinal and/or iris scanners that capture eye anatomy images.

Unconscious eye movement and retina and/or iris can be used to provide double or triple authentication. Retinal and iris scanning technologies do not contain neurological components. Retinal scanning technology simply maps the anatomical structure of the retina to perform authentication based on two-dimensional (2D) scanned images. Unconscious eye movement can add biometric parameters to current eye scanning technology.

U.S. Patent Publication No. 2014/0331315, filed by Birk et al., discloses a method for deriving biometric parameters from eye movements for use in an authentication protocol. Combining conventional authentication methods, such as eye movements based on an action repertoire that actively moves the eye's focus, such as "visually grasp pieces of information embedded in the display" and "passwords or other information known to the user" described in the abstract, have. It is said that Birk's behavioral repertoire may be a path of eye movement, or a feature of the user's eye movement that sequentially places a series of numbers/characters that match the password or code that the user knows.

Birk recognizes that there are two types of eye movements, conscious and unconscious. However, Birk states that the only active and conscious eye movements are used as input to cognitive technology. Birk uses a grid that is essentially defined, and the user's eyes must be recognized by moving around in a specific order. Birk uses a repertoire of user-specific behaviors (habits).

In contrast to Birk, embodiments of the present disclosure use only the unconscious movement of the normal physiological eye controlled by the brainstem and cerebral cortex. The unconscious eye movement obtained by the embodiment is not based on what the user does, but on what the user is.

U.S. Pat.No. 6,785,406 (Kamada), issued on August 31, 2004 to Mikio Kamada, collects data from users using ocular movement and iris contraction, not images or other body parts. It describes the iris authentication device to be possible. The eye movements used in Kamada use motions related to the vestibular control of the eye, which ensures eye-to-head coordinates when viewing objects while the head is moving. Kamada also uses the Optokinetic Nystagmus eye movement, which is a large cumo-intermittent movement in which the subject moves out of sight and brings the center of the retina back to the center.

The eye movements described in Kamada are not related to retinal fatigue and user-specific microscopic movements. The eye movement described in Kamada is a reflective movement. Kamada does not use eye movements to identify the user and checks whether the data obtained is from a real user. Since the eye movement described in Kamada is different for each user and is reflective, it cannot be used to identify the user, so it is like knee reflex and appears in everyone like knee reflex. Different levels of recognition may be accepted in Kamada, but not sufficient to distinguish individuals to provide identity.

U.S. Patent No. 8,899,748, issued to Brandon Lousi Migdal on December 12, 2014, describes a method for detecting ocular nystagmus movement associated with individual vestibular control (balance). However, vestibular ocular eye movement can be either vertical or horizontal eye movement and begins with a general positional reflex. Moreover, vestibular ocular eye movement lacks the precise granularity of unconscious ocular movement useful for distinguishing individuals by embodiments of the present disclosure.

U.S. Patent No. 9,195,890 issued to James R. Bergen on November 24, 2015 (hereafter, Bergen) discloses a biometric recognition method based on iris image comparison. Bergen's comparison is based on unique anatomical features extracted from the iris of the eye. The characteristics of the iris are various and detailed by Bergen. To align and match the iris to the stored iris pattern, Bergen has to correct tilt/position and motion as well as correct other image distortions. The eye movement mentioned in Bergen is large and undesirable because it interferes with the collection of quality data from Bergen's iris recognition system. This can be said to be contrary to the embodiments of the present disclosure.

U.S. Pat.No. 7,336,806 to Schonberg et al. (Schonberg) discloses iris-based recognition of users whose circumference is the key. Schonberg considers other features, including eye movements, to be the noise to be removed.

U.S. Patent No. 7,665,845 to Kiderman et al. (Kiderman) describes a video oculo graphic (VOG) system based on lightweight goggles. Kidderman's goggles are designed not to interfere with clinical measurements due to the weight of the measuring instrument. However, Kidderman does not disclose the use of unconscious eye movements of interest in embodiments of the present disclosure. Moreover, there is no explicit need for spatial resolution with respect to embodiments of the present disclosure.

Electrooculography can directly generate unconscious eye movement signals. When tracking eye movements using the acquired video image, the visual features of the eye are used in the video image. Pattern recognition can be used to detect visible features in each image of a video image.

14A-14C are diagrams of features seen in video images of the eye that can be used to detect unconscious eye movement.

Referring to FIG. 14A, a visible characteristic of the image of the eyeball may be the iris 914, the pupil 912, or the central condyle 916. Edge detection can be used to track unconscious eye movements from the video image of the eyeball 900 can be obtained while the user holds the object. Edge detection may be performed from the location of the perimeter 614 of the iris 1414, the perimeter 1412 of the pupil 912, or the center and 916.

As shown in FIG. 14B, the perimeter 1414 of the iris 914 is well defined and consistent, which is useful for tracking unconscious eye movements. The perimeter 1412 of the pupil 912 can, optionally, be used to track unconscious eye movements. However, the size and position of the pupil changes with light intensity.

As shown in FIG. 14C, the center 916 can be used to selectively track unconscious eye movements. However, tracking the center fossa 916 typically requires a light source that shines through the lens 918 to illuminate the retina 910 of the eye 900.

13A-13C are diagrams showing various combinations of unconscious eye movements that can be used for user identification and authentication. 15 is a block diagram of an electrooculography system for directly generating a user's unconscious eye movement signal. For ease of understanding, the systems of FIGS. 13A to 13C and FIG. 15 using the same will be described together.

In FIG. 13A, three types of unconscious eye movements including intermittent locus trajectory, drift and tremor can be used to determine each user's unique identifier. In one embodiment, features are extracted from all three unconscious eye movements to uniquely identify each user. The system continually extracts the same function for each user.

In FIG. 13B, two unconscious eye movements such as trajectory and drift of intermittent movement can be used to determine the unique identifier of each user. In another embodiment, features are extracted from trajectories and drifts of intermittent motion to uniquely identify each user. The system continually extracts the same function for each user.

In FIG. 13C, one unconscious eye movement, such as a tremor component, can be used to determine each user's unique identifier. In another embodiment, features are extracted from the tremor component of the eye movement to uniquely identify each user. The system continually extracts the same function for each user.

Referring to FIG. 15, an example of a safety record recording system 1500 that directly generates a motion signal of an unconscious eye of the user 1599 is illustrated. The plurality of electrodes 1501A-1501E are located in the eye area around the user's head/face to obtain the voltage of each eye during eye movement. The electrode can be part of a hood or face receiving device to connect the electrode to the surface of the user's head/face. The electrodes 1501A-1501B acquire vertical movement of the eyeball 900. The electrode 1501C-1501D acquires left and right movement of the eyeball 900. One or more electrodes 1501E provide a ground or zero voltage reference to each electrode 1501 -1501D.

The electrodes 1501A-1501D measure unconscious eye movements generated during the fixation process, up and down, left and right, and voltage with respect to one or more ground electrodes 1501E.

The upper and lower voltages of the electrodes 1501A-1501B with or without filtering are connected to the negative and positive inputs of the differential amplifier 1504A to amplify and calculate the difference between the upper and lower voltages. This forms a vertical eye movement signal. The vertical eye movement signal can be coupled to the analog filter 1506A to further enhance the vertical eye movement signal by removing noise and other unwanted signals. The filtered vertical eye movement signal, ie, an analog signal, may be connected to the A/D converter 1508A to convert the analog form to a digital form signal. The digitally filtered vertical eye movement signal may be connected to a first parallel digital input of the digital signal processor 1510. In other embodiments, the signal processor may be manufactured to support mixed analog and digital signals. Thus, a signal processor 1510 can be included.

Similarly, the left and right voltages of the filtered electrode 1501C-1501D with or without filtering are connected to the negative and positive inputs of the differential amplifier 1504B to amplify and calculate the difference between the left and right voltages. This forms a horizontal eye movement signal. The horizontal eye movement signal may be connected to the analog filter 1506B to further enhance the horizontal eye movement signal by removing noise and other unwanted signals. The filtered horizontal eye movement signal, that is, the analog signal, is connected to the A/D converter 1508B to convert the analog form to a digital form signal. The digitally filtered horizontal eye movement signal may be connected to a second parallel digital input of the digital signal processor 1510.

The digital signal processor 1510 performs digital signal processing using both the digital filtered horizontal eye movement signal and the digital filtered vertical eye movement signal to analyze, extract features, and unique identification from the user's unconscious eye movement. Create a pattern. In this case, a unique identification pattern of unconscious eye movement is obtained directly from the user's head/face without using a video camera or scanner and analyzing the image.

16A is a diagram of an electronic device with a video camera that can be used to obtain a user's eye movement image. The electronic device 1600 of FIG. 16B may be the electronic device 200 of FIG. 2 or the electronic device 300 of FIG. 3, and the electronic device 200 of FIG. 2 and the components of the electronic device 300 of FIG. 3 It may include some or all combinations, or may include components not shown in the present disclosure. 16B is a block diagram of an electronic device 1600 with the video camera shown in FIG. 16A that can be used to obtain a user's eye movement image.

16A and 16B, the electronic device 1600 having a video camera is used to acquire an image of the eye movement of the user 1699 during a fixation process.

In FIG. 16B, the electronic device 1600 includes a processor, a microcomputer, or a display device 1607 and a video camera 1612 connected to the microprocessor 1601. The electronic device 1600 further includes a memory 1602 that stores program commands and user-related data. The electronic device 1600 includes one or more (radio frequency transmitter/receiver) radios 1609 and one or more wired connectors 1622, 1624 to provide communication between the electronic device 1600 and other devices (radio frequency transmitters/receivers). ) (For example, includes a USB port 1622 and/or a network interface port. The electronic device 1600 uses the electronic device 1600 to provide a displayable user interface UI to a user. 1607. Software control may be displayed on the touch screen display device 1607 so that the user can control the electronic device. The electronic device 1600 allows the user to further control the electronic device. One or more hardware buttons 1604 may optionally be included.

To obtain eye movement, the processor 1601 creates a fixed target 1650 displayed by the display device 1609. While video camera 1612 acquires, under the control of processor 1601, an image, video, and a time-series sequence of the user's eyes, the user may be asked to fix the gaze on target 1650. The video camera acquires unconscious eye movements of one or both eyes of a user on a frame-by-frame basis. 3D accelerometer data is acquired along with the video to determine the movement of the eye by removing the physical movement of the camera 1612 and the electronic device 1600.

The processor 1601 may provide or include a signal processor function. In any case, the processor executes the instructions of the pattern recognition software and the signal processing software to analyze the video obtained by the user's unconscious eye movement of one or both eyes. Pattern recognition software can be used to identify the eye and iris and pupil of the video.

The acquired video is analyzed as to whether the user has blinked an eye in a time-series sequence of images and whether a sequence sufficient to detect eye movement has been captured. Otherwise, the user is asked to the user interface to repeat the fixation process.

Once sufficient image sequence is obtained, further analysis is performed to determine eye movements from the images. The eye's reference point, such as the iris or pupil, is used to determine the unconscious movement of the video obtained during the fixation or staring process. The 3D accelerometer data is used to exclude the physical movement of the camera 16812 acquired in the video from the raw eye movement data to form real eye movement data.

The eye movement data is further analyzed to extract the type of unconscious eye movement data used to authenticate and/or uniquely identify the user.

The user is initialized with the electronic device 1600 to store an initial data set of unconscious eye movement data obtained initially. From the initial data set, features linked to the time series of unconscious eye movements are extracted and classified in association with the user.

Features extracted from the acquired data set of newly acquired unconscious eye movement data are compared with stored features extracted from the initially acquired unconscious movement data to identify the user. The match ratio can be calculated to determine if the user is authorized to use the electronic device.

The newly acquired unconscious eye movement data is compared with the initially acquired unconscious movement data to determine a matched result. If the match result is within the match rate, the user is identified and authorized to use the device. If the match result exceeds the match rate, the user will not know and will not have permission to use the device.

Unconscious eye movement (eg, tremor) can generate a maximum frequency of about 150 hertz (Hz). The cone-shaped diameter is 0.5 to 4 micrometers.

To capture unconscious eye movement, a suitable recording device in a system, such as a video camera 1612 connected to the processor 1601 of the electronic device 1600, has a minimum frame rate (frames per second) of at least 300 Hz-450 Hz. Or optimally, the frame rate is over 1000Hz.

Cameras and processors in existing electronic devices can be reconfigured in software applications or drivers to run faster than typical settings. Normal (large-scale intermittent motion) guarantees about 300 degrees per second, and can re-align the eyes within 1/3 second. Unconscious fine intermittent motion ensures amplitudes from a few degrees per second to below 0.2 degrees. Therefore, the spatial resolution of the recording device that can be resolved within the range of 0.1-0.2 degrees may be optimal for obtaining a desired unconscious eye movement.

To obtain the desired unconscious eye movement, one type of electronic device 1600 is shown in FIGS. 8A and 8B, but other types of electronic devices may be used to capture the unconscious eye movement.

17A-17C illustrate diagrams and targets of electronic glasses with video cameras that can be used to obtain user eye movement images.

17A and 17B, an electronic glasses 1700 that can be used to obtain an image of a user's eye movement is illustrated. From the acquired image of eye movement, a small-scale unconscious eye movement can be extracted. The electronic glasses 1700 may be temporarily worn by the user to identify, authenticate, and authenticate the user to the system or device.

The electronic glasses 1700 include an eye glass frame 1702 equipped with a left lens 1704L and a right lens 1704R on a pair of eye wires. In one embodiment, the spectacle frame 1702 includes a bridge 1707, a left side 1708L, a right side 1708R, a nose pad, a nose pad arm, and a pair of eye wires. These electronic glasses 1700 may be glasses clips that can be worn on prescription glasses. The lenses 1704L-1704R may or may not be prescription lenses.

A small target 1706 can be formed in the upper right corner of the left lens 1704L to direct the user's eye towards the video camera. Optionally, a target 1706 can be formed in the upper left corner of the right lens 1704R to direct the user's eye towards the video camera. The target 1706 may be formed by printing a target image on the surface of the lens, by engraving the target image on the lens, and illuminating the target light on the surface of the lens. Or it can be done by other known means of applying an image on a transparent surface.

Referring to FIG. 17C, the target 1706 is a short depth target with a central opening or hole 1707. Target 1706 on lens 1704L or 1704R may be too close for the user to focus. However, the user can focus through the central opening 907 of the target 1706. The target 1706 with the central opening 1707 acts like an optical tether so that the pupil is positioned in line with the video camera to better obtain eye movement.

Referring to FIG. 17B, the electronic glasses 1700 includes a power supply (eg, a video camera 1710, a processor 1712, a memory device 1714 ), a wireless transmitter/receiver 1716, and an eyeglass frame 1702. For example, a battery) 1720. The electronic glasses 1700 may further include an optional light emitting diode (LED) 1718 mounted on the frame 1702 or nose pad arm 1722.

The video camera 1710 is a lens having a target, for example, a left lens 1704L having a target 1706, a structure slightly inclined in the direction. Therefore, it can be more aligned with the eye when focusing on the target. The video camera 1710 and the processor 1712 operate together at a frame rate ranging from 300 to 1000 frames per second to obtain unconscious eye movement at a maximum frequency of 150 Hz.

The radio 1716 can be used by a processor to communicate with the computer or server to authenticate the user to the computer or server. The memory 1714 may be used to store initialization data including features of the initial unconscious eye movement extracted from the obtained eye movement.

The electronic glasses 1700 may be temporarily worn by the user to authenticate the user to the system. Optionally, electronic goggles can be used.

18A and 18B are diagrams of a Virtual Reality (VR) device with a video camera that can be used to acquire images of the user's eye movements.

18A and 18B, an electronic virtual reality headset or goggles 1800 is shown that includes a video camera that can be used to capture eye movements of a user's eye. The electronic virtual reality headset includes a frame 1802 and a head strap 1803 to hold the headset attached to the user head. The frame 1802 includes upper, lower, left and right flexible curtains or blinders 1804 configured to receive a face around the user's eyes to provide a hood area 1899 to prevent external light from entering. The lower curtain or blinder 1804B includes a nose opening 1805 for receiving a user's nose.

In FIG. 18B, the headset 1800 further includes a video camera 1810 connected to the frame 1802, a left display device 1812L, and a right display device 1812R. The headset 1800 further includes a processor 1811 and memory 1814 coupled together. The video camera 1810 and the left and right display devices 1812L and 1812R are connected to the processor 1811. The left display device 1812L and the right display device 1812R may provide a user with stereo three-dimensional images recognized at various depths. Video camera 1810 may be connected to frame 1802 in hood area 1899 at different locations to capture eye movement. The video camera 1810 can be positioned on the right side as shown to capture eye movement to avoid interference with the video images displayed by the left and right display devices 1812L, 1812R. In other embodiments, a pair of video cameras 1810 may be positioned on the opposite side to obtain a pair of left and right eye movements such that minute movements of the unconscious eye from one or both eyes are used to authenticate the user.

The stereo three-dimensional target including the left target 1806L and the right target 1806R may be generated by the processor 1811 and displayed on the left display device 1812L and the right display device 1812R, respectively. The left target 1806L and the right target 1806R can make the target look far away to focus the eyes at a long distance, and the video camera 1810 can fix the eye to better acquire unconscious eye movement.

The processor 1811 can be wired to other systems by cables 1850 and plugs 1852. Optionally, the wireless transmitter/receiver 1816 can be connected to the processor 1811 so that it can be wirelessly connected to other systems to use the authentication capabilities of the processor and headset/goggles headset/goggles 1800.

19A is a diagram of an eye including a contact lens with an emitter device to assist in obtaining data about the user's eye movement. 19B-19D are diagrams of contact lenses with various emitter devices that can be used to help users acquire data regarding eye movements. 19E is a functional block diagram of an active emitter device. 19F is a side view of a sensing device near a contact lens mounted on the eye that can be used to obtain data about the user's eye movements.

Referring to FIG. 19F, eye movement may be obtained in other ways by using a contact lens 1900 mounted on one or both eyes 900 of the user on the lens 918. The contact lens 1900 may include one or more emitters 1902 connected (eg, embedded or printed) to the lens material 1104.

The emitter 1902 can be an active device, such as a light emitting diode (LED) or a wireless beacon (e.g., radio transmitter/receiver, diode driver) associated with a driving circuit, or a reflector, retro reflector It can be a retro-reflector, or a passive device such as a mirror.

In the case of an active emitter device, one or more sensors are used that receive the emitted light or radio signals to determine the position of the eye from one point of view to the next, to directly obtain eye movement for a period of time. Power can be wirelessly connected from the base antenna around the eye to the antenna connected to the active emitter device of the contact lens. The wireless signal can also be connected between the base antenna and the antenna connected to the active integrated circuit device. A 3D motion sensor can be included as part of an integrated circuit to capture eye movements, including unconscious eye movements.

In the case of a passive emitter device, the light from the light source is directed to the passive emitter device, which is activated to reflect the light back to one or more photodiode sensors around the eye. The combination of an active emitter device and a passive emitter device can be used in the same contact lens to achieve ocular motion in either or both ways.

19B-19D show one emitter 1902A, two emitters 1902B-11\902C and four emitters 1902D-1902G embedded in the contact lens 1900A-1900C, respectively.

In FIG. 19B, active emitter 1902A is shown connected to two or more antenna lines 1905A-1905B at the perimeter edge of contact lens 1900A. Active emitter 1902A includes an integrated circuit 1906 having a processor/controller and other circuitry coupled externally or integrated internally on the integrated circuit. Optionally, emitter 1902A can be an active emitter.

In FIG. 19C, a pair of passive emitters 1902B-1902C are shown disposed around a portion of the edge of the circumference of the contact lens 1900B. Optionally, emitters 1902B-1902C can be active emitters with two or more antenna feeds 1905A-1905B at portions near the circumferential edge of contact lens 1900B.

In FIG. 19D, a pair of active emitters 1902D-1902E and a pair of passive emitters 1902F-1902G are shown near the perimeter edge of the contact lens 1900C. Optionally, all emitters 1902D-1902G can be active or passive emitters; Only the other can be passive. The other can be activated manually.

Referring to FIG. 19E, additional details of an example of an active emitter 1902 are shown. The active emitter 1902A includes an integrated circuit 1906 having a processor/controller 1910 and other circuitry internally integrated on the external or connected integrated circuit to the processor/controller 1910.

The integrated circuit 1906 may receive power to the power conversion circuit 1912 connected to the processor 1910 through the antenna 1905A-1905B. The radio frequency energy from the oscillating radio frequency (RF) signal produces more antenna feeds 1905A-1905B by the adjacent base antenna. The power conversion circuit 1912 can rectify and regulate AC RF signals with DC power to circuits connected to the IC 1906 as well as other circuits within the IC 1906.

An internal diode driver 1918 is connected between the processor 1910 and the light emitting diode (LED) 1908 by a light emitting diode (LED) 1908 connected to the integrated circuit 1906. The power-to-power conversion processor can generate a signal to activate the diode driver 1918. The processor uses power to activate the diode driver 19118 to supply power to the light emitting diode 1908 to emit light from the user's eyes.

Optionally or additionally, the integrated circuit 1906 can have a 3D motion sensor 1917 coupled to a processor that directly detects eye movement. With power, a processor connected to the wireless transmitter/receiver (transceiver) 1909 may transmit and receive wireless signals to the wireless transceiver 1919 through antenna lines 1905A-1195B. The basic unit with wireless transceiver can collect and further process eye movement data

Referring to FIG. 19F, in the case of an active emitter, a base unit 1999 including a frame 1921, a lens 1922, and a pole of the base antenna 1924A-1924B wound around the lens 1922 is illustrated. do. The base antenna 1924A-1924B is connected to a base radio receiver/transmitter (not shown) and to a base processor (not shown) that processes the obtained eye movement signal.

Induced to carry radio frequency power/energy/radio frequency signals together between the base unit 1199 and the contact lens 1900C, with a base antenna 1124A-1124B near the antenna lines 1905A-1905B of the contact lens It can be inductively coupled to.

When power is applied, eye movements obtained by the motion sensor 1917 can be transmitted from the contact lens 1900C to the base unit 1999 by each wireless transceiver.

In the case of a passive emitter, the base unit 1999 (additively or alternatively) includes one or more photodiode sensors 1934A-1934B connected near the edge of the lens 1912. Base unit 1999 may further include a light source 1932, such as a display device, to illuminate toward one or more passive emitters 1902F, 1902G. Light reflected from one or more passive emitters 1902F, 1902G is acquired by one or more photodiode sensors 1934A-1934B to determine eye position and movement over time. Wires 1914 from photodiode sensors 1934A-1134B are connected to a base processor (not shown) to process the acquired eye movement signals.

The base unit 1999 may be in the form of glasses, VR goggles, an independent eye scanner, or a wall mounted eye scanner.

The electronic device may be worn by the user's eye or a user adjacent to the eye, such as in the case of glasses, goggles, or contact lenses, but the electronic device for acquiring eye movements is a user's eye or eye near a video camera, sensor or antenna. It can be processed by a structure or system that involves unconscious eye movement while positioning.

20A to 20B show an apparatus for acquiring eye movements attached for access control of a building structure.

20A shows an apparatus for acquiring eye movement 2001 attached to a building structure 2000 to control access to one or more doors 2002 in response to a user's unconscious eye movement. The eye motion acquisition device 2001 is connected to an access system 2003 to control access to one or more doors 2002 of the structure 2000. The access system 2003 may perform one or more door 2002 unlock controls in response to the minute movements of the unconscious eye of the authorized user.

20B shows an enlarged view of an eye motion acquiring device 2001 that receives a user's face area around the left eye or right eye. As previously discussed with reference to FIG. 18B, the eye movement acquisition device 2001 includes a video camera 1810, a display device 1812R, and a memory 1814 connected to the processor, as well as some similar structures and functions of the processor 1811. It may include. The processor 1811 can be wired to the access system 2003 by cables and plugs. Optionally, the processor 1811 may be wirelessly coupled to the access system 2003 by a wireless device 1816 coupled to the processor 1811.

21A and 21B show independent eye scanners for authenticating a user to the system.

21A shows a standalone eye scanner 2101 wirelessly connected to the system 2102 with a cable connection or with each wireless transmitter/receiver. System 2102 can be coupled to server 2106. The server can be remote and can be accessed through a wide area network 2104, such as the Internet. The stand-alone eye scanner 2101 can be used to non-invasively authenticate the user to the system 2102 and the server 2106 in response to eye microscopic movements of one or both eyes of the user.

21B shows an enlarged view of a standalone eye scanner 2101 accommodating the eye area of the user's face. As previously discussed with reference to FIG. 18B, the eye motion acquisition device 2101 is coupled to a processor 1811 as well as one or more video cameras 1810, a left display device 1812L, a right display device 1812R, and a processor Memory 1814. Processor 1811 may be wired to system 2102 by cable 1850 and plug 1852. Left and/or right targets 1806L, 1806R are similarly generated on the left and/or right display devices 1812L, 1812R, so that the stand-alone eye scanner 2101 responds to the eye micro-movement of one or both eyes of the user. Users can be authenticated non-invasively.

Claims (20)

An electronic device utilizing biometrics technology,
A motion information acquisition device for capturing at least one movement information of a user including a neuro-muscular tone from a part of the user's body at a preset sampling frequency;
A processor electrically connected to the motion information acquisition device; And
A storage device electrically connected to the processor and configured to store information associated with a unique characteristic value for identifying a user,
When the storage device is executed, the processor,
Using the motion information acquisition device to obtain at least one first motion information of the user including the neuromuscular tone at a preset sampling frequency,
Suppressing a signal component associated with conscious movement (voluntary movement) from at least one first movement information of the acquired user, so as to obtain first non-voluntary movement information,
For extracting at least one measurable first unique characteristic value associated with a neuromuscular tone for identifying the user based on the obtained first unconscious motion information,
To store commands
An electronic device utilizing biometric technology.
According to claim 1,
When the storage device is executed, the processor,
Additionally storing a command to generate a first neuro-machanical fingerprint based on the first unique characteristic value
An electronic device utilizing biometric technology.
According to claim 1,
When the storage device is executed, the processor,
Further storing instructions for generating a prediction model by performing a combination of at least one learning algorithm based on the first unique characteristic value
An electronic device utilizing biometric technology.
The method of claim 3,
When the storage device is executed, the processor,
To further store an instruction to store parameters of the prediction model
An electronic device utilizing biometric technology.
According to claim 1,
When the storage device is executed, the processor,
Further storing an instruction to evaluate the performance of the prediction model
An electronic device utilizing biometric technology.
According to claim 2,
When the storage device is executed, the processor,
Using the motion information acquisition device to obtain at least one second motion information of the user including the neuromuscular tone at a preset sampling frequency,
Suppressing a signal component associated with conscious movement from at least one second movement information of the acquired user, so as to acquire second unconscious movement information,
Extracting at least one measurable second unique characteristic value associated with a neuromuscular tone for identifying the user based on the obtained second unconscious motion information,
To generate a second neuro-mechanical fingerprint based on the unique characteristic value,
To store additional commands
An electronic device utilizing biometric technology.
The method of claim 6,
When the storage device is executed, the processor,
And further storing instructions for determining a matching percentage of information associated with the first neuro-mechanical fingerprint and information associated with the second neuro-mechanical fingerprint.
An electronic device utilizing biometric technology.
The method of claim 7,
When the storage device is executed, the processor,
In response to a case where the matching rate is equal to or higher than a preset access matching rate, an additional command to store whether the user is a previously authenticated user is stored.
An electronic device utilizing biometric technology.
The method of claim 8,
When the storage device is executed, the processor,
Further storing a command to allow the authenticated user access to the electronic device (access)
An electronic device utilizing biometric technology.
According to claim 2,
When the storage device is executed, the processor,
Further storing a command to generate encryption data with an encryption algorithm using the encryption key based on the first neuro-mechanical fingerprint
An electronic device utilizing biometric technology.
The method of claim 10,
When the storage device is executed, the processor,
Using the motion information acquisition device to obtain at least one second motion information of the user including the neuromuscular tone at a preset sampling frequency,
Suppressing a signal component associated with conscious movement from at least one second movement information of the acquired user, so as to acquire second unconscious movement information,
Extracting at least one measurable second unique characteristic value associated with a neuromuscular tone for identifying the user based on the obtained second unconscious motion information,
Generating a second neuro-mechanical fingerprint based on the second unique characteristic value,
To store additional commands
An electronic device utilizing biometric technology.
The method of claim 11,
When the storage device is executed, the processor,
And further storing instructions for determining a matching ratio between information associated with the first neuro-mechanical fingerprint and information associated with the second neuro-mechanical fingerprint.
An electronic device utilizing biometric technology.
The method of claim 12,
When the storage device is executed, the processor,
If the matching ratio is equal to or higher than a preset access matching ratio, additionally storing a command for decrypting the encrypted data using the first neuro-mechanical fingerprint
An electronic device utilizing biometric technology.
The method of claim 13,
When the storage device is executed, the processor,
Further storing a command to transmit the encrypted data to the remote server
An electronic device utilizing biometric technology.
The method of claim 14,
When the storage device is executed, the processor,
Further storing a command to receive the encrypted data from the remote server
An electronic device utilizing biometric technology.
The method of claim 8,
Further comprising an access device for controlling the locking (locking) or unlocking (door unlocking) of the door (door),
When the storage device is executed, the processor,
In response to whether or not the authenticated user, to further store a command to control the door lock or unlock
An electronic device utilizing biometric technology.
A user authentication method using a user authentication device that authenticates a user by utilizing biometrics technology,
The user authentication device acquires at least one first motion information of a user including a neuro-muscular tone at a preset sampling frequency using the motion information acquisition device,
The user authentication device obtains the first non-voluntary movement information by suppressing a signal component associated with conscious movement from at least one first movement information of the acquired user. and,
And the user authentication device extracting at least one measurable first unique characteristic value associated with a neuromuscular tone for identifying the user based on the obtained first unconscious motion information.
User authentication method using biometric technology.
The method of claim 17,
The user authentication device further comprises generating a first neuro-machanical fingerprint based on the first unique characteristic value.
User authentication method using biometric technology.
The method of claim 18,
The user authentication device further comprises generating a prediction model by performing a combination of at least one learning algorithm based on the first unique characteristic value.
User authentication method using biometric technology.
The method of claim 19,
The user authentication device further comprises storing parameters of the prediction model.
User authentication method using biometric technology.

Images