EP4204996A2 - Dispositifs électroniques avec un modèle d'intelligence artificielle statique pour des situations contextuelles, comprenant un blocage de l'âge pour le démarrage de vapotage et d'allumage, à l'aide d'une analyse de données et procédés de fonctionnement associés - Google Patents

Dispositifs électroniques avec un modèle d'intelligence artificielle statique pour des situations contextuelles, comprenant un blocage de l'âge pour le démarrage de vapotage et d'allumage, à l'aide d'une analyse de données et procédés de fonctionnement associés

Info

Publication number
EP4204996A2
EP4204996A2 EP21862899.8A EP21862899A EP4204996A2 EP 4204996 A2 EP4204996 A2 EP 4204996A2 EP 21862899 A EP21862899 A EP 21862899A EP 4204996 A2 EP4204996 A2 EP 4204996A2
Authority
EP
European Patent Office
Prior art keywords
user
liveliness
independent static
predetermined
model
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
EP21862899.8A
Other languages
German (de)
English (en)
Inventor
Martin Zizi
Luke STORK
Kitae Lee
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Aerendir Mobile Inc
Original Assignee
Aerendir Mobile Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Aerendir Mobile Inc filed Critical Aerendir Mobile Inc
Publication of EP4204996A2 publication Critical patent/EP4204996A2/fr
Pending legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G06F21/32—User authentication using biometric data, e.g. fingerprints, iris scans or voiceprints
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/30—Authentication, i.e. establishing the identity or authorisation of security principals
    • G06F21/31—User authentication
    • G06F21/316—User authentication by observing the pattern of computer usage, e.g. typical user behaviour
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F21/00—Security arrangements for protecting computers, components thereof, programs or data against unauthorised activity
    • G06F21/50—Monitoring users, programs or devices to maintain the integrity of platforms, e.g. of processors, firmware or operating systems
    • G06F21/55—Detecting local intrusion or implementing counter-measures
    • G06F21/554—Detecting local intrusion or implementing counter-measures involving event detection and direct action

Definitions

  • the disclosure generally relates to a static artificial intelligence (Al) model for contextual situations using data analysis.
  • FIG. 1 is a classification chart of biometric modalities.
  • FIG. 2 is a block diagram of overall flow processing of situational static models for artificial intelligence.
  • FIG. 3 is a table of examples of various types of motion classifications in body parts of a human being.
  • FIG. 4 is a block diagram illustrating the operating environment (a system) of electronic devices incorporating situational static models for artificial intelligence.
  • FIG. 5 is a more detailed block diagram of an electronic device used in the system shown in FIG. 4.
  • FIG. 6 is a functional block diagram of a feature processing system.
  • FIG. 7 is a flow chart of static model operation to provide artificial intelligence.
  • FIG. 8 is a functional bloc diagram of a Static Model processing system.
  • FIG. 9 is an example sensing structure (4 th dimension sensing stretching material) for the situational static model to provide artificial intelligence.
  • FIG. 10 is a functional block diagram of a sensor for the situational static model to provide artificial intelligence.
  • FIG. 11 is a plot of an acceleration waveform of a sensed acceleration signal from a body part for a single axis (X, Y, or Z) over time.
  • FIG. 12 is a flow chart for collecting movement signal data by an Input Data Handler of an electronic device.
  • FIG. 13 is a flow chart for a sleep mode operation of an electronic device with the situational static model to provide artificial intelligence with low power consumption.
  • FIG. 14 is a flow chart of secure mode operation of an electronic device with the situational static model to provide secure data analysis with a secure core of a multiprocessor.
  • FIG. 15 is a Block Diagram of the Contextual Situations Feature Extractor
  • FIG. 16 is a chart of various types/format of sensor data over time that may be used with the situational static model.
  • FIG. 17 is a flow chart of a preprocessing operation for the Contextual Situations Feature Extractor of FIG. 15.
  • FIG. 18 is a plot of data samples for a single axis Accelerometer over a time series.
  • FIG. 19 is a flow chart of feature extracting operation for generating a feature vector set.
  • FIG. 20 is an example table representing a feature vector set for various feature vectors providing a time series analysis over time.
  • FIG. 21 is a table indicating various features vectors that may be used to obtain a feature vector set for a number of different physiological states.
  • FIG. 22 is a chart of plots to show divergence Feature Distribution to recognize a non-human (e.g., a bot, an animal) signal distinguished from human signals.
  • a non-human e.g., a bot, an animal
  • FIG. 23A is a chart of a first plot of a time series of signal data for entropy feature analysis.
  • FIG. 23B is a chart of a second plot of a time series of signal data for entropy feature analysis with FIG. 23A.
  • FIG. 24 is a chart of a pair of gyroscope X-axis data plots of a low mean band and a high mean band of body motion data to determine blood glucose level for a blood glucose context for the static model of artificial intelligence.
  • FIG. 25 is a table of data sets that may be used to detect various physiological states of a human user with the static model of artificial intelligence.
  • FIG. 26 is a table of data sets that may be used to detect various physical characteristic of a human user with the static model of artificial intelligence.
  • FIG. 27 is a functional block diagram of a static model analyzer shown in FIG. 8.
  • FIG. 28 is a flow chart of training mode operation for the static model analyzer to obtain a time series of data of a data set.
  • FIG. 29 is a flow chart of an operational (inference) mode of the static Al model associated with last steps shown in FIG. 7.
  • FIG. 30 is a functional block diagram of a static Al model framework for a software implementation of the static Al model.
  • FIG. 31 is diagram showing various device types in which the static Al model may be used, including software, field programmable gate array FPGA, and a customized application specific integrated circuit (ASIC).
  • software including software, field programmable gate array FPGA, and a customized application specific integrated circuit (ASIC).
  • ASIC application specific integrated circuit
  • FIG. 32 is a static model processing system of an electronic device that can be used to detect various states of a contextual situation, such as a physiological state of liveliness, age, or gender.
  • FIG. 33 is a flow diagram between client (e.g., smart phone) and server for creating a new user account, associated with the server, using liveliness physiological states determined by a static Al model.
  • client e.g., smart phone
  • server for creating a new user account, associated with the server, using liveliness physiological states determined by a static Al model.
  • FIG. 34 is a flow diagram between client (e.g., smart phone) and server for accessing personal health records from the server using liveliness physiological states determined by a static Al model in conjunction with other authentication parameters.
  • client e.g., smart phone
  • server for accessing personal health records from the server using liveliness physiological states determined by a static Al model in conjunction with other authentication parameters.
  • FIG. 35A illustrates a flow chart of using static artificial intelligence models to pass/fail implementation of age-blocking (parental control).
  • FIG. 35B illustrates a client server implementation of the age blocking and processes in each.
  • FIG. 36 illustrates a flow chart of age based application of artificial intelligence based on neurological information to control vehicle ignition or starting.
  • FIG. 37 is a flow chart illustrating additional Al to provide side-effects protection from Nicotine abuse.
  • FIG. 38A is a diagram of the implementation forms that Al technology based on neurology can be embedded in phones, or any hardware (HW)Zsoftware (SW) system on chip (SOC) device to form a parental control device.
  • HW hardware
  • SW software
  • SOC system on chip
  • FIG. 38B illustrates a vaporizer with a system on a chip and the artificial intelligence to provide age blocking.
  • FIG. 39 illustrates plot of a 3-D vector space of three different features extracted from neural- tagging data.
  • Embodiments in accordance with the disclosure may be implemented as an apparatus, method, server-client apparatus and/or method, cooperation of apparatus and/or method, chipset, computer program or any combination thereof. Accordingly, the embodiments may take the form of an entirely hardware embodiment (including chipset), an entirely software embodiment (including firmware, any type of software, etc.) or an embodiment combining software and hardware. Software and hardware aspects that may all generally be referred to herein as a "module”, “unit”, “component”, “block”, “element”, “member”, “system”, “subsystem” or etc. Furthermore, the embodiments may take the form of a computer program product embodied in any tangible medium of expression (including a computer file) having computer-usable program code embodied in the medium.
  • first means “first”, “second” or etc.
  • these elements do not be limited by these terms. These terms may be used to distinguish one element from another and may be irrelevant to the order or importance of elements.
  • a first sensor could be termed a second sensor, and, similarly, a second sensor could be termed a first sensor.
  • the first sensor and the second sensor are both sensors, but they may not be the same sensor.
  • the term “and/or” as used herein may cover any and all possible combinations of one or more of the associated listed items.
  • a or B may represent all of “including at least one A”, “including at least one B", or “including both at least one A and at least one B".
  • first element when a first element is “connected to”, “coupled to” or “coupled with” a second element, the first element may be directly “connected to”, directly “coupled to” or directly “coupled with” the second element or at least one or more of other elements may be interposed between the first element and the second element.
  • first element when a first element is “directly connected” or “directly coupled” to a second element, another element is not interposed between the first element and the second element.
  • an electronic device can acquire information about the user's contextual situations, there could be various useful types of applications that can be developed.
  • Information regarding the user's contextual situations can also be collected from various sources.
  • the sources may include but not limited to sensor data, user data on the internet, data set, and so on.
  • One of the possible applications based on situational awareness can be about data analysis of the signals from the human body by Al technology.
  • the human body is one of the well-known complex system composed of many components which may interact with each other. It has inherent rich and near infinite variations both at the molecular and at the functional levels, but it also dense and rich with information in its broadest sense.
  • any human contextual situations e.g., physiological state
  • a neuro-muscular tone that is affected by these contextual situations e.g., physiological states
  • some type of these states coming from the human body can be well interpreted by appropriately analyzing the neuro-muscular tone signals that are collected by various types of sensors from the body part of users.
  • these results can be used in electronic devices, a lot of useful applications can be possible, for example, providing visual information to users about their physiological state, improving the function of current biometric applications, securing personal information with more high-security levels, or answering simple binary questions like gender, age and so on.
  • the electronic device may be a hand held type of portable device, a smart phone, a tablet computer, a mobile phone, a telephone, an e-book reader, navigation device, a desktop computer, a laptop computer, a workstation computer, a server computer, a single board computer, a camera, a camcorder, an electronic pen, wireless communication equipment, access point(AP), a drone, a projector, an electronic board, a photo copy machine, a watch, a glasses, a headmounted device, a wireless headset/earphone, an electronic clothing, various type of wearable devices, a television, a DVD player, an audio player, a digital multimedia player, an electronic photo frame, a set top box, a TV box, a game player, remote controller, bank ATM, payment system device (including POS, card reader), a refrigerator, an oven, a microwave oven, an air conditioner, a vacuum cleaner, a washing machine, a dishwasher, an air cleaner, a home automation control device, a smart
  • the electronic device may be a combination or a part of one or more of the aforementioned devices.
  • the electronic device may be a part of furniture, building, structure or machine (including vehicle, car, airplane or ship) or a type of an embedded board, a chipset, computer files or some type of sensors.
  • the electronic device of the disclosure is not limited to the aforementioned devices and may be a new form of an electronic device as technology development advances.
  • FIG. 1 shows classification of biometric modalities that is adapted from " UNAR J A, SENG W C, ABBASI A. A review of biometric technology along with trends and prospects. Pattern Recognition, 2014, 47(8):2673-2688". Measurements and calculations related to human characteristics are often termed as "biometrics”. Although there can be various applications and several advantages using these traditional methods when biometrics is used in the physiological state application, known biometrics is unlikely to offer a highly robust security solution in some aspects.
  • a physiological biometric solution disclosed herein using the Al static model for contextual situations (e.g., physiological states) of the human body related to neuro-muscular tone sensing can offer better improved, effective, solid, and heightened solutions for the physiological state applications including liveliness, identification, authentication, or encryption, or the like.
  • the position of the neuro-muscular tone sensing technology relative to the rest of biometrics is also illustrated in FIG.1 . Contrary to the rest of the field, the neuromuscular tone sensing technology is a live physiologic signal, being never the same and yet allowing on to be recognized.
  • EEG Electroencephalography
  • ECG Electrocardiogram
  • EMG Electrocardiogram
  • EKG Electrocardiogram
  • Behavioral biometric methods are linked to what the user does or his/her habits.
  • Known anatomical biometric methods are linked to physical features of the user, such as fingerprints, iris eye scans, veins, facial scans, and DNA.
  • Certain user motions are habitual or part of a user’s motion repertoire.
  • a user signing a document for example, is a contextual motion that a user develops with behavioral habits.
  • the motions usually analyzed of a signed signature are the macro-motions or large- scale motions that a user makes with a writing instrument. Most of these actions are voluntary movements because they are motions according to the consciousness or intention of the user. For example, from the large motions of a signed signature one may determine with one's eyes whether the writer was left-handed or right-handed.
  • micro-motions very small motions
  • a user makes when signing, making other motions, or simply at rest making no motion.
  • These micro-motions can contain neuro-derived, neuro-based, or neuro-muscular tone and it is invisible to the eyes. Therefore, it belongs to involuntary movement rather than consciousness or intention of the user.
  • These micro-motions of a user are due to the unique neuro-muscular anatomy of each human being and may also include very important signals referred to herein as neuro-derived micro-motions or neuro-muscular tone.
  • These signals of micro-motions are also linked to the motor control processes from the motor cortex of an individual down to his/her hands.
  • motion signals and “micro-motions signals”) can be captured that include the neuro-derived micro-motions of a user.
  • micromotion electronic signals that represent the micro-motions of the user within the motion signals.
  • the resulting data can yield stable physiological states of the users representing liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • physiological states are interpreted or treated as unwritten signatures, these physiological states for unique identifiers derived from the user's neuro-muscular tones are a user’s neuro-mechanical fingerprints.
  • Neuro-mechanical fingerprints may also be referred to herein as NeuroFingerprint (NFP) or Neuro-Print (NP).
  • Micro-motions of a user are linked to the cortical and subcortical control of the motor activities in the brain or elsewhere in the nervous system of a human body.
  • the specific musculoskeletal anatomy of an individual can affect the micro-motions of a user and contribute to the motion signals that include the micro-motions of a user.
  • the signal thus contributed is a signal of movement of the muscles by the nerve signal, which can be called neuro-muscular tone.
  • the motion signals captured from a user can also reflect part of the proprioceptive control loops that include the brain and proprioceptors that are present in a user’s human body.
  • an electronic device may be used with a neurological algorithm to better emulate a human cognitive interface in a machine. This can improve man-machine interfaces. For example, consider a human cognitive interface between a husband and wife or closely-knit persons. When a husband touches his wife on the arm, the wife can often times recognize that it is her husband touching her just from the feel of that touch, because she is familiar with his touch. If the touch feels unique, a human can often recognize what it is that is touching him/her just from that unique feel.
  • the neuro-muscular tone signals are extracted in response to micro-motions that are related to a type or form of tremor.
  • a tremor is an unintentional, rhythmic muscle movement that causes an oscillation in one or more parts of a human body. Tremors may be visible or invisible to the unaided eye. Visible tremors are more common in middle aged and older persons. Visible tremors are sometimes considered to be a disorder in a part of the brain that controls one or more muscles throughout the body, or in particular areas, such as the hands and/or fingers.
  • tremors occur in the hands.
  • a tremor with micro-motions can be sensed when holding a device with an accelerometer or through a finger touching a touchpad sensor.
  • tremors There are different types of tremors. The most common form or type of tremor occurs in healthy individuals. Much of the time, a healthy individual does not notice this type of tremor because the motion is so small and may occur when performing other motions. The micro-motions of interest that are related to a type of tremor are so small that they are not visible to the unaided eye.
  • a tremor may be activated under various conditions (resting, postural, kinetic) and can be often classified as a resting tremor, an action tremor, a postural tremor, or a kinetic or intention tremor.
  • a resting tremor is one that occurs when the affected body part is not active but is supported against gravity.
  • An action tremor is one that is due to voluntary muscle activation, and includes numerous tremor types including a postural tremor, a kinetic or intention tremor, and a task-specific tremor.
  • a postural tremor is linked to support the body part against gravity (like extending an arm away from the body).
  • a kinetic or intention tremor is linked to both goal-directed and non-goal-directed movements.
  • a kinetic tremor is the motion of a moving a finger to one's nose, often used for detecting a driver for driving under the influence of alcohol.
  • Another example of a kinetic tremor is the motion of lifting a glass of water from a table.
  • a task-specific tremor occurs during very specific motions such as when writing on paper with a pen or pencil.
  • Tremors whether visible or not to the eyes, are thought to originate in some pool of oscillating neurons within the nervous system, some brain structures, some sensory reflex mechanisms, and/or some neuro-mechanical couplings and resonances.
  • tremors such as physiological, essential, orthostatic, and enhanced physiological tremors can occur under normal health conditions. These tremors are not pathologies per se. Accordingly, they are often present in the population as a whole. Physiological tremors, as well as others that are common to all users, are of interest because they generate micro-motions at frequencies over a range between 3 to 30 Hz, or 4 to 30 Hz. They may be activated when muscles are used to support body parts against the force of gravity. Accordingly, holding an electronic device in one's hand to support the hand and arm against gravity can generate physiological tremors that can be sensed by an accelerometer. Touching a touchpad of an electronic device with the finger of a hand and supporting it against gravity, can generate physiological tremors that can be readily sensed by a finger touchpad sensor.
  • Essential tremors of a kinetic type may occur and be sensed when a user has to enter a PIN or login ID to gain access to a device or a phone.
  • the frequency range of essential tremors can be between 4 to 12 Hz that could be reduced to a frequency range of 8 to 12 Hz to avoid sensing for tremors that are due to uncommon pathological conditions.
  • the physiological tremor or the enhanced physiological tremor, idem with larger amplitudes
  • the coherence of different body sides is low. That is, a physiological tremor on the left body side is not very coherent to a physiological tremor on the right body side.
  • the Al Static Model system for contextual situations (e.g., physiological states )will require a user to be consistent in using the same side hand or finger for authentication; or alternatively, multiple authorized user calibration parameter sets, one for each hand or one for each finger that will be used to extract neuro-muscular tone signals.
  • Neuro-muscular junction signals contain much more information than just user-specific invariants. They also contain situation-specific information that can be measured across a plurality of users. Such information may be then analyzed and modelized from raw data obtained in situation- constrained conditions. See FIG. 2 that shows overall processing flow of situational static models.
  • the raw signal captured by a finger touchpad sensor in an electronic device or by an accelerometer of a hand-held electronic device, can have a number of unwanted signal frequencies in it. Accordingly, a type of filtration having a response to filter out signals outside the desired frequency range can be used to obtain a micro-motions signal from the raw electronic signal.
  • an isolation/extraction means for signals in the desired frequency range may be used to obtain a micromotions signal from the raw electronic signal.
  • a finite impulse response band-pass filter e.g., the passband of 8 to 30 HZ
  • a low-pass filter e.g., 30 Hz cutoff
  • a high-pass filter e.g., 8 Hz cutoff
  • a high-pass filter e.g., 8 Hz cutoff
  • a low-pass filter e.g., 30 Hz cutoff
  • FIG. 3 shows one example of various types of motion classification according to some embodiments.
  • This exemplary classification table provides a better understanding of what kinds of characteristics should be considered and measured to extract or filter from the user's acquired motion signal to obtain feature data related to the neuro-muscular tone signals.
  • FIG. 4 is a block diagram of electronic devices illustrating exemplary operating environment 400 in accordance with some embodiments.
  • the electronic device 401 may include a processing unit 410, a sensor 420, an input/output interface 430, a display 440, a Static Model Accelerator 450, a memory 460, a power system 470, a communication interface 480 and so on.
  • the electronic devices 401, 402, 403, 404, 405 may communicate with each other and be connected through a network 406 or the communication interface 480.
  • the electronic devices 401 , 402, 403, 404, 405 can include more or fewer components than shown in FIG. 4, two or more components can be combined together, or a certain part of components can be mixed together differently in FIG. 4.
  • the various components shown in FIG. 4 can be implemented in hardware, software, or a combination of hardware and software.
  • the processing unit 410 may include at least one central processing unit and the central processing unit may include at least one processing cores.
  • the processing unit 410 may further include at least one or more of co-processors, communication processors, digital signal processing cores, graphics processing cores, low-power sensor control processors, special purpose controller and so on.
  • various hierarchical internal volatile and nonvolatile memories can be included to perform functions such as an initial booting procedure, an operation for communicating with an external electronic device, an operation for downloading an initial booting or loader related program from an external electronic device, an interrupt operation, an operation for improving performance of an electronic device in a runtime operation of program and so on.
  • the processing unit can load program instructions from a memory, a communication module or external sources, decode the instructions, execute an operation or a data processing, store result according to the decoded instructions, or perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the term processing unit may be often called, by those of ordinary skill in the art, as a processor, an application processor (AP), a central processing unit (CPU), an MCU (Micro Controller Unit), a controller and so on.
  • the sensor 420 can sense or measure the state or physical quantity of the electronic device and convert it into an electric signal.
  • the sensor 420 may include an optical sensor, an RGB sensor, an IR sensor, a UV sensor, a fingerprint sensor, a proximity sensor, a compass, an accelerometer sensor, a gyro sensor, a barometer, a grip sensor, a magnetic sensor, an iris sensor, a GSR(Galvanic Skin Response) sensor, an EEG(Electroencephalography) sensor, an ECG (Electrocardiogram) sensor, an EMG (Electromyography) sensor, an EKG (Electrocardiogram) sensor, external/internal electrode and so on.
  • the sensor 420 may collect signals (e.g., motion signals, neuro-muscular tone, etc.) from a part of the user's body and transmit them to at least one component of the electronic device 401 including the processing unit 410 or the Static Model Accelerator 450 and then may perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • signals e.g., motion signals, neuro-muscular tone, etc.
  • the sensor 420 may collect signals (e.g., motion signals, neuro-muscular tone, etc.) from a part of the user's body and transmit them to at least one component of the electronic device 401 including the processing unit 410 or the Static Model Accelerator 450 and then may perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the input/output interface 430 may include an input interface and an output interface.
  • the input interface receives input from a user or an external device of the electronic device 401 in the form of input including signals and/or instructions and transfers the input to the component of the electronic device.
  • the output interface transfers an output signal through the components of the electronic device 401 or to the user.
  • the input/output interface may include an input button, an LED, a vibration motor, various serial interfaces (e.g., USB(Universal Serial Bus), UART(Universal asynchronous receiver/transmitter), HDMI (High Definition Multimedia Interface), MHL(Mobile High- definition Link), lrDA(lnfra-red Data Association), or etc.) and so on.
  • the display 440 can display various contents such as images, texts, or videos to the user.
  • the display 440 may be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a hologram output device and so on.
  • the display 440 may include a display driver IC (DDI) or a display panel.
  • the display driver IC can transmit an image driving signal corresponding to the image information received from the processing unit 410 to a display panel, the image can be displayed according to the predetermined frame rate.
  • the display driver IC may be implemented in an IC type and may include components such as a video memory capable of storing image information, an image processing unit, a display timing controller, a multiplexer and so on.
  • the display 440 may include an input device such as a touch recognition panel, an electronic pen input panel, a fingerprint sensor, a pressure sensor and so on, or an output device such as a haptic feedback component. According to the specification of the electronic device 401, the display 440 may not be selectively included or may include at least one light emitting diode in a very simple form factor.
  • the display 440 may display a position at which the user contacts a part of the user's body, a status indicator that describes acquisition start status, processing status, or completion status of gathering signals (e.g., motion signals, neuro-muscular tone, etc.) and by doing this, it makes the electronic device perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the memory 460 may include at least one of volatile memory 462 (e.g., DRAM(Dynamic RAM), SRAM(Static RAM), SDRAM(Synchronous Dynamic RAM) ) and non-volatile memory 464 (e.g., NOR flash memory, NAND flash memory, EPROM(Erasable and Programmable ROM), EEPROM(Electrically Erasable and Programmable ROM), HDD(Hard Disk Drive), SSD(Solid State Drive), SD(Secure Digital) Card memory, Micro SD Card memory, MMC(Multimedia Card) ).
  • volatile memory 462 e.g., DRAM(Dynamic RAM), SRAM(Static RAM), SDRAM(Synchronous Dynamic RAM)
  • non-volatile memory 464 e.g., NOR flash memory, NAND flash memory, EPROM(Erasable and Programmable ROM), EEPROM(Electrically Erasable and Programmable ROM), HDD(Hard Disk Drive
  • At least one or more of boot loaders, an operating system 491 , a communication function 492 library, a device driver 493, a Static Model library for Contextual Situations 494, an application 495, or user data 496 can be stored in the non-volatile memory 464.
  • the processing unit 410 can load programs or data stored in the nonvolatile memory into the volatile memory 462. By interfacing with the processing unit 410 during operation of the electronic device, the volatile memory 462 can play a role of main memory in the electronic device.
  • the power system 470 may serve to supply, control and manage power to the electronic device 401.
  • the power system may include a PMIC (Power Management Integrated Circuit), a battery 472, a charging IC, a fuel gauge and so on.
  • the power system can receive AC or DC power as a power source.
  • the power system 470 can provide wired and wireless charging functions to charge the supplied power to the battery 472.
  • the wireless communication interface 480 may include, for example, cellular communication, Wi-Fi communication, Bluetooth, GPS, RFID, NFC and so on and may further include an RF circuitry unit for wireless communication.
  • the RF circuitry unit may include an RF transceiver, a PAM (Power Amp Module), a frequency filter, an LNA (Low Noise Amplifier), an antenna and so on.
  • FIG. 5 is a block diagram of an exemplary electronic device in accordance with some embodiments.
  • the electronic device 500 may include a processing unit 501 , a camera 550, an input/output interface 553, a haptic feedback controller 554, a display 555, a near field communication 556, an external memory slot 557, a sensor 570, a memory 590, a power system 558, a clock source 561, an audio circuitry 562, a SIM card 563, a wireless communication processor 564, a RF circuitry 565, a Neuro-Print (NP) accelerator 566 and so on.
  • a processing unit 501 may include a processing unit 501 , a camera 550, an input/output interface 553, a haptic feedback controller 554, a display 555, a near field communication 556, an external memory slot 557, a sensor 570, a memory 590, a power system 558, a clock source 561, an audio circuitry 562, a SIM card 563, a
  • the electronic device is merely one example of the embodiment of the disclosure.
  • the electronic device optionally may have more or fewer components than shown, optionally may combine two or more components, or optionally may have a different arrangement or configuration of the components.
  • the various components shown in FIG. 5 may be implemented in hardware, software or a combination of both hardware and software.
  • the processing unit 501 may include at least one central processing unit 502 and the central processing unit may include at least one processing core.
  • the processing unit 501 may further include at least one or more of co-processors, communication processors, digital signal processing cores, graphics processing cores, low-power sensor control processors, special purpose controller and so on.
  • the processing unit 501 may be implemented as an SoC (System On Chip) including various components in the form of a semiconductor chip.
  • SoC System On Chip
  • the processing unit 501 may comprise a graphics processing unit (GPU) 520, a digital signal processor (DSP) 521, an interrupt controller 522, a camera interface 523, a clock controller 524, a display interface 525, a sensor core 526, a location controller 527, a security accelerator 528, a multimedia interface 529, a memory controller 530, a peripherals interface 531 , a communication/connectivity 532, an internal memory 540 and so on.
  • GPU graphics processing unit
  • DSP digital signal processor
  • various hierarchical internal volatile and nonvolatile memories can be included to perform functions such as an initial booting procedure, an operation for communicating with an external electronic device, an operation for downloading an initial booting or loader related program from an external electronic device, an interrupt operation, or an operation for improving performance of an electronic device in a runtime operation of program and so on.
  • the processing unit can load program instructions from a memory 590, a communication/connectivity 532, or wireless communication processor 564, can decode the instructions, execute an operation or a data processing, store result according to the decoded instructions, or perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the term processing unit may be often called, by a person having ordinary skill in the art, as a processor, an application processor (AP), a central processing unit (CPU), a micro controller unit (MCU), a controller and so on.
  • the central processing unit 502 may include at least one processor core 504, 505, 506.
  • the central processing unit 502 may include a processor core having relatively low power consumption, a processor core having high power consumption with high performance and at least one or more core clusters including multiple cores, for example, a first cluster 503 or a second cluster 514.
  • This structure is a technique that is used to improve the performance of the electronic device and the power consumption gain by allocating the core dynamically in consideration of the calculation amount and the consumed current in the multi core environment.
  • Processor cores may be equipped with techniques to enhance security.
  • ARM processors one of the well-known mobile processors, have implemented this type of security technology on their processors called TRUSTZONE.
  • the first core 504 which is one physical processor core, can operate both in the normal mode 507 and the security mode 508.
  • the processor's registers and interrupt processing mechanism can be operated separately so that access to resources (e.g., peripherals or memory areas) requiring security is allowed to access only in a secure mode.
  • the monitor mode 513 may enable the mode switching between the normal mode 507 and the security mode 508.
  • the mode In the normal mode 507, the mode can be switched to the security mode 508 through a certain instruction or interrupt.
  • the applications executed in the normal mode 507 and the security mode 508 are isolated between each other so that they cannot affect the applications executed in the respective modes, thereby allowing applications requiring high reliability to be executed in the security mode 508, consequently, the reliability of the system can be enhanced. It is possible to increase security by making it possible to execute a part of the operations in performing static model processing for contextual situations including physiological states of the users in the security mode 508.
  • the camera 550 may include a lens for acquiring an image, an optical sensor, an image signal processor (ISP) and so on and may acquire still images and moving images. And the camera 550 may include a plurality of cameras (e.g., the first camera 551 , the second camera 552) to provide various functions associated with enhanced camera function.
  • ISP image signal processor
  • the input/output interface 553 may include an input interface and an output interface.
  • the input interface receives input from a user or an external device of the electronic device 500 in the form of input including signals and/or instructions and transfers the input to the component of the electronic device.
  • the output interface transfers an output signal through the components of the electronic device 500 or to the user.
  • the input/output interface may include an input button, an LED, a vibration motor, various serial interfaces (e.g., universal serial bus (USB), universal asynchronous receiver/transmitter (UART), high definition multimedia interface (HDMI), mobile high-definition link (MHL), infra-red data association (IrDA)), or other known interfaces.
  • USB universal serial bus
  • UART universal asynchronous receiver/transmitter
  • HDMI high definition multimedia interface
  • MHL mobile high-definition link
  • IrDA infra-red data association
  • the haptic feedback controller 554 may include a vibration motor, which is usually called as an actuator, in order to provide the user with the ability to feel a certain sensation through a tactile sense.
  • the display (touch sensitive display) 555 can display various contents such as images, texts, and videos to the user.
  • the display 555 may be a liquid crystal display (LCD), an organic light emitting diode (OLED) display, a hologram output device and so on.
  • the display 555 may include a display driver IC (DDI) or a display panel.
  • the display driver IC can transmit an image driving signal corresponding to the image information received from the processing unit 501 to a display panel, the image can be displayed according to the predetermined frame rate.
  • the display driver IC may be implemented in an IC type and may include components such as a video memory capable of storing image information, an image processing unit, a display timing controller, a multiplexer and so on.
  • the display 555 may include an input device such as a touch recognition panel, an electronic pen input panel, a fingerprint sensor, a pressure sensor and so on, or an output device such as a haptic feedback component. According to the specification of the electronic device 500, the display 555 may not be selectively included or may include at least one light emitting diode in a very simple form factor.
  • the display 555 may display a position at which the user contacts a part of the user's body, a status indicator that describes acquisition start status, processing status, or completion status of gathering motion signals and by doing this, it makes the electronic device perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the near field wireless communication 556 may be implemented in a wireless communication system in order to perform communication with other electronic devices in close proximity such as Near Field Communication (NFC), Radio Frequency Identification (RFID), MST (Magnetic Secure Transmission) and so on.
  • NFC Near Field Communication
  • RFID Radio Frequency Identification
  • MST Magnetic Secure Transmission
  • the external memory slot 557 may include an interface capable of mounting a memory card (e.g., an SD card, a Micro SD card and so on) so as to expand the storage space of the electronic device 500.
  • a memory card e.g., an SD card, a Micro SD card and so on
  • the power system 558 may serve to supply, control and manage power to the electronic device 500.
  • the power system may include a PMIC (Power Management Integrated Circuit), a battery 559, a charging IC 560, a fuel gauge and so on.
  • the power system can receive AC or DC power as a power source.
  • the power system 558 can provide wired and wireless charging functions to charge the supplied power to the battery 559.
  • the clock source 561 may include at least one of the system clock oscillators serving as a reference for the operation of the electronic device 500 and a frequency oscillator for transmitting and receiving an RF signal.
  • the audio circuit 562 may include an audio input unit (e.g., a microphone), an audio output unit (receiver, speaker, etc.) and/or a codec that performs a conversion between the audio signal and the electrical signal, thereby providing an interface between the user and the electronic devices.
  • the audio signal can be obtained through the audio input unit may be converted into an analog electric signal and then can be sampled or digitized to be transmitted to another component (e.g., a processing unit) in the electronic device 500 to perform audio signal processing.
  • the digital audio data transmitted from other components in the electronic device 500 may be converted into analog electric signals to generate audio signals through the audio output unit.
  • the SIM card 563 is an IC card that implements a subscriber identification module for identifying a subscriber in cellular communication.
  • a SIM card is mounted in a slot provided in the electronic device 510 and may be implemented in the form of an embedded SIM coupled to an electronic device, depending on the type of the electronic device.
  • Each SIM card may have its own unique number and the unique number may include a fixed number ICCI (Integrated Circuit Identifier) and IMSI (International Mobile Subscriber Identity) information that varies from one subscriber line to another.
  • ICCI Integrated Circuit Identifier
  • IMSI International Mobile Subscriber Identity
  • Wireless communication processor 564 may include, for example, cellular communication, WiFi communication, Bluetooth, GPS and so on. Through the wireless communication processor 564, static model processing for contextual situations including physiological states of the users can be performed over the network in cooperation with at least one or more other electronic devices (including the server).
  • the RF circuit 565 may include a transceiver, a PAM (power amp module), a frequency filter, an LNA (low noise amplifier), an antenna and so on. It is possible to exchange control information and user data with the wireless communication processor and the processing unit to perform transmission and reception through a radio frequency in a wireless environment.
  • PAM power amp module
  • LNA low noise amplifier
  • the Static Model Accelerator 566 may be used to increase the speed of performing computations that process signals obtained from a part of the user's body, or to increase the performance of the entire system by performing computations or some part of computations that are required to execute static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the sensor 570 can sense or measure the state, physical quantity, etc. of the electronic device and convert it into an electric signal.
  • the sensor 570 may include a compass 571 , an optical sensor 572, a fingerprint sensor 573, a proximity sensor 574, a gyro sensor 575, an RGB sensor 576, a barometer 578, a UV sensor 579, a grip sensor 580, a magnetic sensor 581 , an accelerometer 582, an iris sensor 583 and so on.
  • the sensor 570 can collect motion signals from portions of the user's body and transmits them to at least one component of the electronic device 500, including the processing unit 501 , the Physiological States Accelerator 566 and can perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the memory 590 may include at least one of volatile memories 591 (e.g., DRAM(Dynamic RAM), SRAM(Static RAM), SDRAM(Synchronous Dynamic RAM) ) and non-volatile memory 592 (e.g., NOR flash memory, NAND flash memory, EPROM(Erasable and Programmable ROM), EEPROM(Electrically Erasable and Programmable ROM), HDD(Hard Disk Drive), SSD(Solid State Drive), SD(Secure Digital) Card memory, Micro SD Card memory, MMC(Multimedia Card) ).
  • volatile memories 591 e.g., DRAM(Dynamic RAM), SRAM(Static RAM), SDRAM(Synchronous Dynamic RAM)
  • non-volatile memory 592 e.g., NOR flash memory, NAND flash memory, EPROM(Erasable and Programmable ROM), EEPROM(Electrically Erasable and Programmable ROM), HDD(Hard Disk Drive
  • At least one or more of boot loaders, an operating system 593, a communication function 594 library, a device driver 595, a Static Model Library for Contextual Situations 596, an application 597, or user data 598 can be stored in the non-volatile memory 592.
  • the volatile memory 591 starts operating.
  • the processing unit 501 can load programs or data stored in the non-volatile memory into the volatile memory 591.
  • the volatile memory 591 can play a role of main memory in the electronic device.
  • the electronic device 500 may acquire a signal from the portion of the user's body via the sensor 570 and provide the acquired signal to at least one of the processing unit 501 , Static Model Accelerator 566 and/or memory 590, through the interaction between these components, it is possible to perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the static model processing for contextual situations including physiological states of the users may be performed independently by the electronic device 500 and may be performed over the network in cooperation with at least one or more other electronic devices.
  • FIG. 6 depicts a block diagram of an embodiment of a Feature processing system 600.
  • the Feature processing system 600 can perform the static model processing for contextual situations including physiological states of the users.
  • the Feature processing system 600 can be implemented in the electronic device 500 in FIG. 5 or the electronic device 401 in FIG. 4 and additional hardware components or software modules can be used.
  • the Feature processing system 600 can be configured in combination with at least one or more examples of the various embodiments described herein of each of the functions of FIG. 6.
  • the Feature processing system 600 may include an Input Data Handler 602, a Feature Extractor 604, a Feature Analyzer 606, and a Feature Application Framework 608.
  • the Feature processing system 600 may be implemented in hardware, implemented in software, or implemented in a combination of hardware and software.
  • the Input Data Handler 602 may include various types of sensors including an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, an electroencephalography (EGE), an electrocardiogram (ECG), an electromyography (EMG), a galvanic skin response (GSR), and the like.
  • Image information data may be obtained from a camera, and data may be collected and processed in the form of a computer file.
  • the Feature Extractor 604 receives certain data from the Input Data Handler 602, performs preprocessing to remove unwanted signals or performs a specific process for processing efficiency, and perform extraction of numerical feature data representing a characteristic of the observed data.
  • the Feature Analyzer 606 analyzes the feature data based on the characteristic feature data extracted by the Feature Extractor 604.
  • the feature data acquired from the Feature Extractor can be used, data in the form of computer files already collected through other paths can be analyzed, and the combination of these data can be analyzed.
  • the Feature Analyzer 606 can derive information associated with the analyzed feature data and store this derived information. By using the information associated with the previously stored feature data, it is possible to derive an analysis result for the new input feature data.
  • the Feature Application Framework 608 can utilize the result information of the Feature Analyzer 606 to perform identification, authentication, liveliness, encryption, or a function using the same.
  • One of the contextual situations can be a physiological state of the users.
  • the physiological measurements for the users that can be derived from accelerometer and gyroscope data:
  • This static model for contextual situations including physiological states of the users can reside in the electronic device 401 or the electronic device 500, for example, in any embedded system, medical device, web application, or the like.
  • the model is static in the sense that it does not require any interaction with outside training data to function effectively or to improve its detection.
  • a static model quantifies whether a physiological state is present, the probability that a certain physiological state is present, or the degree to which a physiological state is present.
  • FIG. 7 is an example of a flow chart of static model operation for contextual situations (e.g., physiological states) of the user on the electronic device 401 or the electronic device 500.
  • the electronic device 401 , 500 can load information of a static model operation mode including constraint set of contextual situations (e.g., physiological states) of a user, configure the static model parameter set according to the static model operation mode, collecting sensor signal data containing neuromuscular tone from a body part of the user’s body with a predetermined sampling frequency over a predetermined sample period, suppress signal components associated with a voluntary movement of the user from the sensor data, generate data sets of mathematical representation regarding contextual situations (e.g., physiological states) of the user based on the static model operation mode from the sensor data suppressed signal components associated with the voluntary movement, construct a feature vector table containing multiple sets of feature vectors based on the data sets of mathematical representation, execute the static model using the feature vector table according to the static model operation mode and generate report information on the contextual situations (e.g., physiological states) based
  • the constraint sets of contextual situations can be blood glucose levels, female hormone levels, stress hormone levels, presence of drugs (alcohol, nicotine, caffeine, THC, CBD, prescription drugs, etc.), sleep deprivation, the presence of human neuro-muscular tremor or motion, the absence of human neuro-muscular tremor or motion and so on.
  • Information of a static model operation mode including the constraint sets of physiological states of a user can be stored in memory 460, 590, or be downloaded over the communication from the server computer prior to the FIG. 7 operation.
  • the user can select at least one of the constraint sets of contextual situations (e.g., physiological states) for the static model operation or an application program can automatically select the constraint sets of contextual situations (e.g., physiological states) according to the application program purpose.
  • the constraint sets of contextual situations e.g., physiological states
  • FIG. 8 depicts a block diagram of an embodiment of a Static Model processing system 800.
  • the Static Model processing system 800 can be implemented in the electronic device 500 in FIG. 5 or the electronic device 401 in FIG. 4 and additional hardware components or software modules can be used.
  • the Feature processing system 600 of FIG. 6 can also be implemented in the form of the Static Model processing system 800 that processes physiological states such as FIG. 8.
  • the Static Model processing system 800 may be configured in combination with at least one or more examples of the various embodiments described herein.
  • the Static Model processing system 800 may include an Input Data Handler 802, a Contextual Situations Feature Extractor 804, a Static Model Analyzer 806, and a Static Model Application Framework 808.
  • the Static Model processing system 800 may be implemented in hardware, implemented in software, or implemented in a combination of hardware and software.
  • the Static Model processing system 800 may be in the form of software that is executed in the electronic device 401 of FIG. 4 or the electronic device 500 of FIG. 5.
  • Some components of the Static Model processing system 800 may be implemented in the electronic device 401 or the electronic device 500 in the form of software associated with a special purpose hardware accelerator.
  • the Input Data Handler 802 can collect data from various types of sensors including an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, an electroencephalography (EEG), an electrocardiogram (ECG), an electromyography (EMG), an electrocardiogram (EKG), an external/internal electrode, a galvanic skin response (GSR), an electromagnetic sensing (EMS) and the like.
  • Image information data may be obtained from a camera, and data may be collected and processed in the form of a computer file.
  • the Contextual Situations Feature Extractor 804 receives certain data from the Input Data Handler 802, performs preprocessing to remove unwanted signals or performs a specific process for processing efficiency, and perform extraction of numerical feature data representing a characteristic of the observed data.
  • the Static Model Analyzer 806 analyzes the feature data based on the characteristic feature data extracted by the Contextual Situations Feature Extractor 804.
  • the feature data acquired from the Contextual Situations Feature Extractor can be used, data in the form of computer files already collected through other paths can be analyzed, and the combination of these data can be analyzed.
  • the Static Model Analyzer 806 can derive information associated with the analyzed feature data and store this derived information.
  • the Static Model Application Framework 808 can utilize the result information of the Static Model Analyzer 806 to perform static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on.
  • the Input Data Handler 802 can collect data from various types of sensors including an acceleration sensor, a gyro sensor, a geomagnetic sensor, an optical sensor, an electroencephalography (EEG), an electrocardiogram (ECG), an electromyography (EMG), an electrocardiogram (EKG), an external/internal electrode, a galvanic skin response (GSR), an electromagnetic sensing (EMS) and the like.
  • Image information data may be obtained from a camera, and data may be collected and processed in the form of a computer file.
  • the Input Data Handler 802 may collect a movement signal data from a body part of the user's body that can be obtained by the sensor of electronic device 500.
  • the sensor may include a sensor capable of detecting a user's movement or vibration.
  • the senor may include a compass 571 , a gyro sensor 575, an acceleration sensor 582, a geomagnetic sensor, a camera 550, an optical sensor, a touch sensor of the touch sensitive display 555, an electroencephalography (EEG), an electrocardiogram (ECG), an electromyography (EMG), an electrocardiogram (EKG), an external/internal electrode, a galvanic skin response (GSR), an electromagnetic sensing (EMS) or a combination thereof.
  • EEG electroencephalography
  • ECG electrocardiogram
  • EKG electromyography
  • GSR galvanic skin response
  • EMS electromagnetic sensing
  • the sensor can sense motions, vibrations, movement associated with neuro-muscular derived signals and the like that are generated in a portion of the user's body that is in contact with the electronic devices. Movements or micro movements associated with neuro-muscular derived signals can be sensed in the form of analog electrical signals in the sensor. For example, in the case of a sensor made using MEMS technology, the physical quantity that is changed by the force of movement generated in contact with a part of the user's body can be measured as an electrical analog signal using a method such as capacitance, piezoelectric, piezo resistive or thermal sensing.
  • FIG. 9 shows an example of the sensing structure in the sensor on the electronic device 401 or the electronic device 500.
  • Acceleration or angular velocity actually measures the force exerted on a substance and indirectly measures the acceleration or angular velocity through forces applied from the outside of the substance. Therefore, the micro motion or micro movement of the muscle caused by the neuro-derived mechanism is transmitted as a force applied to the electronic device, and the measured force may be indirectly calculated in the form of acceleration or angular velocity.
  • An external force is applied from the outside of the electronic device to which the sensor is attached, and the moving plate (MASS) of FIG. 9 moves and the change of capacitance occurs because the distance of the electrode in the sensing structure changes.
  • MMS moving plate
  • the changed capacitance is converted into the form of analog voltage and an analog voltage signal is applied to the input of the A / D converter via an amplifier.
  • Multiple of the sensing structure can make it possible to measure multiple axes' values of the acceleration and angular velocity and these values can be used for more sophisticated applications.
  • the measured electrical analog signal may be sampled by a predefined sampling frequency for a predefined period (e.g., 3, 5, 10, 20, 30 seconds, etc.) in the A/D converter.
  • FIG. 10 shows the block diagram of the sensor on the electronic device 401 or the electronic device 500.
  • the Sensor 1010 may include an Acceleration Sensing Structure 1012, a Gyroscope Sensing Structure 1014, a Temperature Sensor 1016, an EMS 1017, an A / D converter 1018, a Signal Conditioning 1020, a Serial interface 1022, an Interrupt Controller 1024, a FIFO 1026, a Registers 1028, a Memory 1030, a Processor 1032, an External Sensor Interface 1034 and a System bus 1036.
  • the Acceleration Sensing Structure 1012 may include a plurality of sensing structures to measure the acceleration of a plurality of axes.
  • the acceleration measured in the acceleration sensing structure can be analog output in the form of analog voltage and it can be converted into digital data through an A / D converter.
  • Measured acceleration from the Acceleration Sensing Structure 1012 may be drifted to the temperature change due to a characteristic of material consist of sensing structure. The drift of sensing value can be compensated with the help of the temperature sensor 1016.
  • the Signal Conditioning 1020 may include a signal processing filter required for signal processing to improve signal quality.
  • the processor 1032 can control the configuration of the signal processing filter.
  • Acceleration values measured may be stored in the Registers 1023 through Signal Conditioning 1020.
  • the acceleration values stored in the Registers 1023 can be recorded in the range of ⁇ 2g, ⁇ 4g, ⁇ 8g, ⁇ 16g depending on the predefined configuration.
  • the Gyroscope Sensing Structure 1014 may include a plurality of sensing structures to measure the rotation of a plurality of axes.
  • the rotation measured in the Gyroscope Sensing Structure 1014 can be analog output in the form of analog voltage and it can be converted into digital data through an A / D converter.
  • Measured rotation from the Gyroscope Sensing Structure 1014 may be drifted to the temperature change due to a characteristic of material consist of sensing structure. The drift of sensing value can be compensated with the help of the temperature sensor 1016.
  • the Signal Conditioning 1020 may include a signal processing filter required for signal processing to improve signal quality.
  • the processor 1032 can control the configuration of the signal processing filter.
  • Rotation values measured may be stored in the Registers 1023 through the Signal Conditioning 920.
  • the rotation values stored in the Registers 1023 can be recorded in the range of ⁇ 125, ⁇ 250, ⁇ 500, ⁇ 1000, ⁇ 2000 degrees/sec depending on the predefined configuration.
  • the Host Processor 1040 does not need to constantly monitor the sensor data, thereby reducing the current consumption of the electronic device.
  • the host processor 1040 may be a processing unit 410 of the electronic device 401 and a processing unit 501 of the electronic device 500. Data sensed by the sensor may be delivered to the Host Processor 1040 through the Serial Interface 1022.
  • the Serial Interface 1022 also allows the Host Processor 1040 to set the sensor's control registers.
  • the Serial interface 1022 may include SPI, I2C, and the like.
  • the Interrupt Controller 1022 can configure an external interrupt pin connected to the Host Processor 1040, interrupt latching and clearing method, and send an interrupt trigger signal to the host processor 1040.
  • An interrupt signal can be triggered when the sensor data is ready, or when the data is ready in the FIFO to be read by the host processor 1040.
  • an interrupt may be triggered even when the host processor 1040 reads data from an external predecessor.
  • the Host Processor 1040 may enter a sleep mode, and if the data is not prepared from the external sensor 1060 connected to the sensor 1010, the Host Processor 1040 may remain in the sleep mode continuously.
  • the sensor 1010 can also act as a sensor core or sensor hub by waking the host processor through the sensor's interrupt and enabling the necessary data processing for the Host Processor 1040.
  • FIG. 11 is a plot of an acceleration waveform 1100 of a hand acceleration signal for a single axis (X, Y, or Z) shown over time. A portion 1101 of the hand acceleration waveform 1100 is magnified as waveform 1100T as shown. While analog signal waveforms may be shown in the drawings, it is understood that analog signal waveforms may be sampled over time and represented by a sequence of digital numbers at discrete periodic timestamps (a “digital waveform”). While an accelerometer senses acceleration over time, if a sensor senses displacement over time instead, it may be converted into acceleration by twice differentiating the displacement signal with time.
  • the hand acceleration for each axis is sampled over a predetermined sample time period 1105, such as 5, 10, 20 or 30 second time spans for example.
  • the sampling frequency is selected so that it is compatible with the filtering that follows.
  • the sampling frequency may be at 250 Hz (4 milliseconds between samples).
  • the sampling frequency can be 330 Hz or 200 Hz, for example.
  • the sampling may be performed on an analog signal by a sampling analog to digital converter to generate the samples S1 -SN represented by a digital number over the time stamps T 1 -TN during the given predetermined sample time period. Assuming a 20 second sample time period and a sampling frequency of 250 Hz, a dataset for acceleration would include 3 (3 axes) times 5000 samples over the time period for a total of 15 k samples.
  • sampling frequency of the input data hander 802 can be, for example, 60Hz, 200Hz, 250Hz, 330Hz, 400Hz and so on, which is more than twice the 30 Hz frequency.
  • the collected data of the Input Data Handler 802 may further perform an operation of removing noise or improving signal quality to improve signal quality.
  • the analog value sampled by the predefined sampling frequency may be converted into a digital signal through a quantization process in the A/D converter 1018. In the quantization process, quantization may be performed according to a predefined bit rate.
  • linear quantization can be performed with a constant quantization width, and nonlinear quantization, which expands or compresses the quantization width according to a predefined value in a certain range, can be used to obtain a high-quality signal-to-noise ratio for an application.
  • FIG. 12 is an example of a flow chart of collecting a movement signal data of Input Data Handler 802 on the electronic device 401 or the electronic device 500.
  • the electronic device can collect sensor signal data containing neuro-muscular tone from a body part of the user’s body with a predetermined sampling frequency over a predetermined sample period, converts analog voltage values measured from sensing structure including a mass plate to digital values, compensate the digital value that is drifted by temperature with the help of the temperature sensor 1016, store a plural of the digital values into FIFO 1026, and generates interrupt signal to the Host Processor 1040 when the FIFO data is ready to be transferred.
  • FIG. 13 is an example of a flow chart of sleep mode operation of the Static Model processing system 800 on the electronic device 401 or the electronic device 500.
  • the power consumption may become important issues.
  • the electronic device 401 , 500 may operate in a sleep mode.
  • various methods can be applied such as shutting down the power of some components in the electronic device 401 , 500, switching to the low power mode, lowering the frequency of the operation clock for the minimum power consumption and so on.
  • the power consumption efficiency may be increased when the processing unit 501 enters the sleep mode.
  • a coprocessor such as the sensor core 526 may be included inside the processing unit or in the electronic device. Even when the processing unit 501 enters the sleep mode, the sensor core 526 can continuously observe the signal detection from the sensors 570. When it is determined, by the sensor core 526, that processing of the processing unit 501 is required, the sensor core 526 can generate interrupt signal to the processing unit 501 and the processing unit 501 gets out of the sleep mode. At this time, the power can be supplied again to some of the components that were into sleep mode and the processing unit 501 exits from the low power mode and changes the frequency of the operation clock to be operated at the fast clock in order to wake up from the sleep mode.
  • FIG. 14 is an example of a flow chart of secure mode operation of Static Model processing system 800 on the electronic device 401 or the electronic device 500.
  • the Static Model processing for Contextual Situations can be considered an operation required security.
  • the operation of handling data collection from the sensor may be operated by switching the first core 404 in the processor unit 501 to the secure mode 508.
  • the signal transmitted by the sensor or the sensor core 526 via a bus or an interrupt can be transmitted to the monitor mode 513 to switch the first core 504 to the secure mode 508.
  • the execution mode of the first core 504 is switched to the secure mode, the execution environment for security runs isolated from the normal execution environment.
  • the core entering the secure mode 508 can access or control the system resources of the electronic device accessible only to the secure operating system in the secure execution environment.
  • the Input Data Handler 802 can identify data collection mode from the user.
  • the data acquisition mode can include a data acquisition mode for learning and a data acquisition mode for inference.
  • the signal acquisition for training can be performed simultaneously to improve the performance of the previously trained model.
  • a Ul-related component may be displayed on the electronic device screen to collect data in a sitting posture, a standing posture, a walking posture, or the like.
  • the Ul- related components may be displayed so that the user can input by distinguishing the activity state such as whether the user is running, riding a bicycle, or riding in a car.
  • the collected data may be analyzed to determine a posture or an activity state of the user to process the corresponding information.
  • the electronic device 401 , 500 in performing static model processing for contextual situations including physiological states of the users which can be liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers and so on, can be performed by assigning such the function to a cluster of high-performance processor cores. For example, if the first cluster 503 is a cluster of high-performance cores, the first cluster 503 can be assigned.
  • FIG. 15 shows the block diagram of the Contextual Situations Feature Extractor 1500.
  • the Contextual Situations Feature Extractor 1500 can include a Preprocessing Handler 1510, a Signal Filtering Handler 1512 and a Feature Extracting Handler 1514.
  • the Contextual Situations Feature Extractor 804 can be configured as the Contextual Situations Feature Extractor 1500 of FIG. 15.
  • the Contextual Situations Feature Extractor 804, 1500 can get numerical data such as the first sensor data 1502 and the second sensor data 1504 from the Input Data Handler 802. If the input data is received from the acceleration sensor or the gyroscope sensor, the numerical data can be collected as shown in FIG. 16.
  • FIG. 16 is an example of various types of sensor data and format can be used herein.
  • the multidimensional sensor data may be referred to as raw data.
  • Signal processing such as preprocessing, filtering, etc. can be performed on the raw data to achieve optimal performance in the next step.
  • FIG. 17. is an example of a flow chart of a preprocessing operation for the Contextual Situations Feature Extractor 804, 1500 on the electronic device 401 or the electronic device 500.
  • methods for performing preprocessing may be determined according to the use of the collected signal.
  • the collected signals can be used for authentication, posture estimation, and activity information estimation.
  • the preprocessing method can be processed differently depending on the usage and partially overlapped.
  • the preprocessing handler 1010 can check for the following input data:
  • the preprocessing handler 1010 can process the input data by determining the quality of the input data or determining it is an error (e.g., outside an expected range of data).
  • An input state machine operation can be performed according to the quality of the input data.
  • an operation of collecting the input data may be performed again, or a user interface may be generated that requires a user to perform an additional operation in order to collect more input data.
  • the signal obtained from the motion sensor for about 1 to 2 seconds at the beginning of the signal acquisition may include a large amount of the signal of the macro motion of the user and may be affected a lot by the shaking of the electronic device. As a result, the signal may be discarded at the beginning of the signal acquisition or at a certain interval immediately before the acquisition is completed.
  • the Preprocessing Handler 1510 can perform resampling procedure or interpolation on input data.
  • the resampling function can uniform or nonuniform data to new fixed rate data.
  • the input data derived from sensors that are sampled at a high level of hardware abstraction and are subject to many variations depends on hardware components manufactured by a certain company or sampling configuration on the sensor component. As a result, input data from sensors written in the raw data format may be non-uniformly sampled. Input data can be corrected by a resampling procedure of the Preprocessing Handler 1510 to a new uniform rate before further analysis.
  • the resample procedure can correct the small deviations in non-uniform samples through linear or cubic interpolation and provides a constant time between samples.
  • the resample procedure can use a cubic ‘spline’ to correct deviations in the sampling rate.
  • [Ax, T] resample( Axyz(:, 1), time, 'spline');
  • the Signal Filtering Handler 1512 can perform following filtering processing on the input data.
  • the Signal Filtering Handler 1512 can perform filtering to remove an unnecessary signal for micro motion data extraction from the collected signal.
  • Unnecessary signals may include, for example, noise, macro motion signals, distortion due to gravity, and the like. Since the power noise may be often generated in the collected signal when the electronic device is being charged, the signal may be filtered in consideration of characteristics due to power noise.
  • the frequency of neuro-muscular micromotions, derived from nerves or due to the inherent neuro-muscular anatomy of a human based nerve can be observed mainly in the range of 3 Hz to 30 Hz.
  • a signal in the range of 3Hz to 30Hz or 4Hz to 30Hz from the collected input motion data can be extracted by using a signal processing algorithm.
  • a signal in a range of 4Hz to 30Hz may be extracted, and in another embodiment, a signal in a range of 8Hz to 30Hz may be extracted. In another embodiment, signals in the range of 4 Hz to 12 Hz or 8 Hz to 12 Hz may be extracted.
  • the Signal Filtering Handler 1512 can use a signal processing that analyzes input data and then classifies/identifies input data as small signals and large signals which are separated out from the small signal amplitude of the micro-motions.
  • Signal Filtering Handler 1512 can suppress/filter macro motion (large movements of the user's body, large movements of the arm or walking, running, jogging, hand gestures, etc.) from the collected input data.
  • the example analysis may be of the form described in “Time Series Classification Using Gaussian Mixture Models of Reconstructed Phase Spaces” by Richard J. Povinelli et al., IEEE Transactions on Knowledge and Data Engineering, Vol. 16, No. 6, June 2004.
  • a separation of the large signals due to voluntary motion may be made by using a BMFLC-Kalman filter as is described in “Estimation of Physiological Tremor from Accelerometers for Real-Time Applications” by Kalyana C. Veluvolu et al., Sensors 2011 , vol. 11 , pages 3020-3036.
  • the Feature Extraction Handler 1514 can extract unique characteristics from the extracted neuro-muscular micro motion data according to the static model operation mode.
  • FIG. 18 shows examples of time series of a single axis Accelerometer data sample that shows crossings of the center axis based on the extracted neuro-muscular micro motion data generated and processed by the Preprocessing Handler 1510 and the Signal Filtering Hander 1512. These crossings can be indicative of physiological states and measured using mathematical functions such as the Barlow Feature.
  • the Barlow Feature is commonly used in EEG electroencephalography analysis.
  • the Barlow feature is one example of hundreds of potential features that can measure the presence of the physiological states.
  • the global tendency of a measured neuro-muscular tremor to cross a central line of origin can be quantified. This measurement alone, or in combination with any number of other features and preprocessing techniques can be used to train a static machine learning model.
  • the scale of the signal data or the extracted feature data may differ according to the type and structure of the electronic device, the variation of the sensor component, the sampling frequency of the signal, the contact type between the user and the electronic device, and the like.
  • the signal data or the first feature data may be measured on a scale of 1 to 10
  • the second feature data may be measured on a scale of 1 to 1000.
  • standardization can be performed on the signal data or the feature data.
  • the signal data or the feature data can be made into normal distribution by centering the data such that the standard deviation is one and the average is zero.
  • normalization may be performed instead of standardization as needed for processing components of the Static Model Analyzer, and both normalization and standardization may be used.
  • normalization or standardization may be performed on the sensor data, may be performed on the feature data, or may be performed on all or part of the sensor data or the feature data. The normalization or standardization process may be skipped depending on the characteristics of the sensor data or the feature data.
  • the initial step can include a step of subtracting each data value from the mean of measured data so that its empirical mean is zero and each variance of data is equal to one. After this initial step, based on the correlation between data, the direction of maximal variance in high-dimensional data can be found and the number of data can be reduced by projecting them into new subspaces with the same or smaller dimensions than the original.
  • a simple procedure can be to standardize on n-dimensional data, create a covariance matrix, decompose it into eigenvectors and eigenvalues, and select the eigenvector that corresponds to the largest eigenvalue to produce a projection matrix.
  • a transformation through the projection matrix can be performed to the signal data or the feature data.
  • FIG. 19 is an example of a flow chart of feature extracting operation of the Contextual Situations Feature Extractor 804 on the electronic device 500 or the electronic device 401 .
  • the following values or output data from some processing can be obtained for the preprocessed data and used as feature vectors.
  • the following values may be obtained for the preprocessed data, and the values may be directly used, partially modified, or some combination of them in order to be used as feature vectors.
  • micro-motion data can be collected from various people and analyzed in the laboratory.
  • various sources such as age, gender, region, body physique and etc.
  • FIG. 20 shows an example of a feature vector set according to some embodiments.
  • the features can also be selected in the laboratory, based on an analysis of the various types of motion classification characteristics shown in FIG. 3.
  • the feature vector set may be configured differently according to the usage of the collected signal. For example, a set of features used for authentication and a set of features for posture estimation or activity information estimation may partially overlap but may be configured differently.
  • FIG. 21 shows an example of a feature vector set for physiological states according to some embodiments.
  • the features can also be selected in the laboratory, based on an analysis of the various types of experiment results in the laboratory.
  • the feature vector set may be configured differently according to the physiological states.
  • a set of features or weight values of features used for each physiological state may partially overlap or be configured differently.
  • a feature that contains high information can be Barlow Activity.
  • This feature is able to provide information to a machine learning algorithm on the sum of zero-crossings of a time series accelerometer or gyroscope reading in Static Model Analyzer 806. There are less consistent zero-crossings and zerocrossings of greater magnitude with neuromuscular signals when compared to many mechanically, or non-human, generated signals which affect the value of a feature such as Barlow Activity.
  • This quantitative feature value can contribute information to help make a Liveliness determination by a machine learning classifier.
  • a separate example of a feature that contains information to help make a classification determination in blood glucose levels can be Mean Band Power.
  • the Mean Band power feature contains information on the power contained in certain frequency bands of the time series signal. This feature may be higher or lower in certain frequency bands based influenced by a user’s blood sugar levels. The values of this feature can be used by a machine learning algorithm to determine the presence of low blood glucose or the degree to which the physiological state of low blood glucose is present in Static Model Analyzer 806.
  • Using a variety of features in combination may be useful when determining the presence of stress hormones or the degree to which stress hormones are present with a machine learning algorithm in Static Model Analyzer 806.
  • More features may be used in the training of the machine learning algorithm to help differentiate the presence of stress hormones in a neuromuscular signal from physiological states where stress hormones may be elevated, such as low blood glucose or sleep deprivation.
  • the information contained in the Mean Band Power feature, LD Metric, Entropy based features, and Barlow Activity and Mobility, can train the machine learning algorithm to recognize the presence of stress hormones more accurately, and provide unique determinations from other physiologic states where stress hormones may also be present, like low blood glucose.
  • FIG. 22 shows divergence feature distribution that can differentiate humans from non-human as one of the examples in the disclosure.
  • This plot shows the distribution of one feature computed across 9 humans, and the same feature from non-human (static) recordings computed from accelerometer data.
  • the accelerometer data was filtered to frequencies between 10 and 15 hertz which is in the spectrum of human physiological neuro-muscular tone.
  • the distributions show how the value of the feature varies across roughly 100 recordings per person and that non-human signals can be easily distinguished from human signals.
  • This feature can be combined with other features computed from the same signal to further differentiate human signals from non-human signals.
  • the choice or combination of features can vary in the type of physiological states.
  • the features or combination of them that clearly describe physiological states can be determined after appropriate experiments in the laboratory and these experimental results can be directly or indirectly stored on the device as a constraint sets for a static model operation mode.
  • the constraint set can also be downloaded or updated into the device over the network.
  • entropy feature analysis can be applied in order to perform static model processing for physiological states of the users including liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers, and so on.
  • Entropy is a candidate feature class that is used to predict or measure one or more of the above physiologic states. It is a non-linear feature that is used to quantify the predictability or the information content of a sample of time-series sensor data. Kolmogorov, Approximate, Shannon, and Sample Entropy are all different methods used to quantify the complexity of a time series sample such as accelerometer or gyroscope data.
  • an entropy feature used to quantify a physiologic state involves breaking the time series data sample into “N” equal length segments. The computation that is then performed on each segment compares the similarity, or distance, of each segment to all the others based on an experimentally derived distance metric threshold. Each segment receives a score of how many segments it was similar to out of the total number of segments and the average of the logarithm of each proportion is computed. The same calculations are then carried out for segments of increasing length. This entropy value quantifies how repetitive a signal is.
  • Physiological signals such as neuro-muscular tremor and motion, contain nonlinear, non-repetitive, and complex patterns that express themselves across time.
  • a mechanically created time-series signal, or a signal from a non-biological system, would have a lower entropy value than one influenced by human physiology. Sleep deprivation is known to influence neuro-muscular tremor and the degree to which it does may be quantified using entropy analysis.
  • FIG. 23 shows time-series signals of entropy feature analysis.
  • the time-series signal on the left has lower entropy than the time-series signal on the right in FIG. 23.
  • physiologic states can be quantified and represented in an N-dimensional feature subspace used to train a machine learning static model.
  • This static machine learning model can be used to make determination of the presence or absence of any of the above physiologic states and/or the degree or probability that any of the above states are present in a sample of time-series sensor data.
  • frequency analysis can be applied in order to perform static model processing for physiological states of the users including liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers, and so on.
  • Frequency analysis is a feature class that is used to measure or predict one or more of the above physiological states.
  • One example of a feature that falls within this class is the mean band power feature.
  • the mean band power feature can be used to determine the presence of a physiological state such as low blood sugar. For example, when low blood sugar is present, the amplitude of the signal present may be increased in the accelerometer or gyroscope time series data. This increase in amplitude is likely to present in specific frequency bands related to human physiology and can be measured with the mean band power feature.
  • FIG. 24 shows a plot of Gyroscope x-axis Low Mean Band Power Raw Signal.
  • the above figure shows how the orange signal, a signal that may represent a physiological state of low blood sugar, differs from a signal without low blood sugar.
  • the mean band power in specific frequency bands will be higher in the physiological state of low blood sugar.
  • FIG. 25 shows an example of data sets according to physiological states
  • FIG. 26 shows an example of data sets according to physiological characteristics respectively, that are of interest.
  • the data sets according to physiological states or physiological characteristics can be determined in the laboratory and these sets can be also updated in the electronic device 401 , 500 over the network.
  • the electronic device 401 , 500 can configure the Static Model processing system 800 with at least one of the above data sets according to the application.
  • FIG. 27 is a block diagram of the Static Model Analyzer 806, 2700 according to one embodiment.
  • the Static Model Analyzer 806, 2700 can include a Classifier Engine 2740, a Training Interface 2710, a Static Model Running Interface 2720, and a Tuning Interface 2730.
  • the Classifier Engine 2740 can include a Training Engine 2741 , a Static Model Running Engine 2742, a Tuning engine 2743, and a Classifier Kernel 2744.
  • FIG. 28 is an example of a flow chart of training mode operation of the Static Model Analyzer 806, 2700 on the electronic device 401 or the electronic device 500.
  • the Static Model Analyzers 806, 2700 can be operated in training mode in the laboratory to build static model parameters according to physiological states.
  • the electronic device 401, 500 can enter into this training mode in the laboratory to build some parameters for future use of the static model for physiological states.
  • building some static model parameters can be performed not on the user devices, but on some laboratory electronic devices as well.
  • the electronic device 401 , 500 for users doesn't necessarily need to enter into training mode except updating or modifying parameters for the static models.
  • the FIG.26 or FIG.27 can be used to build some static model parameters.
  • the feature data 2750 of the authorized user extracted from the Contextual Situations Feature Extractors 804, 1500 can be collected.
  • the collected feature data can be transferred to the Training Engine 2741 of the Classifier Engine 2740 through the Training Interface 2701 for processing.
  • the user's feature data 2750 may be processed by various data processing algorithms or machine learning algorithms through cooperative operations of the Training Engine 2741 and the Classifier Kernel 2744 to determine the parameters of the static model.
  • the extracted feature data can be divided and processed into User's Feature Data 2750, Verification Feature Data 2752 and Test Feature Data 2754.
  • the User's Feature Data 2750 can be used for training to determine the parameters of the static model.
  • the Validation Feature Data 2752 can be used to improve the model's performance or accuracy during training mode prior to evaluating the static model to select an optimal model.
  • the Validation Feature Data 2752 can be used to tune the learning rate or perform validation while evaluating the performance of the model during training mode.
  • the Test Feature Data 2754 can be used to evaluate the final model instead of being used to select the model.
  • the Noise Feature Data 2758 can be a sort of feature data generated through a noise collection process.
  • the Noise Feature Data 2758 can be extracted from a signal collected in an environment in which a large number of components other than micromotion associated with neuro-muscular tones such as the existence of large movements or large vibrations around the electronic device.
  • the Landscape Feature Data 2758 can be a feature data collected from various people and feature extraction performed in a laboratory. Extracted landscape feature data may be stored in the storage of the electronic device in some sets and used to improve the performance of the static model.
  • FIG. 29 is an example of a flow chart of static model running mode operation of the Static Model Analyzer 806, 2700 on the electronic device 401 or the electronic device 500.
  • the Static Model Analyzer 806, 2700 can be operated in a static model ruining mode.
  • the electronic device 401 , 500 can have been operated in the training mode before the electronic device 401, 500 operates in the static model running mode so that parameters for the model for the feature data set of the physiological states for the user are already configured.
  • the Static Model Running Engine 2742 of the Classifier 2740 can be operated in collaboration with the Classifier Kernel 2744 through the Static Model Interface 2720 for the user's new feature data 2750.
  • the Classifier Kernel 2744 based on the previously generated static model can perform an operation on the newly extracted feature data to generate a numerical degree of physiological states to the previously authorized user.
  • FIG. 30 shows a block diagram of the Static Model Application Framework 808 according to one embodiment.
  • the Static Model Application Framework 808 can provide to enable various applications using the output of the Static Model Analyzer 806.
  • the Static Model Application Framework 808 can include an output state machine for performing static model processing for contextual situations including physiological states of the users.
  • the Static Model Application Framework 808 can provide application programming interfaces (APIs) regarding physiological states of the users including liveliness, blood glucose levels, stress hormone levels, presence of drugs, identifiers, and so on.
  • APIs application programming interfaces
  • the Static Model Application Framework 808 can use the user's extracted feature data associated with contextual situations including physiological states of the users.
  • User's feature data can be acquired by the Contextual Situations Feature Extractor 804 and they may be stored in the secure storage of the electronic device.
  • the Static Model Application Framework 808 can use temporarily feature data associated physiological states of the users and then discard them after using them.
  • FIG. 31 shows an example of a device type of Situational Static Models.
  • the Static Models may be implemented in hardware, implemented in software, or implemented in a combination of hardware and software.
  • the Static Models may be in the form of a software library that is executed in the microcontroller. Some components of the Static Models may be implemented inside SoC in the form of an accelerator.
  • the Static Models may be fully implemented in the form of the standalone chipset (or ASIC) as in FIG.31 as well but the implementation form of the Static Models is not limited to present examples.
  • One way that authentication efforts can be strengthened is by coupling authentication requests with secondary authentication input, (multifactor authentication), and/or by adding a “liveliness” credential to validate that the authentication request is being made by a live requestor and not by a robot or counterfeit inanimate data.
  • the goal of current “liveliness” determination technologies such as CAPTCHA and others, is to determine whether a human is participating in an online-transactions versus spambots.
  • CAPTCHAs are in common use in the on-line world and are seen frequently in desktop computing, when requesting access to a website. What is proposed is a method to supplement any authentication method with an indicator of “liveliness” verifying that the users employing any of the many authentication methods is a physiologically active, living/breathing individual.
  • sensor data from the device can be collected in the background. Contact and interaction with the device, will be evident in accelerometer, gyroscope, and other sensor data streams.
  • One component of the sensor data is associated with the neuro-muscular tone generated by the body’s proprioceptive system where the brain is continually communicating with the peripheral nerves to assess the body’s position in the environment.
  • Other physiologic biometric parameters can be used as well (i.e., ECG, EEG, HVR, heart sounds, etc.) to provide a liveliness indicator, however, these parameters would require additional hardware to collect those signals.
  • Extracting neuro-muscular data is an invisible process that is easily collected from the accelerometer data being collected by the phone, whenever the user is holding it. This data can be processed remotely or on the device to determine if a human is present. Neural networks, or other methods of machine learning can be used to determine if the signal present is uniquely human. If the result is positive, a user will proceed to submit the form, or gain access to a site, without any additional information. If the result is negative, a traditional CAPTCHA or OTP can be used as a second level authentication request, or the user could be requested to hold the device again. [0176] An alternative implementation uses a combination of the above method with external verification where a user receives a prompt via text or phone call and told to hold their device in hand while sensor data is collected from their device.
  • accelerometers would have to added to the mouse or touch pad interface to provide micro-motion data.
  • FIG. 32 depicts an example of the static model processing system for liveliness physiological states on the electronic device 401 or the electronic device 500 according to some embodiments.
  • the electronic device can sense signals in Sensing 3211 step from outside of the electronic device by the Input Data Handler 802, for example, a movement signal data from a body part of the user’s body.
  • the electronic device can perform preprocessing the signals collected from the Input Data Handler 802 in Preprocessing 3212 step by the Contextual Situations Feature Extractor 804, for example, suppressing signal components associated with a voluntary movement of the user, noise, sensor errors, gravity force, electronic power noise, and other noise-related signal and generating data sets associated with neuro-muscular tone.
  • the electronic device can perform feature extraction from the preprocessed signals in Feature Extraction 3213 step by the Contextual Situations Feature Extractor 804, for example, extract feature vector sets by generating data sets of mathematical representation regarding contextual situations of the user, for example, liveliness state.
  • the electronic device can perform training operations using the feature vector set by calculating parameters of static models and evaluating each static model in Learning 3214 step by the Static Model Analyzer 806.
  • the electronic device can perform Static Model running operations by configuring the model parameter set for each predetermined static model and generating a numerical degree of the matching level to the previously authorized user in Prediction 3215 step by the Static Model Analyzer 806.
  • the electronic device can determine the user access to the electronic device in response to the numerical degree of physiological states in Decision 3216 step by the Static Model Application Framework 808.
  • FIG. 33 is an example of a flow chart of static model operation for liveliness on the electronic device 401 or the electronic device 500.
  • the electronic device 401, 500 can configure a static model operation mode as liveliness constraint of a user, load information of a static model operation mode including constraint set of contextual situations (e.g., physiological states) of a user, configure the static model parameter set according to the static model operation mode, collecting sensor signal data containing neuro-muscular tone from a body part of the user’s body with a predetermined sampling frequency over a predetermined sample period, suppress signal components associated with a voluntary movement of the user from the sensor data, generate data sets of mathematical representation regarding contextual situations (e.g., physiological states) of the user based on the static model operation mode from the sensor data suppressed signal components associated with the voluntary movement, construct a feature vector table containing multiple sets of feature vectors based on the data sets of mathematical representation, execute the static model using the feature vector table according to the static model operation mode and generate report information on the contextual situations (e.g.
  • FIG. 34A depicts an example of creating a new account operation using liveliness physiological states.
  • the electronic device of the user can send a request message to access the website that provides interfaces of creating the new account
  • the electronic device on a server-side can send web documents that provide user interfaces for creating the new account based on the request of the electronic device of the user.
  • the electronic device of the user and the electronic device on the server-side can exchange information to proceed with creating account procedures.
  • the user enters the website where they wish to create the account and immediately sensors present on the device begin collecting information in the background [initiated by the website] by processing the static model for liveliness physiological states.
  • sensors are collecting information passively with or without the user’s knowledge.
  • a liveliness determination will be made from the passively collected data. This liveliness determination will decide whether human physiological signals are present in the passively collected data.
  • the passively collected data comes from accelerometers, gyroscopes, and magnetometers in the user’s mobile device. Ballistocardiographs (heartbeats), as well as neuro-muscular tone in the limbs, can be detected from this sensor data. If the signal is determined to contain these physiological signals, the user will be determined to be alive and human and be allowed to create an account on the website or application.
  • FIG. 34B depicts an example of accessing personal health records using liveliness physiological states.
  • the electronic device of the user can send a request message to access the website that provides interfaces of the database containing personal health records, then the electronic device on a server-side can send web documents that provide user interfaces for accessing personal health records based on the request of electronic device of the user.
  • the electronic device of the user and the electronic device on the server-side can exchange information to proceed with login procedures.
  • a liveliness determination will be performed to ensure a human is performing this action by processing the static model for liveliness physiological states.
  • the user After submitting their login credentials, but before gaining access to their health records, the user can be requested to use a heart rate sensor on the device they wish to access the health records on. Information collected from this sensor will be computed and a liveliness determination will allow the user to access their health records if the signal collected from the heart rate sensor is determined to contain a human physiological signal.
  • Any authentication technology - especially image-based methods would benefit from a liveliness verification performed automatically - invisible to the user when the user is attempting to access a website or device.
  • a user wants to access their online banking web portal and as they open the application and proceed to provide a username and password - or to place their finger on the fingerprint sensor, the phone is simultaneously capturing the neuro-mechanical micromotion data being collected continuously by the accelerometers in the phone and provides that liveliness verification along with the authentication input.
  • the static model processing for liveliness physiological states can be coupled with another abstract authentication parameter, i.e., password, PIN number, OTP, etc. or with another behaviorbased “liveliness” parameter, i.e., CAPTCHA challenge, motion repertoire, voice commands, swipe patterns, etc.
  • the authentication sample for the static model could be any one of a number of image- based, physical features, or other authentication technologies to include: fingerprints, handprints, iris scans, facial recognition, face veins, etc.
  • the static model processing can satisfy a liveliness assessment automatically by combining signals representative of any number of physiologic functions, the information received from the user during an authentication request via a conventional modality by sampling physiologic data simultaneously while acquiring an authentication sample using another authentication modality, (i.e., Password/PIN entry, voice commands, facial recognition, handprint analysis, iris scan, behavior repertoire, etc.), by assessing the physiologic data to determine if it is consistent with known physiologic functions, or by then forwarding that determination that a physiologic process is present to satisfy a liveliness assessment to be combined with the authentication sample being collected.
  • another authentication modality i.e., Password/PIN entry, voice commands, facial recognition, handprint analysis, iris scan, behavior repertoire, etc.
  • a neural tapping interface and platforms allow for the capture of signals originating in the nervous system and relayed by the neuromuscular junctions. Those signals are electronic in nature but can readily be captured by their micro-mechanical effects on the muscle cells, using devices that are equipped with micro-electromechanical system (MEMS) sensors. Such device would include smartphones, tablets, but also any system on chip (SOC) system. The signals originating in the nervous system are ubiquitous throughout the human body, and could be acquired nearly anywhere using devices equipped with the appropriate sensors as long as there is a contact between human body and device. Such devices would include vaping systems if equipped with SOC and MEMS.
  • MEMS micro-electromechanical system
  • Al codes can be fully cloudless, it allows a company to build products, which can be trained using very cheap electronic chips (like microcontrollers for example), or to embed fully trained Al codes that could answer/infer some well-defined questions.
  • HRV heart rate variability
  • Previously a proof of concept (POC) was formed for an age-blocking (or parental control) application.
  • the Al code for the age-blocking application used smartphones for data capture and inference.
  • the signal processing and data extraction techniques of the implementation (Al code) is not based on classical ‘statistical big data Al’.
  • Such POC depending on the code versions and implementations, can have an effectiveness (accuracy) between 87-94%.
  • Ninety-four effectiveness (accuracy) percent 94% an age-blocking (or parental control) Al code based on the human nervous system is most likely the absolute limit of the technology, given that there are likely limits to the informational content related to age at the level of the neuro-muscular junctions.
  • 35A illustrates a flow chart of using static artificial intelligence models to pass/fail implementation of age-blocking (parental control). If a buyer/user is under age, he is blocked (Fail) from a purchase or sale. If a buyer is of age (greater than equal), the purchase is not blocked (Pass) from a purchase or sale.
  • a question, for which a physiological marker exists, can be readily addressed by a constrained data set collected from a cohort of volunteers.
  • the constraint can be hormonal status, gender, muscular stiffness, age, etc. This constraint is the condition.
  • an Al model can be trained and an inference engine can be built to resolve the condition.
  • Programing code and a framework can be used to deploy pre-trained Al on chipsets such as a microcontroller or a systems on chip (SOC).
  • the inference results can be used to control an ignition switch in a vaping device on any SW or HW decision point relative to any parental control system based on agerestriction. It is a Pass/Fail model.
  • SOC is a cheap chipset with the appropriate sensors that allow for data capture, processing for extracting the relevant information (feature functions) and inference.
  • the result of the inference step can be used either in a pass/fail implementation or in a pass/check implementation.
  • the inference results are used to control the ignition switch of said device.
  • the accuracy of the Al could be boosted at some higher level (like 94-95%), and while this is way better than any other parental control that actually exists, it leaves space for some false positives and false negatives.
  • FIG. 35B illustrates a client server implementation of the age blocking and processes in each.
  • the process makes use of 2 checks performed at the point of sale of any device, under the supervision of the accredited salesperson. It is the understood that some form of control is implement according to legal statutes barring the sales of vaping to minor.
  • the cutoff ages may vary between 18, 21 , or 26 as the case may be but with the fused data implementation remain the same, but with adapted inference of the model for each different age limit.
  • vaping device or any other device
  • vaping device is shipped in a locked state (via its firmware, and will have to be un-locked/activated upon the completion of said sale with the age blocking artificial intelligence based on neurological signals.
  • an age blocking solution based on the nervous system combines manual age verification (carting) with a Tag that is user-specific but cannot lead to the user recognition - maintaining the user’s privacy.
  • POS point of sale
  • An age blocking solution based on the nervous system combines manual age verification (carting) with a Tag that is user-specific but cannot lead to the user recognition - maintaining the user’s privacy.
  • any PI I is collected by the device.
  • the links in the database between the identification (ID) of the consumer and the TAG may or may not be made. Such link is not essential to a working solution.
  • serial# The specific device (serial#) is fully non-active and needs to be activated after the sale.
  • Step 1 Buyer holds the device in one hand for a period of 30-60 sec
  • Step 2 Following this step, the SOO extracts a TAG (less than one second).
  • TAG a 3 numbers vector
  • a TAG is a downgraded inference that extracts a few appropriately chosen features, such as three features. Such TAG is a 3 number vector plus some other parameters. Those parameters reflect the future user, but - on their own - would not be able to allow to identify said user. This is due a.o. to the standard deviations linked to those features, but also to the nature of those features themselves (that is their physiological and use-specific information content). [0201] But this TAG can perfectly be used as a checking and decision-point mechanism to enhance the global accuracy of the Age-block technology. A TAG makes the device rather very personal, without breaching GDPR.
  • Age-block By the combining use of Age-block with a TAG, control at POS, and adding friction in the case of abuses, a cloudless, friction-free Al implementation age based control of any vaping device (or any device or parental control system) with a very high degree of accuracy can be achieved, thereby alleviating regulatory and branding risks.
  • FIG. 36 illustrates another application of an age based application of artificial intelligence based on neurological information.
  • a workflow of an age-controlled ignition control function for a vehicle is shown in FIG. 2. In this manner, children under an age limit would be unable to start the vehicle to operate it.
  • This solution may be implemented with or without 2 mini LED’s that could report the results of the 2 different inferences (red/green).
  • This solution requires that the vendor has a way to interact with the firmware of the device, this is ideally implemented via a connection between device and some other device provided to the POS terminal.
  • Tagging can be effective at 90% (it does not allow for identification, but the max overlap see FIG. 5 is at 7-8%), when combined with Age blocking effective at 90%, the cumulative error rate when both processes are fused and used together the error rate is at or below 1 %, such as can be desirable to unlock an ignition of a vehicle.
  • FIG. 37 is a flow chart of providing additional Al to provide side-effects protection from Nicotine abuse.
  • Nicotine is one obvious choice of drug that can be monitored, possibly positioning and branding vaping technology as a way to help quit smoking. Such effects will of course require a context-specific dataset, that will have to be collected (2,000 users at various nicotine doses), in order to train this additional static Al model aimed at infer nicotine at a few nicotine usage thresholds. This is a safety feature of the vaping technology, that would only allow Ignition at safe levels (or some defined levels of Nicotine). This is Software driven switch.
  • the device can be equipped with a third static Al model, pre-trained with a neuro-muscular general dose-response curve. The inference can be triggered at regular time intervals during vaping device usage, or can be used as an additional condition for ignition (this would be a product choice).
  • Al technology based on neurology can be embedded in phones, or any HW/SW SOC device to form a parental control device.
  • FIG. 38B illustrates a vaporizer with a system on a chip and the artificial intelligence to provide age blocking.
  • FIG. 39 illustrates a three dimensional plot of bubbles representing fused data one of which is a tag to improve accuracy and decrease error by tagging.
  • Three numbers are combined as vectors into a single bubble plot for a user.
  • Each bubble that is plotted represents a different person, for whom 3 numbers (from 3 different Al functions) were extracted 30-60 times and fused together.
  • the repeated tests were averaged together and represented as points in 3-D space in FIG. 39.
  • the diameter of the circles (bubbles) represent some statistical dispersion (SD and others) of the tests of a user.
  • the data needed to calculate those repeats is taken at a POS terminal during the period of 30 to 60 sec during which the buyer is asked to hold the device in one hand.
  • the computation time to extract and average is around 20 milli-seconds (msec), then the vector is stored on device memory (or could be exported via connection to a user DB).
  • FIG. 39 illustrates a 3-D vector space built on 3 different key features extracted from the neural-tagging data.
  • Each point represents the average of 30-60 sweeps of data from the same individual. Those averages are expressed at the center of a sphere, the radius of which is the standard deviation (SD).
  • SD standard deviation
  • the elements of the embodiments are essentially the code segments of instructions that can be executed by one or more processors to perform and carry out tasks and provide functionality.
  • the program or code segments can be stored in a processor readable medium or storage device that are coupled to or at least in communication with the one or more processors.
  • the processor readable medium can include any medium or storage device that can store information. Examples of a processor readable medium include, but are not limited to, an electronic circuit, a semiconductor memory device, a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM), a floppy diskette, a CD-ROM, an optical disk, a hard disk, or a solid-state drive.
  • the program or code segments can be downloaded or transmitted between storage devices, for example, over computer networks such as the Internet, Intranet, etc.

Landscapes

  • Engineering & Computer Science (AREA)
  • Computer Security & Cryptography (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Software Systems (AREA)
  • Computer Hardware Design (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Social Psychology (AREA)
  • Health & Medical Sciences (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

Selon un mode de réalisation, l'invention divulgue un procédé de génération de résultats reflétant un ou plusieurs états physiologiques d'un utilisateur. Le procédé comprend les étapes consistant à générer une pluralité d'ensembles de données de contraintes associés à une pluralité de contraintes prédéterminées liées à une pluralité d'états physiologiques prédéterminés ; construire une pluralité de modèles statiques indépendants sur la base d'une pluralité d'états physiologiques prédéterminés, chaque modèle statique indépendant étant lié à une contrainte spécifique ; installer la pluralité de modèles statiques indépendants dans un dispositif comprenant un processeur pour exécuter des instructions et un capteur pour collecter des données de capteur liées à la pluralité de modèles statiques indépendants ; exécuter, par un utilisateur, un ou plusieurs de la pluralité de modèles statiques indépendants par l'intermédiaire d'une interface utilisateur sur la base des données de capteur détectées par l'utilisateur ; et fournir un ou plusieurs résultats (inférences) à l'utilisateur par l'intermédiaire de l'interface utilisateur associée à l'exécution du ou des modèles statiques indépendants, le ou les résultats reflétant un ou plusieurs états physiologiques de l'utilisateur.
EP21862899.8A 2020-08-29 2021-08-28 Dispositifs électroniques avec un modèle d'intelligence artificielle statique pour des situations contextuelles, comprenant un blocage de l'âge pour le démarrage de vapotage et d'allumage, à l'aide d'une analyse de données et procédés de fonctionnement associés Pending EP4204996A2 (fr)

Applications Claiming Priority (3)

Application Number Priority Date Filing Date Title
US202063072099P 2020-08-29 2020-08-29
US202163138519P 2021-01-17 2021-01-17
PCT/US2021/048120 WO2022047272A2 (fr) 2020-08-29 2021-08-28 Dispositifs électroniques avec un modèle d'intelligence artificielle statique pour des situations contextuelles, comprenant un blocage de l'âge pour le démarrage de vapotage et d'allumage, à l'aide d'une analyse de données et procédés de fonctionnement associés

Publications (1)

Publication Number Publication Date
EP4204996A2 true EP4204996A2 (fr) 2023-07-05

Family

ID=80353300

Family Applications (1)

Application Number Title Priority Date Filing Date
EP21862899.8A Pending EP4204996A2 (fr) 2020-08-29 2021-08-28 Dispositifs électroniques avec un modèle d'intelligence artificielle statique pour des situations contextuelles, comprenant un blocage de l'âge pour le démarrage de vapotage et d'allumage, à l'aide d'une analyse de données et procédés de fonctionnement associés

Country Status (3)

Country Link
EP (1) EP4204996A2 (fr)
KR (1) KR20230058440A (fr)
WO (1) WO2022047272A2 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN115116097A (zh) * 2022-08-29 2022-09-27 成都体育学院 戒毒人员复吸风险预测方法、装置及可读存储介质
CN116369911B (zh) * 2023-06-05 2023-08-29 华南师范大学 基于生理信号的心脏信息检测方法、装置以及设备
CN117271969A (zh) * 2023-09-28 2023-12-22 中国人民解放军国防科技大学 辐射源个体指纹特征在线学习方法、系统、设备及介质

Family Cites Families (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8419650B2 (en) * 1999-04-16 2013-04-16 Cariocom, LLC Downloadable datasets for a patient monitoring system
WO2008031209A1 (fr) * 2006-09-16 2008-03-20 Terence Gilhuly Détecteurs améliorés et détection destinée à surveiller le blocage neuromusculaire
US20160128638A1 (en) * 2014-11-10 2016-05-12 Bloom Technologies NV System and method for detecting and quantifying deviations from physiological signals normality
EP3718056A1 (fr) * 2017-12-01 2020-10-07 Telefonaktiebolaget LM Ericsson (publ) Sélection de modèle d'apprentissage
US11484273B2 (en) * 2018-03-06 2022-11-01 International Business Machines Corporation Determining functional age indices based upon sensor data

Also Published As

Publication number Publication date
KR20230058440A (ko) 2023-05-03
WO2022047272A2 (fr) 2022-03-03
WO2022047272A9 (fr) 2022-04-21
WO2022047272A3 (fr) 2022-09-29

Similar Documents

Publication Publication Date Title
US11050747B2 (en) Data encryption and decryption using neurological fingerprints
TWI708205B (zh) 用於監測使用者之神經變性疾病、飲酒和心率變異性之方法、系統及電子裝置
TWI726864B (zh) 本機鑑認經授權使用者及對電子裝置之控制存取之方法、使用者鑑認之方法、電子裝置、生理鑑認控制器、產生使用者之唯一生理識別之方法及登入至系統或應用程式中之方法
WO2022047272A2 (fr) Dispositifs électroniques avec un modèle d'intelligence artificielle statique pour des situations contextuelles, comprenant un blocage de l'âge pour le démarrage de vapotage et d'allumage, à l'aide d'une analyse de données et procédés de fonctionnement associés
Dahia et al. Continuous authentication using biometrics: An advanced review
TW202105213A (zh) 使用神經及神經力學指紋之無鑰匙進出控制
Zhu et al. Blinkey: A two-factor user authentication method for virtual reality devices
Martinovic et al. Pulse-response: Exploring human body impedance for biometric recognition
KR102132612B1 (ko) 생체인식기술을 활용한 사용자 인증방법 및 인증장치
Cabra et al. Mechanisms of Authentication toward Habitude Pattern Lock and ECG: An overview.
US20230328417A1 (en) Secure identification methods and systems
CN116830124A (zh) 具有使用数据分析的用于包括对吸电子烟和点火启动进行年龄分段的上下文情境的静态人工智能模型的电子装置及其操作方法
US11947648B2 (en) Electronic device related to user identification, authentication, liveliness, encryption using biometrics technology and methods for operation thereof
Enamamu Bioelectrical user authentication
Vhaduri et al. mWIoTAuth: Multi-wearable data-driven implicit IoT authentication
WO2021232051A1 (fr) Dispositifs électroniques associés à un certificat physiologique utilisant la technologie biométrique et leurs procédés de fonctionnement
KR102504526B1 (ko) 생체 인식 기술을 활용한 사용자 식별, 인증 및 암호화 장치 및 그의 동작 방법
Clevenger Classification of Heart Sound Biometrics for Active User Authentication and Clinical Cardiac Applications
Jenkins et al. Authentication, privacy, security can exploit brainwave by biomarker
Yu et al. ThumbUp: Secure Smartwatch Controller for Smart Homes Using Simple Hand Gestures
Smith-Creasey Biometrics for Continuous Authentication
Qi et al. Piezoelectric Touch Sensing and Random-Forest-Based Technique for Emotion Recognition
Zhao Secure Personal Healthcare Using Fine-Grained PPG-Based Sensing and Deep Learning
Rasmussen Pulse− Response: Exploring Human Body Impedance for Biometric Recognition

Legal Events

Date Code Title Description
STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: THE INTERNATIONAL PUBLICATION HAS BEEN MADE

PUAI Public reference made under article 153(3) epc to a published international application that has entered the european phase

Free format text: ORIGINAL CODE: 0009012

STAA Information on the status of an ep patent application or granted ep patent

Free format text: STATUS: REQUEST FOR EXAMINATION WAS MADE

17P Request for examination filed

Effective date: 20230323

AK Designated contracting states

Kind code of ref document: A2

Designated state(s): AL AT BE BG CH CY CZ DE DK EE ES FI FR GB GR HR HU IE IS IT LI LT LU LV MC MK MT NL NO PL PT RO RS SE SI SK SM TR

DAV Request for validation of the european patent (deleted)
DAX Request for extension of the european patent (deleted)