RU2814781C1 - Method for tracking psychoemotional states of user and their correction - Google Patents

Abstract

FIELD: medicine.

SUBSTANCE: invention refers to medicine, namely to a method for tracking psychoemotional states of a user and their correction. Portable neural interface in the brain electroencephalogram (EEG) data acquisition mode performs user brain examination. Collected EEG data of the user are transmitted via a Bluetooth communication channel to a computing device for further processing. Computing device receives EEG data from the neuroamplifier, filters and preprocesses them by means of a fourth-order Butterworth filter. Filtered and preprocessed data are classified. To calculate psychoemotional and psychophysiological states, an individual calibration mechanism is used, during which the user goes through a cycle: 20 seconds of closed eyes, 20 seconds of open eyes, 20 seconds of closed eyes, 20 seconds of open eyes, 40 seconds of quiet wakefulness. During calibration the following values are calculated: value of individual peak of alpha rhythm (iAPF); alpha rhythm peak suppression value; individual boundaries of the alpha range; calculation of a base level of a cognitive fatigue index, a concentration index, an alpha-gravity index specific for a given user (Baseline). When calculating Baseline, values of rhythm power and their ratios are accumulated in a cyclic buffer with dimension of 5 seconds until accumulation of artefact-free values for total of 30 seconds. After calibration and calculation of an individual baseline, the values of the metrics with decoding are displayed to the user for monitoring. Based on the detected conditions, the user is presented with notifications with recommendations for completing the biofeedback trainings.

EFFECT: higher accuracy of tracking psychoemotional states of the user, as well as possibility of their correction.

6 cl

Description

TECHNICAL FIELD

This technical solution relates to the field of computer technology, in particular to a method for tracking the psycho-emotional states of the user and correcting them.

BACKGROUND OF THE ART

The solution chosen as the closest analogue, WO2022169376A1, is known from the prior art. This invention discloses a hardware and software system for improving the cognitive functions of a user, comprising: a computing device configured to display information and containing a client application; a brain-computer interface containing a neuro-amplifier; a software interface configured to control the operating modes of the neuroamplifier, namely, configured to: connect to the neuroamplifier and activate commands to obtain electrode resistances and/or to take and record brain electroencephalogram (EEG) data; a server contained as part of a software interface and including: a communication module with a brain-computer interface; software interface core; EEG signal classification module; module for sending data to the client application.

The proposed technical solution is aimed at eliminating the shortcomings of the current level of technology and differs from known solutions in that the proposed method provides high-quality and accurate tracking of the user’s psycho-emotional states. This solution uses an algorithm for cleaning the EEG signal from myographic (muscle) artifacts, which allows the use of this system (method) outside laboratory conditions.

SUMMARY OF THE INVENTION

The technical problem that the claimed solution is aimed at is creating a method for tracking the user’s psycho-emotional states and correcting them. Additional embodiments of the present invention are presented in the dependent claims.

The technical result consists in accurate and high-quality tracking of the user’s psycho-emotional states, as well as the possibility of their correction.

The claimed result is achieved by implementing a method for tracking the user’s psycho-emotional states and correcting them, which includes the stages of:

a portable neural interface, in the mode of collecting electroencephalogram (EEG) data, conducts surveys of the user’s brain;

the collected EEG data of the user is transmitted via a Bluetooth communication channel to a computing device for further processing;

the computing device receives EEG data from the neuroamplifier, after which it filters and preprocesses it;

filtered and preprocessed data are classified, while an individual calibration mechanism is used to calculate psycho-emotional and psychophysiological states, and their further detection and correction, which includes:

cycle: 20 seconds closed eyes, 20 seconds open eyes, 20 seconds closed eyes, 20 seconds open eyes, 40 seconds quiet wakefulness;

in this case, during the calibration process, the following values are calculated to calculate psycho-emotional and psychophysiological states: the value of the individual peak of the Alpha rhythm (iAPF); the amount of suppression of the Alpha rhythm peak; individual boundaries of the Alpha range; calculation of the base level of the cognitive fatigue index, concentration index, alpha gravity index, characteristic of a given user (Baseline);

after calibration and calculation of the individual baseline level, the metric values with interpretation are displayed to the user for monitoring;

Based on detected conditions, notifications are displayed to the user with recommendations for completing biofeedback training.

In a particular embodiment of the described method, data filtering and preprocessing is carried out using a fourth-order Butterworth filter, which is used for basic filtering of incoming data;

by applying the Fast Fourier Transform (FFT), the power spectral density (PSD) of the main brain rhythms is calculated - Delta, Theta, Alpha, Beta;

through the use of power and amplitude filtering, it is possible to get rid of myographic artifacts and isolate the useful physiological EEG signal.

In another particular embodiment of the described method, during the calibration process for calculating psycho-emotional and psychophysiological states, the following values are calculated:

the value of the individual peak of the Alpha rhythm (iAPF), is defined as the highest power in the measurement range of the Alpha rhythm at a certain frequency;

the amount of suppression of the Alpha rhythm peak is defined as the difference in power between the absolute peak in the Alpha range during the calibration stages with open and closed eyes;

The individual boundaries of the Alpha band are defined as the first value for the high-frequency alpha range, and the last value for the low-frequency alpha range, where the power difference between eyes open and eyes closed will be positive.

In a particular embodiment of the described method, the value of the individual peak frequency of the Alpha rhythm (iAPF) is defined as the frequency value in the Alpha rhythm measurement range from 7 to 13 Hz, at which the power is greatest.

In a particular embodiment of the described method, when calculating Baseline, the values of rhythm power and their ratios are accumulated in a cyclic buffer of 5 seconds, until a total of 30 seconds of artifact-free values are accumulated, and the mean square averaging method is used as the averaging method.

In another particular embodiment of the described method, all measurement results are saved in a local database and after the end of the session, along with raw EEG data, are sent to a remote web platform to track statistics and progress.

In a particular embodiment of the described method, the portable neural interface, in the mode of collecting electroencephalogram (EEG) data, conducts surveys of the user’s brain with a frequency of 250 Hz, for each of the electrodes with a set of 8 samples.

DETAILED DESCRIPTION OF THE INVENTION

The following detailed description of the invention sets forth numerous implementation details designed to provide a clear understanding of the present invention. However, it will be apparent to one skilled in the art how the present invention can be used with or without these implementation details. In other cases, well-known methods, procedures and components have not been described in detail so as not to unduly obscure the features of the present invention.

In addition, from the above discussion it will be clear that the invention is not limited to the above implementation. Numerous possible modifications, alterations, variations and substitutions, while retaining the spirit and form of the present invention, will be apparent to those skilled in the art.

This technical solution is intended for individual use and was created to analyze the psycho-emotional state of the user, as well as correct (relieve) negative conditions (with regular use).

The method for tracking the user’s psycho-emotional states and correcting them is implemented through the following technical elements:

brain-computer interface (can be implemented in various form factors) containing a neuro-amplifier;

a computing device that provides reception, filtering, analysis and processing of data coming from the neural amplifier;

cloud web platform, as well as a local database (all measurement results after the end of the session, along with raw EEG data, are sent to a remote web platform to track statistics and progress).

The basic principle of operation of a device for recording an EEG signal (neuro-computer interface) is the principle of measuring the potential difference between the target electrode and the reference electrode, with a certain sampling frequency, each measurement cycle (data from all electrodes) is collected into a separate sample and accumulated 8 samples are packaged into a BLE packet for transmission to the target device for further filtering, processing and classification.

The accuracy of measurements is achieved thanks to a system for filtering external interference - notch filters for cutting off external noise at frequencies of 50/60 Hz, a main filter based on a 4th order Butterworth filter for clearing the EEG signal from noise, as well as through the use of paradigms associated with individual alpha frequency -peak (iAPF) to select more accurate mechanics for tracking psycho-emotional states among different users.

In one of the implementations of the device for recording an EEG signal, the target electrodes are leads T3, T4, O1, O2, with a reference electrode on the forehead. The sampling frequency is 250 Hz. To start working, the device must be turned on with the button and the procedure for pairing the Bluetooth module, according to the user manual, with the target device (computing device), then put it on the head, making sure that the reference electrode touches the skin of the forehead, and the impedance values of the target electrodes in the application interface are less than 1 MOhm (megaohm). After this, you can switch to the EEG signal measurement mode.

In another implementation of a device for recording an EEG signal, the target electrodes are leads C3, C4, A1, A2, with the reference electrode at the CZ location (on the top of the head). The sampling frequency is 250 Hz. To start working, you need to turn on the device with the button and carry out the procedure for pairing the Bluetooth module, according to the user manual, with the target device - separately for the neuroamplifier, and separately for the stereo headset. Next, put it on your head, making sure that all electrodes are adjacent and the impedance values of the target electrodes are less than 1 MOhm. After this, you can switch to the EEG signal measurement mode. While working with the device, it is possible to use a stereo headset for its intended purpose - listening to music, making audio calls, in parallel with measuring the EEG signal.

Neurointerfaces support the following operating modes: resistance measurement - to determine the quality of electrode adhesion, sending the charge values of the built-in battery, and collecting an EEG signal. The devices do not provide any hardware filtering - processing is performed on the side of the computing device (Mind Tracker BCI).

Computing device is a hardware and software device that provides the user with obtaining information/data and/or executing the required operating algorithm and/or interacting with local or global communication networks (systems) and information systems (for example, a mobile device, laptop, etc.).

A computing device may consist of: information input/output devices (audio, graphic, text, video, etc.); sensors (position, orientation, illumination, accelerometers, proximity, etc.); communication modules (peripheral devices); kernels of a personal device (computer equipment (microprocessor, read-only memory, random access memory), graphics accelerators, audio stream processing tools).

The computing device carries out the necessary data processing using three main modules:

- The core is responsible for managing devices, receiving signals, converting, filtering and sending to the interface module, local database module and sending to the remote server of the web platform;

- Interface, a module responsible for displaying and entering user information on a computing device.

- Local database, a module that stores session data and sends it to a remote web platform server for storage and analysis.

The modules communicate with each other using the C API.

After starting the devices, upon request, a neural interface is searched for, ready to connect via Bluetooth. If such a neural interface has been found, an indication of the quality of electrode application is displayed. If the quality is satisfactory and the resistance values for each electrode are less than 1000 kOhm, it becomes possible to switch the device to the EEG recording mode.

In the EEG recording mode, the neural interface conducts surveys with a frequency of 250 Hz, for each of the electrodes, with a set of 8 samples, they are packaged in a BLE package and sent via a Bluetooth communication channel to a computing device.

To eliminate floating delays when transmitting data via the BLE protocol, the kernel module implements buffering functionality, with data output for processing every 0.1 seconds.

After receiving EEG data from the buffer, filtering and transformation algorithms are applied to them:

– fourth order Butterworth filter;

– fast Fourier transform (FFT) to obtain signal frequency and amplitude values;

– calculation of power spectral density (PSD) of the main brain rhythms – Theta, Alpha, Beta, including subbands.

All further measurements are carried out after additional power and amplitude filtering of myographic artifacts to isolate the useful physiological EEG signal. Filtration is carried out by a sliding window, 0.8 seconds wide. Only artifact-free values are taken into account in the calculations.

Classification of psycho-emotional states is possible after individual calibration, which takes 2 minutes. During the calibration process, the user is asked to sit quietly and go through a cycle: 20 seconds eyes closed, 20 seconds eyes open, 20 seconds eyes closed, 20 seconds eyes open, 40 seconds quiet wakefulness.

During the calibration process, the following parameters are measured: Artifact-free values of the powers of Alpha, Beta, Theta rhythms, the value of the individual peak of the Alpha rhythm (iAPF), individual boundaries of the Alpha range, powers of the subranges of the Alpha rhythm. Measurements are made in a sliding window of 5 seconds, followed by averaging - separately for the calibration stage with eyes closed, separately for the stage with eyes open.

In the last 40 seconds, the Alpha, Beta, Theta rhythms of the brain are measured, including the A1 and A2 subranges, and the base level characteristic of a given user is calculated (Baseline).

Baseline calculation: when calculating Baseline, the values of rhythm power and their ratios are accumulated in a cyclic buffer of 5 seconds, until a total of 30 seconds of artifact-free values are accumulated; the root mean square averaging method is used as an averaging method. The 75th percentile of the resulting number series is taken as the Baseline value.

After calculating Baseline, the following values are displayed in the user interface: iAPF value, and decoding of the values.

This technical solution allows the user to monitor their condition in real time in monitoring mode. The following metrics are available to the user on the display device:

– Cognitive Score (cognitive load level in percentage) – values in real time or in the form of a graph for a period of up to 60 minutes.

– Indication of psycho-emotional states: currently states are displayed in the form of indicators: Relaxation, Involvement, Mild fatigue, Moderate fatigue, Chronic fatigue, Anxiety, Stress. States become active as soon as the application algorithm detects them.

Based on the detected conditions, the application displays notifications with recommendations for completing biofeedback training (different for different negative conditions).

Examples of biofeedback training:

– Stress relief – training is offered to the user when conditions such as fatigue and stress are detected.

The training helps users relieve stress and experience deep relaxation, suitable for users who find it difficult to relax.

– Relaxation of the mind – training is offered to the user when conditions such as fatigue and anxiety are detected. The training allows the user to gradually learn to concentrate and not be distracted by external stimuli; it is suitable for users who find it easy to relax, but relaxation is easily interrupted.

The trainings have a similar structure: they consist of three levels, with each level the difficulty of passing increases, in each level you need to score 100 points. At the end of each training, the result is displayed to the user in the interface - the maximum number of points scored by the user for the training.

Algorithm for interaction between modules of the system for detecting psycho-emotional states and their correction (Mind Tracker BCI):

- The user interface acts as a client, allows you to establish communication and control the operating modes of the neuroamplifier. Displays information to the user.

- The core receives EEG data directly from the neuroamplifier, filters and preprocesses them; for basic filtering of incoming data, a fourth-order Butterworth filter and fast Fourier transform (FFT) are used to obtain power spectral density values, then the filtered and preprocessed data are sent to classification modules for calculating psycho-emotional and psychophysiological conditions. Ready metrics are sent to the user for display. Sends EEG data to a local database for storage. The kernel is a set of components that ensure the functioning and communication between interface modules, such as:

1. EEG signal classification module.

The module contains a receive buffer for Bluetooth data to provide a constant 4 ms latency (at 250 Hz sampling rate) when outputting data to the state classification modules. The module also contains filters for clearing the signal from interference and a module for calculating Band Powers (power spectral density values). After cleaning the signal and calculating the powers, the data goes to the state classification modules for further processing and output to a display device.

2. Communication module with brain-computer interface.

This module contains the logic for controlling the connection to the neural amplifier via a Bluetooth communication channel; the module is also responsible for sending commands to switch the operating modes of the neural amplifier (impedance measurement mode, data acquisition mode)

3. Module for sending data to the user.

This module contains the logic of the internal C API for communication between the classification module, the communication module with the brain-computer interface and the client application. The module is a set of API methods that allows you to issue and receive data, and receive commands from the application interface in a user-friendly form.

4. Local database, a module that stores session data and sends it to a remote web platform server for further storage and analysis.

This module contains a local SQL Lite database for recording EEG data and metrics received from classification modules in the absence of communication with the remote web platform server. The stored data contains: RAW EEG Data in HDF5 format, time-stamped event marking data, time-stamped classified state data, session metadata, including the neural amplifier model, number of channels, absolute start and end times of the session. The module also contains functionality for resending data when the connection to the Internet is lost.

Taking into account the “floating” delay in receiving data via the wireless protocol, the interface uses a buffer to collect data, thereby allowing for a uniform delay in transmitting EEG data for classification and its further transfer to the application interface.

An additional advantage of the system for detecting psycho-emotional states and their correction (Mind Tracker BCI) is its flexible architecture: it can be integrated with various interfaces and various signal classifiers.

Data filtering and preprocessing in the kernel (includes):

Application of a fourth-order Butterworth filter, which is used for basic filtering of incoming data;

Application of Fast Fourier Transform (FFT), which is used to calculate the power spectral density (PSD) of the main brain rhythms - Delta, Theta, Alpha, Beta;

Filtration is carried out by a sliding window, 0.8 seconds wide. Only artifact-free values are taken into further calculations.

During the calibration process, the following values are calculated to calculate psycho-emotional and psychophysiological states:

• the value of the individual peak of the Alpha rhythm (iAPF), is defined as the highest power in the measurement range of the Alpha rhythm at a certain frequency (in a particular embodiment of the described method, the value of the frequency of the individual peak of the Alpha rhythm (iAPF), is defined as the value of the frequency in the measurement range Alpha rhythm from 7 to 13 Hz, at which the power is greatest);

• the amount of suppression of the Alpha rhythm peak, defined as the difference in power between the absolute peak in the Alpha range during the calibration stages with open and closed eyes;

• individual boundaries of the Alpha range are defined as the first value (for the high-frequency alpha range), and the last value (for the low-frequency alpha range), where the difference in power with open and closed eyes will be positive;

• calculation of the base level of the cognitive fatigue index, concentration index, alpha gravity index, characteristic of a given user (Baseline).

Power spectral density (PSD) of the ranges of the main rhythms of the brain, as well as their subranges, are measured in a sliding window of 5 seconds.

To calculate the main indices, the following formulas are used:

Cognitive fatigue index (FS) = (α + θ) / β, where α is power in the Alpha rhythm range, θ power in the Theta rhythm range, β power in the Beta rhythm range.

Alpha Gravity Index (GS) = A2/A1, where A2 is the power in the Alpha rhythm subrange from the iAPF value to the upper limit, and A1 is the power in the Alpha rhythm subrange from the iAPF value to the lower limit.

Concentration index = B/A, where B is the power in the Beta rhythm subrange, and A is the power in the Alpha rhythm subrange

After calibration and calculation of an individual baseline level, metric values with interpretation are displayed to the user for monitoring.

Values are displayed: iAPF, Cognitive Score graph (scale from 0 to 100%), indication of psycho-emotional states.

iAPF – values from 7 to 13 Hz

Cognitive Score – graph of involvement in the presented task (productivity graph):

for low users (iAPF<10) is calculated based on the fatigue index, with correction factors;

for users with high alpha frequency (iAPF>10) is calculated based on the Alpha Gravity (GS) index, with correction factors.

Indication of psycho-emotional states is implemented in the form of indicators: Relaxation, Involvement, Mild fatigue, Moderate fatigue, Chronic fatigue, Anxiety, Stress. Status indicators become active as soon as the system algorithm detects them.

The algorithm recognizes each state individually, based on the index values described above.

• States of relaxation and engagement are identified with high iAPF and low FS values, relative to Baseline.

• Mild, moderate and chronic fatigue states are identified with lower iAPF and higher FS values relative to Baseline.

• States of anxiety and stress are identified at low iAPF values and high stress index values in relation to Baseline indicators.

The mechanics are based on auditory feedback, depending on maintaining the level of its Alpha peak, determined at the calibration stage (the amount of Alpha rhythm suppression). When maintaining relaxation, the sounds of nature are clearly pronounced, but when falling, extraneous sounds of artificial origin are mixed in with them (for example, the sounds of a highway, a construction site, etc.)

All measurement results (sessions) are sent, stored in a local Postgress SQL database and after the end of the session, together with raw EEG data, are sent to a remote web platform, which allows you to track statistics and progress when reusing the system.

The proposed technical solution allows you to track metrics and states “in the moment”, as well as sessionally - keeping statistics by day, week, month.

If negative psycho-emotional states are detected in the user, recommendations are displayed on the display device to help get rid of such states (recommendations for planning a work/rest mode, or recommendations for completing training).

Additionally, the solution allows you to build an individual predictive model and notifies the user about the most favorable periods of work/rest, based on accumulated data.

This solution uses an algorithm for cleaning the EEG signal from myographic (muscle) artifacts, which allows the proposed technical solution to be used outside laboratory conditions.

Although the invention has been described with reference to the disclosed embodiments, it will be apparent to those skilled in the art that the specific experiments described in detail are provided for purposes of illustrating the present invention only and should not be construed as limiting the scope of the invention in any way. . It should be understood that various modifications can be made without departing from the spirit of the present invention.

Claims (18)

1. A method for tracking the user’s psycho-emotional states and correcting them, including the stages of: a portable neural interface, in the mode of collecting electroencephalogram (EEG) data, conducts surveys of the user’s brain; the collected EEG data of the user is transmitted via a Bluetooth communication channel to a computing device for further processing; the computing device receives EEG data from the neural amplifier, after which it filters and preprocesses it using a fourth-order Butterworth filter; filtered and preprocessed data are classified, and to calculate psycho-emotional and psychophysiological states and their further detection and correction, an individual calibration mechanism is used, during which the user goes through a cycle: 20 seconds closed eyes, 20 seconds open eyes, 20 seconds closed eyes, 20 seconds open eye, 40 seconds of quiet wakefulness; in this case, during the calibration process, the following values are calculated to calculate psycho-emotional and psychophysiological states: the value of the individual peak of the Alpha rhythm (iAPF); the amount of suppression of the Alpha rhythm peak; individual boundaries of the Alpha range; calculation of the basic level of the cognitive fatigue index, concentration index, alpha gravity index, characteristic of a given user (Baseline), and when calculating Baseline, the values of the power of rhythms and their ratios are accumulated in a cyclic buffer of 5 seconds, until a total of 30 artifact-free seconds are accumulated values, and the root mean square averaging method is used as the averaging method; after calibration and calculation of the individual baseline level, the metric values with interpretation are displayed to the user for monitoring; Based on detected conditions, notifications are displayed to the user with recommendations for completing biofeedback training. 2. The method according to claim 1, in which filtering and preprocessing of data is carried out using a fourth-order Butterworth filter, which is used for basic filtering of incoming data; by applying the fast Fourier transform (FFT), the power spectral density (PSD) of the main brain rhythms - Delta, Theta, Alpha, Beta is calculated; By using power and amplitude filtering, it is possible to get rid of myographic artifacts and isolate the useful physiological EEG signal. 3. The method according to claim 1, in which during the calibration process the following values are calculated to calculate psycho-emotional and psychophysiological states: • the value of the individual peak of the Alpha rhythm (iAPF) is defined as the highest power in the measurement range of the Alpha rhythm; • the amount of suppression of the Alpha rhythm peak is defined as the difference in power between the absolute peak in the Alpha range during the calibration stages with open and closed eyes; • individual boundaries of the Alpha range are defined as the first value for the high-frequency alpha range, and the last value for the low-frequency alpha range, where the difference in power between eyes open and eyes closed will be positive. 4. The method according to claim 3, in which the value of the individual peak frequency of the Alpha rhythm (iAPF) is defined as the frequency value in the Alpha rhythm measurement range from 7 to 13 Hz, at which the power is greatest. 5. The method according to claim 1, in which all measurement results are saved in a local database and after the end of the session, together with raw EEG data, are sent to a remote web platform to track statistics and progress. 6. The method according to claim 1, in which the portable neural interface, in the mode of collecting electroencephalogram (EEG) data, conducts surveys of the user’s brain with a frequency of 250 Hz, for each of the electrodes with a set of 8 samples.