RU2740256C1 - System and method for determining psychoemotional states based on biometric eeg signal - Google Patents

Abstract

FIELD: medicine.

SUBSTANCE: group of inventions relates to medicine, specifically to a method and a system for determining psychoemotional states based on an EEG signal. In the method implementation, the EEG signal is obtained from the user's neurointerface by the signal reception unit. Primary treatment is performed by spectral decomposition of the obtained signal into cerebral rhythms by means of a primary signal processing unit. Primary processing unit is connected to the signal reception unit. Signal is checked for presence of interference and aggregation of primary transformed signals with selection of brain rhythms corresponding to at least one preset state by means of filtration unit of primary processed signals. Filtering unit is connected to signal primary processing unit. Predetermined state is determined by the preset state determining unit configured to determine the correlate of that state. Unit for determination of preset state is connected to filtration unit. Boundaries of the normal state for the given user are determined by statistical analysis of historical values by the state of the user based on the obtained EEG signal using the personalization unit of the given state. Personalisation unit is connected to unit for determination of preset state. Data are output by providing the user with the obtained results using the data output unit. Output unit is connected to personalization unit.

EFFECT: higher reliability of determining the current psychoemotional state by improving the accuracy of processing the obtained data and the extended interpretation of the data on the patient's cerebral activity in real time using automatic personalization based on accumulation of statistical parameters collected by an individual user of those invariant relative to particular user signal characteristics, which were detected during the research.

8 cl, 45 dwg

Description

The present group of inventions relates to the field of psychophysiology, in particular to user diagnostics, and can be used to register and analyze electrical signals of the human brain to determine various psycho-emotional states of the user.

Among psycho-emotional states, such basic states are distinguished as enthusiasm or monotony, cognitive load, stress, fatigue or vigor, concentration, as well as a resource state due to a combination of states of enthusiasm, stress and concentration.

The claimed technical solution involves the determination of the selected indicators of any psycho-emotional state, the determination on their basis of indices characterizing various aspects of the state of users, the determination of some boundary values of the indices or their combinations (triggers) and the generation of diagnostic messages to users issued to various external devices (monitor screen, mobile device).

Modeling the states of enthusiasm or monotony in the user is based on determining the nature of his activity. In this case, enthusiasm is understood as a state corresponding to "creative activity" - an interesting activity that allows an employee to maintain an efficient state. Monotony is understood as a state corresponding to "routine activity" - an activity associated with stereotypical, constantly repetitive mental operations leading to a decrease in performance.

Assessment of the level of cognitive load is one of the important aspects and significant factors that determine the effectiveness of human activity. A high level and excess of cognitive load leads to a sharp decrease in learning ability and a weakening of understanding, an increase in the likelihood of errors when completing tasks. At the moment, the following forms of cognitive load are distinguished [Sweller J. Cognitive load during problem solving: Effects on learning. Cognitive science, 12 (2), 257-285. [Magazine]. - 1988]: internal load (intrinsic load), external load (extraneous load), corresponding load (germane load). Internal cognitive load is determined by the interaction between the nature of the object of activity and the person's experience. The working memory load depends on the number of elements that must be processed simultaneously in the working memory, and the number of elements that must be processed simultaneously, in turn, depends on the degree of interactivity of the element. An element in this situation is any object under study. The more elements are included in the studied subject, the more difficult it is to study. External cognitive load reflects efforts to overcome the form of information presentation (visual, auditory, etc.). The external load is largely determined by the design of the materials used in the implementation of the activity. For example, the external load when explaining the features of geometric shapes will be lower in the graphical presentation of the objects under study and higher in the auditory one. The corresponding load reflects the costs of the human brain for the creation and translation of the interaction patterns of the object's elements from working to long-term memory.

The goal of performance optimization in accordance with the theory of cognitive load is to reduce external cognitive load, increase the corresponding load and manage internal user load [Ayres P. Using subjective measures to detect variations of intrinsic cognitive load within problems. Learning and Instruction, 16 (5), 389-400. [Magazine]. - 2006].

There are various methods for assessing cognitive load. Cognitive load can be assessed as a decrease in the tempo characteristics of activities and the quality of work performance. However, this approach requires complex laboratory testing and cannot be used in a real work or educational process.

At the present stage, the most common empirical approach to assessing cognitive load. The empirical approach is aimed at assessing the mental effort and performance of a person based on the collection of subjective data on mental efforts using rating scales and subsequent measurement of psychophysiological parameters of activity.

Currently, approaches are being developed to assess the cognitive load associated with the use of instrumental monitoring of psychophysiological parameters of cognitive processes: based on oculographic indicators [M.A. Shurupova, A.V. Krasnoperov, L.V. Tereshchenko, A.V. Latanov Influence of cognitive tasks on the parameters of eye movements when viewing static and dynamic scenes. C. 202-212] and, in particular, based on the assessment of indicators of the total electrical activity of the brain (EEG). Within the framework of the latter approach, a sufficient number of works have been carried out that show that cognitive load is correlated with the spectral characteristics of the EEG, mainly in the ranges of theta (4-7) and lower frequencies of alpha (8-10 Hz) rhythms [Klimesch W. EEG alpha and theta oscillations reflect cognitive and memory performance: a review and analysis. Brain research reviews, 29 (2-3), 169-195. [Magazine]. - 1999, Polikanova I. S., & Sergeev, A. V. Influence of long-term cognitive load on EEG parameters. National Psychological Journal, (1 (13)). [Magazine]. - 2014].

In addition, research in the field of assessing cognitive load has shown the relationship between the subjective assessment of the state of users and objectively measured psychophysiological indicators. Thus, a relationship was found between cognitive load and parameters of electroencephalography (EEG), motor activity of the eyes (DA), galvanic skin response (GSR).

Changes in human performance may be associated not only with cognitive characteristics or a general change in functional state, but also with an excessive increase in the level of activation (in accordance with the Yerkes-Dodson law on the optimal level of activation) under conditions of short-term stress, as well as with a decrease in motivational component of activity in case of prolonged stress. Traditionally, the objective level of stress is measured by recording indicators of the autonomic nervous system and analyzing the balance of the contribution of sympathetic and parasympathetic activation. It is also possible to measure the level of stress in its relationship with the emotional state, measured on the basis of EEG activity.

Findings from numerous studies reflect an emerging consensus that left frontal area activity is associated with general motivational system and behavioral activation, while right frontal area activity is associated with general avoidance or withdrawal cessation tendencies. This led to the formulation of Davidon's highly influential motivation model - the motivational model of emotion. According to this model, left frontal activity indicates a tendency to strive for a stimulus, while a relatively large right frontal activity indicates a tendency to avoid a stimulus.

Early research suggests that frontal EEG asymmetry can serve as a moderator involved in predicting emotional responses in infants. For example, Davidson and Fox found that babies who cried in response to the separation of the mother had more activity in the right frontal region at rest than those who did not. This result was replicated by Foxetal., Who also found the effect to be moderately stable over 5 months.

Similar results were obtained in adults. For example, at the request of Tomarken et al. To report emotional reactions at more intense levels of negative impact on negatively-rated clips, especially those involving fear. In a similar report, Wheeler et al. Found that individuals with more right frontal activity showed more intense negative responses to negative images, people with more left frontal activity exhibited more intense positive responses to positive images. These studies show that individual differences in emotional response are partly due to individual differences in EEG asymmetry. Indeed, Davidson cited this evidence to argue that affective style, as evidenced by his asymmetry of the frontal EEG, may partially determine the risk of certain affective disorders such as depression and anxiety, a proposition that clearly identifies frontal EEG asymmetry as a moderator of emotions and related with them affective processes.

The degree of tension of regulatory systems is an integral response of the body to the entire complex of factors affecting it, regardless of what they are associated with. Under the influence of a complex of factors of an extreme nature, a general adaptation syndrome arises, which is a universal response of the body to stressful influences of any nature and this syndrome manifests itself in the same type in the form of mobilization of the body's functional reserves. A healthy organism, possessing a sufficient reserve of functional capabilities, responds to stressful effects with the usual, normal, so-called working tension of the regulatory systems. So, for example, if we have to climb the stairs, then, naturally, energy consumption increases and additional resources need to be mobilized. However, for some people, such mobilization is not accompanied by a significant tension of the regulatory systems, and the pulse rate when climbing, for example, to the 5th floor, increases by only 3-5 beats, i.e. cardiovascular homeostasis practically does not change. For other people, this load is too great and there is a pronounced tension of the regulatory systems with an increase in heart rate by 15-20 or more beats: which already indicates the presence of violations of homeostasis.

Even under conditions of rest, the tension of regulatory systems can be high if a person does not have sufficient functional reserves. This is expressed, in particular, in the high stability of the heart rate, which is characteristic of the increased tone of the sympathetic division of the autonomic nervous system. This department of the regulatory mechanism, responsible for the emergency mobilization of energy and metabolic resources in all types of stress, is activated through the nervous and humoral channels. It is a component of the hypothalamic-pituitary-adrenocorticotropic system, which implements the body's response to stress. An important role in this belongs to the central nervous system, which coordinates and directs all processes in the body.

Already from the very sound of the term "mental" fatigue, it is clear that the dynamics of changes in this state can be associated with the activity of the brain. Among the methods of recording brain activity for the tasks of assessing the functional states of a person, the most pragmatic is the registration of electroencephalography - the total electrical activity of the brain - in connection with the possibility of such registration outside laboratory conditions. Today, technologies for registration of mobile EEG are actively developing, some of which are available as a market product for non-specialists.

EEG-based approaches to assessing mental fatigue are associated with a variety of ideas about the cerebral mechanisms of mental fatigue. One of the approaches developed by Jesper Hopstaken (Hopstaken, 2016) is related to the assumption that mental fatigue is associated with the activity of norepinephrine neurons in the nucleus accumbens, which regulate the involvement in the task. Despite the fact that it is impossible to non-invasively measure the activation of the norepinephrine systems of the brain, there are indicators of total electrical activity that can be indirectly considered as their correlates. The P3 component of the EEG event-related potential (ERP) is considered as the main indicator of this type. The arguments in favor of such an interpretation of P3 were obtained in the works of Nieuwenhuis and colleagues (2011), who directly linked the positive shift in the P3 range with the activity of the norepinephrine system of the nucleus accumbens. It is known that the P3 component can be subdivided into P3a and P3b subcomponents associated with the novelty of the stimuli presented and the motivational characteristics of the task, respectively (Polich & Kok, 1995; Polich, 2007). Begleiter, Porjesz, Chou and Aunon (1983) have suggested that the parietal component of P3 may reflect the motivational properties of the stimulus. It was later found that the effect of motivational value on P3 amplitude is also modulated by the amount of attention given to the stimulus (Johnson, 1993). The combination of sensitivity with both the motivational and attentive aspects of the task has prompted researchers to associate P3 with task characteristics (eg, Murphy, Robertson, Balsters, & O'connell, 2011). The presence of a motivational component in the P3a component also fits into the idea of its connection with the norepinephrine system of the brain.

A more common model is that mental fatigue in the brain is associated with the activity of the brain's dopaminergic systems responsible for assessing reward from current activity. The role of trade-offs in cost-reward ratios has been highlighted in fatigue studies led by Maarten Boksem (eg, Boksem & Tops, 2008). According to his theory, mental fatigue occurs when the cost of participating in a task exceeds predicted rewards. Consistent with this concept, increasing task rewards when fatigued can cause shifts in motivation that can draw attention to tasks. This effect has been confirmed in previous studies (Boksem et al., 2006). They showed that after two hours of cognitive activity, fatigued participants can partially recover their performance if they receive a monetary reward. This effect was accompanied by an increase in the component of the event-related potential (ERP) of the EEG, which was called error-related negativity (ERN).

The ERN component consists of a large negative shift in ERP that occurs after subjects have made an erroneous response (Falkenstein et al., 1990; Gehring et al., 1990) and is generated in the anterior cingulate cortex (ACC) (Dehaene et al., 1994 ), a structure central to processing information about reward, punishment, and required effort. Holroyd and Coles (2002) hypothesized that ERN is the result of a phase decrease in the activity of mesenfacilic dopaminergic neurons after an error in predicting reward (ie, when reward is less than expected). This decrease in the activity of dopaminergic neurons in the striatum, in turn, results in disinhibition of the apical dendrites of motor neurons in the ACC, creating an ERN (Holroyd and Yeung, 2003). In addition, many studies have shown that ERN is associated with a motivational component: when the requirements for performing the best work are reduced, a decrease in the amplitude of the ERN can be observed (Gehring et al., 1993). Indeed, motivation appears to be necessary to observe robust ERNs (Gehring et al., 1993; Gehring and Knight, 2000). Bush et al. (2000) argued that the ERN component and its associated ACC activities represent a general scoring system that processes the motivational meaning of events. According to Sarter, Gehring, and Kozak (2006), mental effort associated with the same neural system is able to maintain robust cognitive functioning even under suboptimal conditions, given that motivation is high. Besides motivation, energy costs are central to this structure. Loss of energy (exhaustion) and loss of motivation due to uncertainty about task duration and boredom can add to the overall picture of mental fatigue.

The characteristics of electrical activity described above, despite their knowledge in laboratory conditions, are only of limited use in the case of pragmatic tasks aimed at recording the objective characteristics of mental fatigue in real-life conditions. The difficulty in using these characteristics is due to the fact that ERPs are recorded only under certain conditions and are reliably distinguished only when the tasks of the same type are presented repeatedly and their exact synchronization with the EEG.

The user's state of concentration implies concentration of attention (one of the indicators of sustained attention). Attention can be described in terms of various theoretical models. One of them is the general model of Knudsen (2007), which assumes four key (nuclear) processes of attention, which include: storage of working memory; competitive selection, which determines what information gets access to working memory; top-down control, which adjusts the signal intensity in the channels, and thus gives preference to certain types of information in accessing the working memory; upward filters that automatically increase the response to infrequent stimuli or stimuli important for the implementation of instincts or learned biologically significant behavior (exogenous attention). Another well-known model of attention was developed by M. Posner: attention consists of several different-level systems - excitation-vigilance, orientation (posterior associative system), executive control (anterior associative attention system) (Posner, 2004). If the systems of excitation and orientation are associated with automatic processes, then the control control - with arbitrary. The posterior associative attention system covers the parietal lobe of the cerebral cortex, the thalamus, and the areas of the midbrain associated with eye movement, and reflects the processes of visual-spatial attention. Managing control constitutes one of the systems of attention, responsible for the selection of information, coordination and execution of relevant processes and suppression of irrelevant ones. According to M. Poznner, this system is realized during the activation of working memory, overcoming conflicts. Thus, this system can be considered as a necessary apparatus for maintaining concentration. The functioning of the executive control involves the work of the anterior cingulate gyrus, the medial and ventrolateral prefrontal cortex, and the basal ganglia.

The user's resource state is determined on the basis of the parameters of states of stress, enthusiasm or monotony and concentration and is an integral indicator of the activation of the nerve centers that accompany the current human activity.

A known method for assessing the emotional state of a person [invention patent RU 2291720, publ. 20.01.2007], which consists in presenting to the patient four groups of verbal characteristics reflecting various emotional states and the degree of their severity. Then the verbal characteristics selected by the person, reflecting his condition, are analyzed using a scale of points. The method allows to give a quantitative assessment of the degree of emotional maladjustment and allows the subject to independently realize the nature of the emotions experienced.

The disadvantage of this method is the low degree of automation of data collection and processing and the reliability of the information obtained due to the subjective method of data collection.

The known system and method of training [application for invention US 20170330475, publ. 11/16/2017], which uses electroencephalogram (EEG) data to analyze the results of test execution and determine whether it is necessary to take a break from educational activity or, on the contrary, complicate the task being performed. The learning system includes an output unit that displays the first problem and a message on the display prompting the user to take a break, a collection unit that receives a response to the first problem from the user, an EEG measurement unit that measures the user's EEG, and a control unit. The control unit determines whether the first motivation is present based on the potential associated with the first event included in the EEG and, from the moment the first problem is displayed, determines whether the second motivation of the user is present based on the potential associated with the second event included in the EEG, and starting from the moment at which the response was received, and instructs the output unit to display a message on the display prompting the user to take a break if the first motivation is absent and the second motivation is not.

The disadvantages of the known system are the limited scope (only education), lack of information about the current psycho-emotional state of the user, a limited set of recommendations issued (take a break).

A known system and method for modulating content based on data on electrical signals from the human brain [invention application US 2018278984, publ. 09/27/2018], including changing the presentation of digital content on at least one computing device. The content can also be modulated based on a set of rules supported or available to the computer system based on user input, including by receiving a presentation control command, which can be processed by the computer system to modify the presentation of the content. Content can also be transmitted with associated brain state information. The computer system can process the biosignal data using the biological signal processing pipeline to determine at least one state of the user's brain. It can be determined that the brain state data occurred in response to the presentation of certain digital content to the user. Hence, the computer system can associate a certain state of the user's brain with the presented digital content. Instead of, or in addition to, performing this association, the computer system may modify the representation of digital content on at least one computing device based at least in part on the received biosignal data and on at least one representation modification rule associated with the presented digital content. The application of any such rules can be used for processing by the modulator component that interacts with the content source.

The disadvantages of the known technical solutions are the limited scope of application (only the media sphere), linking the state of the brain only with the presented digital content, narrow interpretation of data on the user's brain activity.

The present group of inventions is aimed at achieving a technical result consisting in increasing the reliability of determining the current psychoemotional state by increasing the accuracy of processing the obtained data and expanded interpretation of data on the user's brain activity in real time.

The claimed technical result in terms of the system is achieved due to the fact that the system for determining psychoemotional states based on the EEG signal includes a signal receiving unit configured to receive an EEG signal from the user's neurointerface, to which the primary signal processing unit is connected, configured to spectrally decompose the received signal. signal to the rhythms of the brain, to which the filtering unit of the primary processed signals is connected, made with the possibility of checking the signal for the presence of interference and aggregation of the primary converted signals with the selection of the brain rhythms corresponding to at least one specified state, to which the determination unit is connected at least one specified state, configured to determine the correlate of this state, to which the personalization unit of a given state is connected, configured to determine the boundaries of the normal state for a given user by statistically analysis of historical values by the state of the user, with which the data output unit is connected, made with the ability to provide the user with the obtained results.

There are options for the development of the main technical solution:

- block of primary signal processing is configured to implement preliminary smoothing of the received signal;

- the block for filtering the primary processed signals allows you to select the brain rhythms corresponding to states of enthusiasm or monotony, cognitive load, stress, vigor or fatigue, concentration, and the block for determining a given state allows you to determine the correlate of these states;

- introduced a unit for fixing deviations from the user's normal state, connected to the state personalization unit and the data output unit and made with the possibility of determining the user's presence in a given state.

The claimed technical result in terms of the method is achieved due to the fact that the method for determining psychoemotional states based on the EEG signal includes receiving an EEG signal from the user's neurointerface by means of a signal receiving unit, its primary processing by spectral decomposition of the received signal into cerebral rhythms using a primary signal processing unit , checking the signal for the presence of interference and aggregation of the primary transformed signals with the extraction of brain rhythms corresponding to at least one predetermined state using the filtering unit of the primary processed signals, determining at least one predetermined state by means of the predetermined state determination unit configured to determine the correlate of this state, determination of the boundaries of the normal state for a given user by means of statistical analysis of historical values according to the user state using the block of personalization of a given state, output yes data by providing the user with the results obtained using the data output unit.

There are options for the development of the main technical solution:

- at the stage of primary processing of the EEG signal, preliminary smoothing of the received signal is performed;

- at the stage of brain rhythms aggregation, rhythms corresponding to states of enthusiasm or monotony, cognitive load, stress, vigor or fatigue, concentration are distinguished, and then the correlates of these states are determined;

- after determining the boundaries of the normal state for a given user by means of statistical analysis of historical values according to the user's state and before outputting the obtained data, it is determined that the user is in a given state using a block for fixing a deviation from the user's normal state.

Thus, due to the claimed sets of features of the system and method, the reliability of determining the current specified state of the user is increased by increasing the accuracy of processing the received data and expanded interpretation of data on the user's brain activity in real time. This became possible due to the aggregation of the primary transformed signals with the extraction of brain rhythms corresponding to at least one predetermined state using the filtering unit of the primary processed signals, and the subsequent determination of at least one predetermined state by means of the predetermined state determination unit configured to determine the correlate of this state ... At the same time, the scope of application of the declared technical solutions (education, interviews, content assessment, increase in labor productivity) and the volume of recommendations provided were expanded. Assessment of a given state is carried out without reference to content, at any time, in various activities. With the help of the claimed system and method, any psychoemotional states can be assessed. As examples, this application considers the main states: enthusiasm or monotony, cognitive load, stress, fatigue or vigor, concentration, resource state.

The essence of the proposed group of inventions is disclosed in more detail using the figures and further description.

FIG. 1 is a block diagram of the system.

FIG. 2 shows the initial data for determining the state of enthusiasm or monotony.

FIG. 3 shows the initial data for determining the state of cognitive load.

FIG. 4 shows the initial data for determining the state of stress based on the EEG signal and EDA.

FIG. 5 shows the initial data for determining the state of fatigue or vigor.

FIG. 6 shows the initial data for determining the state of concentration.

FIG. 7 shows a block diagram of a block for determining the state of enthusiasm or monotony.

FIG. 8 shows a summary of the calculated indices in block 5 for determining the state of enthusiasm or monotony.

FIG. 9 shows a summary of the triggers used in block 5 for determining the state of enthusiasm or monotony.

FIG. 10 shows a list of messages in block 5 for determining the state of enthusiasm or monotony.

FIG. 11 is a block diagram of a model for determining states of entrainment or monotony of a user.

FIG. 12 shows a block diagram of a block for determining the state of cognitive load.

FIG. 13 shows a summary of the calculated indices in block 5 for determining the state of cognitive load.

FIG. 14 provides a summary of the triggers used in block 5 for determining the state of cognitive load.

FIG. 15 shows a list of messages in block 5 for determining the state of cognitive load.

FIG. 16 shows a block diagram of a model for determining the state of the user's cognitive load.

FIG. 17 shows a block diagram of a block for determining the state of stress based on the EEG signal and electrodermal activity.

FIG. 18 shows a summary of the calculated indices in block 5 for determining the state of stress.

FIG. 19 provides a summary of the triggers used in the stress determination block 5.

FIG. 20 shows a list of messages in block 5 for determining the state of stress.

FIG. 21 is a block diagram of a model for determining user stress states.

FIG. 22 is a block diagram of a fatigue or vigor state determination unit.

FIG. 23 shows a summary of the calculated indices in block 5 for determining the state of fatigue or vigor.

FIG. 24 provides a summary of the triggers used in block 5 for determining the state of fatigue or vigor.

FIG. 25 shows a list of messages in block 5 for determining the state of fatigue or vigor.

FIG. 26 is a block diagram of a model for determining a user's fatigue or alertness states.

FIG. 27 is a block diagram of a concentration state determination unit.

FIG. 28 shows a summary of the calculated indices in block 5 for determining the state of concentration.

FIG. 29 is a summary of the triggers used in the concentration state determination block 5.

FIG. 30 is a list of messages in block 5 for determining the state of concentration.

FIG. 31 is a block diagram of a model for determining user concentration states.

FIG. 32 shows a block diagram of the block 5 for determining the resource state.

FIG. 33 shows a summary of the calculated indices in block 5 for determining the resource state.

FIG. 34 is a block diagram of a model for determining the resource state of a user.

FIG. 35-39 shows the results of the study to determine the resource state.

FIG. 40 shows the results of the study to determine the state of cognitive load.

FIG. 41-45 summarize the results of the study to determine the state of stress.

The system for determining psycho-emotional states based on the EEG signal (Figs. 1 and 2) of the user 1 includes serially connected to each other a signal receiving unit 2, a primary signal processing unit 3, a filtering unit 4 of primary processed signals, a unit 5 for determining at least one specified state , block 6 data output.

Additionally, the system may comprise a personalization unit 7 for at least one state connected to a unit 5 for determining a predetermined state and to a data output unit 6, as well as a unit 8 for recording a deviation from the normal state of user 1, connected to a state personalization unit 7 and a data output unit 6 ...

The signal receiving unit 2 is configured to receive an EEG signal from the user's neurointerface 1, captured using a neuroheadset. The neuro-headset provides registration of EEG with 7 derivations and motor activity of the head (including blinking), as well as broadcasting these data wirelessly to the receiving unit 2 via a USB-BLE adapter connected to the USB connector of a personal computer (PC).

Block 3 for primary signal processing is configured to spectrally decompose the received signal into the rhythms of the brain and is a converter of the original EEG wave into harmonic oscillations according to the Fourier algorithm with the determination of the absolute and relative spectral powers of each of the given frequency ranges. Also, block 3 can be configured to implement preliminary smoothing of the received signal by checking for excess of the specified signal amplitude with subsequent removal of the recording area in which the amplitude was exceeded.

The block 4 for filtering the primary processed signals is configured to check the signal for the presence of interference by determining the amplitude burst based on statistical analysis and aggregation of the primary transformed signals with the extraction of brain rhythms corresponding to at least one given state.

So, for example, block 4 for filtering the primary processed signals allows you to select the rhythms of the brain corresponding to the states: enthusiasm or monotony - left hemispheric theta rhythms, right hemispheric alpha and theta rhythms; cognitive load - left hemispheric theta rhythms; stress - left hemisphere and right hemispheric alpha rhythms; cheerfulness or fatigue - left hemispheric alpha and theta rhythms; concentration - left hemispheric beta rhythms.

Methods of analysis of electrocardiograms (ECG) and EEG were taken to be used in the developed model for determining the nature of intellectual activity and determining states of enthusiasm and monotony.

When analyzing an electroencephalogram, signals from leads from points P4, Cz, F3 are used according to the 10-20 system. Signals are measured in millivolts (mV).

FIG. 2 shows the initial data for determining the state of enthusiasm or monotony.

EEG registration is used as a method for assessing the electrical activity of the brain within the framework of the developed model for determining cognitive load. The signal flow from the electrode at point F7 according to the "10-20" system is taken as an indicator of the electrical activity of the brain in the developed model.

A simple but effective method for assessing a person's condition based on the registration of oculomotor activity involves analyzing the frequency of a person's blinking over a certain period of time. This indicator is a common, easily observable and readily available phenomenon that reflects the activity of the central nervous system. Another easily measurable indicator is the duration of the blink. This metric is considered particularly useful for predicting sleepiness that accompanies a loss of productivity. Therefore, in the case of determining the state of cognitive load, block 4 allows you to additionally determine the parameters of blinking.

Thus, two indicators of oculomotor activity are applied:

- The number of blinks;

- Blink duration.

The list and description of the input (recorded) data for determining the state of cognitive load are shown in Fig. 3.

When analyzing the EEG, to determine the state of stress based on the EEG signal, signals from leads from points F3, F4, F6, F7 are used according to the 10-20 system. Signals are measured in millivolts (mV).

Methods for the analysis of galvanic skin response (GSR or EDA) and EEG were taken to be used in the model for monitoring the level of stress based on the EEG signal and electrodermal activity and to optimize stress factors in the process of intellectual activity.

At the same time, a unit (not shown in the drawing) for receiving signals of electrodermal activity is additionally introduced, connected to the unit 3 for primary signal processing and made with the possibility of receiving signals of electrodermal activity from the bracelet worn on the hand of the user 1 and providing transmission of these data via a wireless communication channel to this block.

The measured parameter in the analysis of skin electrodermal activity (EDA) is the electrical conductivity of the skin between two electrodes. This indicator can be measured in two ways:

1) passing a weak current from an external source through the skin and measuring the dynamics of its electrical resistance. This technique is called the exosomatic method;

2) Measurement of the electrical activity of the skin surface without using an external current source. This is an endosomatic method.

EDDA is recorded using electrodes placed on the palm or fingers. Signal magnitude is measured in millivolts (mV).

FIG. 4 shows the initial data for determining the state of stress based on the EEG signal and EDA.

Among the EEG indicators that can be recorded outside the conditions of specially specified actions, it is necessary to highlight the spectral characteristics in different frequency ranges. Thus, it has been shown that the spectral characteristics of the EEG can act as valid indicators of mental fatigue (Lal & Craig, 2002). In general, a shift in the EEG towards slow-wave activity simultaneously with a decrease in the contribution of high-frequency components was repeatedly associated with a decrease in the functional state of a person (Aeschbach et al., 1997; Cajochen, Brunner, Krauchi, Graw, & Wirz-Justice, 1995; Phipps-Nelson, Redman, & Rajaratnam, 2011; Zhao, Zhao, Liu, & Zheng, 2012, Lal & Craig, 2001b). Such changes are associated with a general change in the activation of the organism during fatigue (Tops & Boksem, 2010), as well as with the transition to a state of dormancy (De Gennaro et al., 2007; Loomis et al., 1937; Santamaria and Chiappa, 1987). Changes in theta activity are most pronounced in the prefrontal areas of the cortex, which may be associated with the processes taking place in the anterior cingulate gyrus. These changes are also associated with changes in the motivational characteristics of activity, which fits into the model of Maarten Boksem and colleagues (Boksem & Tops, 2008). These data are consistent with the data obtained in the study of Trejo et al. (2015) based on the classification of EEG states by spectral characteristics using machine learning, where the authors observed the relationship between changes in subjective characteristics of fatigue with changes in spectral power in theta range in the prefrontal areas of the brain.

Thus, the initial data for determining the state of fatigue and antipode - vigor are the data on the slow-wave EEG activity from the electrode at point AF7 according to the MCN system, shown in Fig. 5.

Neurophysiological characteristics of concentration can be reflected in various EEG indicators. Spectral characteristics reflecting the concentration of attention on a particular object can include high-frequency indicators, for example, gamma waves in the range over 40 Hz. They are generated in the parieto-frontal regions of the brain, which show synchronization of neuronal activity in these regions (Kaiser and Lutzenberger, 2003; Siegel et al., 2008; Gregoriou et al., 2009; Baldauf, Desimone, 2014). The involvement of the parietal regions is due to the activation of the dorsal flow: the “Where?” System, which is responsible for processing information about the spatial position of an object, and, as a consequence, participates in active tracking of the object (concentration of attention on it) (Mishkin and Ungerleider, 1992). The involvement of the frontal regions is due to the activation of controlling processes involved in organizing the work of the parietal cortex. Another high-frequency indicator of attention concentration can be the power in the beta range (12.5-30 Hz), which appears in the frontal regions of the brain and spreads during the activation of attention to other areas (Jensen et al., 2005). Beta trainings have been shown to increase the performance of attention-focused tasks in people with attention deficit hyperactivity disorder (ADHD) (Linden et al, 1996; Egner, Gruzelier, 2004; Kropotov et al., 2005). It has been reported that deactivation of the beta rhythm may be a mechanism for impaired attention in ADHD (Clarke et al., 2001). One of the possible mechanisms that determine the involvement of beta activation in attention processes may be genetic: for example, polymorphisms of the DAT gene (dopamine transporter gene) (Loo et al., 2003). This is consistent with the so-called “dopamine hypothesis,” suggesting the presence of the necessary level (optimum) of dopamine, which is required for the effective implementation of not only attention, but also other cognitive functions through neurons in the prefrontal cortex (Schacht, 2016). Thus, activity in the beta range can be a biomarker of the state of concentration (stability) of attention associated with maintaining the required level of dopamine.

Thus, the initial data for determining the state of concentration was selected an indicator of high-frequency EEG activity in the beta range from the electrode at point AF7 according to the MCN system, shown in Fig. 6.

The initial data for determining the resource state are the initial data for determining the states of enthusiasm, stress and concentration.

Unit 5 for determining at least one predetermined state or combinations of predetermined states is configured to determine the correlate of this / these states. Block 5 is a programmable module configured to perform a series of computational steps.

If it is necessary to determine the state of enthusiasm or monotony, the unit 5 for determining the state of enthusiasm or monotony is configured to determine the correlate of the state of enthusiasm or monotony by summing the primary transformed signals, averaging the sum of leads over a sliding window, determining the standard deviation of the indicator of enthusiasm or monotony and the global average indicator of enthusiasm or monotony , determining the state boundary and going beyond it (Fig. 7).

Based on the measured data, a number of indices are calculated that comprehensively characterize the state of enthusiasm or monotony among users.

Alpha-index - calculated based on the signal flow from the electrode at point P4. Fast Fourier Transform (FFT) is used for signal processing

I _ai = FFT (D _p4),

where D _{p4 is the} signal from the electrode at point P4,

then the contribution of frequencies 8-13 Hz to the total spectrum is calculated.

Theta-index left - calculated based on the signal flow from the electrode at point F3. Fast Fourier Transform (FFT) is used for signal processing

I _til = FFT (D _f3),

where D _{f3 is the} signal from the electrode at point F3,

then the contribution of frequencies 4 - 7 Hz to the total spectrum is calculated.

Theta-index right - calculated based on the signal flow from the electrode at point P4. Fast Fourier Transform (FFT) is used for signal processing

I _tir = FFT (D _p4),

where D _{p4 is the} signal from the electrode at point P4,

then the contribution of frequencies 4 - 7 Hz to the total spectrum is calculated.

Sum index - summation of deposits, frequent:

I _{sum =} I _ai + I _tir + I _til

The z-score index is a personalization of values for a specific user based on his registered statistical indicators of expectation and standard deviation of the parameter over the observation period

I _z = I _sum -mean (I _sum) / std (I _sum)

FIG. 8 shows a summary of the calculated indices in block 5 for determining the state of enthusiasm or monotony.

According to the concept of the developed model, after calculating the indices, it is necessary to determine their boundary values, the achievement of which means a certain level of the cognitive state of users (triggers). The following triggers are used in the model for determining states of entrainment and monotony in a user.

Trigger "Creative activity". The operator performs activities that are regarded as creative. Trigger condition to check:

I _z > 1.96

Trigger "Routine activities". The operator is performing an activity that is considered compulsive. Trigger condition to check:

I _z <-1.96

FIG. 9 shows a summary of the triggers used in block 5 for determining the state of enthusiasm or monotony.

When implementing the model under consideration as part of a software package for recording and analyzing electrophysiological parameters of a person and providing biological and optical feedback (Mentor PC), a certain PC response to the fulfillment of the trigger conditions discussed above is provided. The reaction is expressed by the output of certain messages to external devices from the PC: a monitor screen or a wearable bracelet.

The message "Mark about a creative task (passion)". A mark in the journal that the state of the operator corresponds to creative activity.

Monotony mark message. A note in the log that the state of the operator corresponds to routine activities.

FIG. 10 shows a list of messages in block 5 for determining the state of enthusiasm or monotony.

Thus, within the framework of the model for determining the states of enthusiasm or monotony in the user, a method was developed for determining the nature of the activity being performed, which does not require self-report, visual observation of the actions of a specialist and professionalism. To implement the model, a method for recording indicators of the electrical activity of the brain was chosen. FIG. 11 is a block diagram of a model for determining states of entrainment or monotony of a user.

If it is necessary to determine the state of cognitive load, the unit 5 for determining the state of cognitive load is configured to determine the correlate of the state of enthusiasm or monotony by determining the number of blinks per minute, averaging the sum of leads over a sliding window, determining the standard deviation of the indicator of cognitive load and the global average indicator of cognitive load, determining the boundary state and exit from it (Fig. 12).

For the flow of the EEG signal from the F7 electrode, recorded in its raw form, the following sequential processing takes place:

1. The signal passes the band-stop filter;

2. The signal undergoes additional filtering, which removes sharp high-amplitude surges in the current sliding processing window;

3. Fast Fourier transform is performed;

4. The power of the spectrum in the theta range is estimated, the mean values of the amplitudes, the standard deviation (RMS) for the current analysis window are allocated;

5. Data on average values and standard deviations of a particular user are stored in the Database (DB);

6. Statistical indicators (mode, median, standard deviation, etc.) are calculated for the entire recording period.

Based on the measured data, a number of indices are calculated that comprehensively characterize the state of users' cognitive activity.

EEG fluctuations in the alpha and theta bands reflect the effectiveness of cognitive functions and memory. Good performance is associated with two types of EEG phenomena (i) tonic increase in alpha but decrease in theta power and (ii) large phasic (event-related) decrease in alpha but increase in theta, depending on the type of memory. Since the alpha frequency shows large interpersonal differences associated with age and memory, this double dissociation between alpha and theta tonic and phase changes can only be observed in the event of failure of fixed frequency bands. This is accomplished by applying corrected alpha and theta frequency windows for each subject using separate alpha frequencies as the reference point. Based on this procedure, it becomes possible to consistently interpret different results. As an example, like an increase in human brain volume, upper alpha strength increases (but theta strength decreases) from early childhood to adulthood, while the opposite is true in later life. Alpha power is reduced and theta power is increased in subjects with various neurological disorders. In addition, after prolonged wakefulness and during the transition from awakening to sleep, when the ability to respond to external stimuli ceases, the upper alpha power decreases, while theta increases. Event-related changes show that the degree of alpha desynchronization is positively correlated with (semantic) long-term memory, while theta timing is positively correlated with the ability to encode new information. The results in question are interpreted on the basis of brain vibrations.

The Low-Alpha-index is calculated based on the signal flow from the electrode at point F7 (D _f7 ). Fast Fourier Transform (FFT) is used for signal processing

I _ai = FFT (D _{f 7),}

then the contribution of frequencies 8-10 Hz to the total spectrum is calculated.

Theta-index calculated based on the signal flow from the electrode at point F7 (D_f7). Fast Fourier Transform (FFT) is used for signal processing

I _ai = FFT (D _{f 7),}

then calculation of the contribution of frequencies 4-7 Hz to the total spectrum.

Blink rate index is calculated based on the measured number of blinks (D _blink )

I _br = 60 / D _blink ,

Blink duration index calculated based on the measured blink duration (D_bl) as the average value of measurements for the recording area

I _bl = (D _{bl one} +…. D _blN ) / N

FIG. 13 shows a summary of the calculated indices in block 5 for determining the state of cognitive load.

The characteristics of various aspects of a person's states are determined by determining and interpreting some boundary values of indices or their combinations - triggers. Triggers are defined as conditions, the fulfillment of which leads to the formation of feedback to the user.

A trigger is based on one or more indicators and assumes that these indicator values follow a certain pattern. The simplest type of trigger is the exit of an indicator beyond the limit of permissible values from above or below. For example, exceeding the heart rate of the maximum permissible threshold.

In this model for determining cognitive load, the trigger is a strong increase in theta rhythm, calculated as the output of the current value of theta rhythm for 2.25 SD, calculated from all available user data. To filter out abrupt changes in the initial parameter that are not associated with the transition to a state of high cognitive load, an additional condition is used - to trigger the trigger, going beyond the initial parameter beyond the boundaries must be held for more than 10 seconds. This classifier is binary and only indicates that the cognitive load is out of range. When the condition in the trigger is met, this trigger raises one or more events, the total space of which is given by the array of possible feedback messages S (T) = [S ₁ , ... S _l ]. The set of triggers makes up the set T = [T ₁ ,… T _k ].

The following triggers are used in the model.

The trigger "Increase in low-frequency activity" is formed on the basis of the Low-Alpha-index (I _ai ) and Theta-index (I _ti ) indices. Checked condition:

[I _ai > (meanI _ai + 1.5 * stdI _{ai )]} And [I _ti > (meanI _ai + 2.25 * stdI _{ti )]}

The fulfillment of the condition means a strong increase in low-frequency activity, which indicates an increase in cognitive load. Hereinafter:

1) The affix "mean" means that it is necessary to take the average value of the indicator for three previous measurements

2) The "std" affix means that it is necessary to take the average standard deviation of the parameter over the three previous values.

Trigger "Increase in blinks" calculated based on blink rate indices (I_br) and average duration of blinking (I_bl). Checked condition:

[I _br > (meanI _br + stdI _{br )]} And [I _{bl >} (meanI _bl + stdI _{bl )]}

The fulfillment of the condition means an increase in the frequency and duration of blinking, which indicates cognitive fatigue.

FIG. 14 provides a summary of the triggers used in block 5 for determining the state of cognitive load.

As a result, the model for monitoring and assessing cognitive load provides for the generation and display of diagnostic messages to the user when certain conditions occur.

A cognitive fatigue message is generated if a Tblink (increased blinking) trigger is executed. At the same time, a vibration signal is sent to the wearable bracelet for 5 seconds, the frequency is 0.5 Hz. When a signal appears, it is recommended to take stimulants (caffeine), or use activating exercises (exercise / activating beta-rhythm training).

The task complexity message is generated if the Tlow condition is triggered (an increase in low-frequency activity). At the same time, a phrase is formed on the monitor screen: “Now you tend to overestimate the complexity of the task. This increases the likelihood of errors. Track them especially carefully. It will also be useful to seek help from colleagues. "

FIG. 15 shows a list of messages in block 5 for determining the state of cognitive load.

The model being developed involves measuring selected indicators of cognitive load, calculating indices on their basis that characterize various aspects of the state of fatigue and working capacity of users, determining some boundary values of indices or their combinations (triggers) and generating diagnostic messages to users that are issued to various external devices (monitor screen, mobile devices).

This model provides the basic framework for the model for determining the user's cognitive load. The model integrates all types of activity and sets the parameters for switching from one state to another.

FIG. 16 shows a block diagram of a model for determining the state of the user's cognitive load.

If it is necessary to determine the state of stress, the unit 5 for determining the state of stress is configured to determine the correlate of the state of stress by determining the frontal asymmetry, averaging the sum of leads over the sliding window, determining the standard deviation of the stress indicator and the global average of the stress indicator, determining the state boundary and going beyond it, assessing the level emotional reaction by electrodermal activity, determination of the absolute value of the stress indicator (Fig. 17).

Based on the measured data, a number of indices are calculated that comprehensively characterize the stress state of users.

The asymmetry index is calculated based on the indicators of the EEG signal flow from the leads at points F3, F4, F6, F7 (D _f3, D _f4, D _f6, D _f7, respectively). The index is calculated as the difference between the logarithms of the sums of powers in leads F3, F7 and the sums of powers in leads F4, F8:

I _asymmetry = log (sqrt (a)) –log (sqrt (b))

The average EDA index is calculated based on the electrical conductivity of the skin between two electrodes ( D _EDA ) as the average value of the indicators over the measurement period

I _eda = mean (D _eda 1 .... D _eda N)

FIG. 18 shows a summary of the calculated indices in block 5 for determining the state of stress.

According to the concept of the developed model, after calculating the indices, it is necessary to determine their boundary values, the achievement of which means a certain level of the cognitive state of users (triggers). The following triggers are used in the model.

The trigger "EDA growth" is formed on the basis of the average EDA index (Ieda). Checked condition:

[I _eda > (mean I _eda + 1.5 * stdI _eda) ]

The execution of a trigger means a strong increase in the tonic component of GSR under similar temperature conditions. Hereinafter:

1) The affix "mean" means that it is necessary to take the average value of the indicator for three previous measurements

2) The "std" affix means that it is necessary to take the average standard deviation of the parameter over the three previous values

Trigger "Asymmetry with a predominance of right hemispheric activity"

The trigger is formed on the basis of the asymmetry index (Iasymmetry). Checked condition:

[I _asymmetry > 0] AND [I _asymmetry > (meanI _asymmetry + 2stdI _asymmetry )]

The execution of a trigger means an increase in asymmetry in the alpha activity of the frontal leads towards the prevalence of the right hemisphere.

FIG. 19 provides a summary of the triggers used in the stress determination block 5.

When the model under consideration is implemented as part of a software package for recording and analyzing electrophysiological parameters of a person and providing biological and optical feedback, a certain response of the PC to the fulfillment of the trigger conditions discussed above is provided. The reaction is expressed by the output of certain messages to external devices from the PC: a monitor screen or a wearable bracelet.

The message is generated when the triggers "Asymmetry with a predominance of right hemispheric activity" (T _asymmetry ) or "EDA growth" (T _eda ) are _fulfilled . The text is displayed on the screen: “You are now overwhelmed with emotions. If you feel that it prevents you from concentrating on the task, take a short BFB-training on relaxation according to the alpha rhythm with your eyes closed. "

FIG. 20 shows a list of messages in block 5 for determining the state of stress.

Thus, as part of the development of a model for determining the state of balance and stress, an express method for assessing the functional state was developed for the user, which does not require complex registration equipment. Methods for recording indicators of electrical activity of the brain and electrodermal activity were selected for implementation. FIG. 21 is a block diagram of a model for determining a user's stress state.

If it is necessary to determine the state of fatigue or vigor, the unit 5 for determining the state of fatigue or vigor is configured to determine the correlate of the state of fatigue or vigor by determining the ratio of the alpha rhythm to theta rhythms, averaging the sum of the leads over a sliding window, determining the standard deviation of the indicator of fatigue or vigor and global the average indicator of fatigue or vigor, determination of the state boundary and going beyond it (Fig. 22).

On the basis of the measured data, a number of indices are calculated that comprehensively characterize the state of fatigue or vigor of users.

Alpha power index is calculated based on the signal flow from the electrode at point AF7 (Da _f7 ). The signal is processed using the Fast Fourier Transform (FFT):

I _{Aaf 7} = FFT (D _{af 7),}

then the power of the rhythms is calculated in the frequency band 8-13 Hz.

The Theta power index is calculated based on the signal flow from the electrode at point AF7 (Da _f7 ). The signal is processed using the Fast Fourier Transform (FFT):

I _{Taf 7} = FFT (D _{af 7),}

then the power of the rhythms in the 4-7 Hz frequency band is calculated.

Index The Alpha / Theta ratio is calculated as the ratio of the Alpha power index to the Theta power index using the formula:

I _fatigue = I _{Aaf 7} / I _{Taf 7.}

A summary of the calculated indices is shown in FIG. 23.

According to the concept of the developed model, after calculating the indices, it is necessary to determine their boundary values, the achievement of which means a certain level of the cognitive state of users (triggers). The following triggers are used in the model.

The "Vivacity" trigger is formed on the basis of the Alpha / Theta Ratio (I _AT ) index. Checked condition:

I _fatigue > Mean (I _fatigue ) + 1.96 * std (I _fatigue ).

The fulfillment of the condition means that the user is in a state of vigor.

The "Fatigue" trigger is formed based on the Alpha / Theta Ratio (I _AT ) index. Checked condition:

I _fatigue <Mean (I _fatigue ) -1.5 * std (I _fatigue )

The fulfillment of the condition means that the user is in a state of fatigue.

A summary of the triggers used is shown in FIG. 24.

When the model under consideration is implemented as part of a software package for recording and analyzing electrophysiological parameters of a person and providing biological and optical feedback, a certain response of the PC to the fulfillment of the trigger conditions discussed above is provided. The reaction is expressed by outputting certain messages to an external device from the PC (monitor screen) or making a note in the database. A total of one message and one mark in the database is provided (Fig. 25).

A fatigue message is generated if the Tfatique trigger is executed. When a signal appears, it is recommended to take stimulants (caffeine), or the use of activating exercises (exercise / activating BFB training in the beta rhythm).

A vigor marker is generated if the Tenergy trigger is executed. At the same time, a note is made in the database that the user is in a state of cheerfulness.

Thus, within the framework of the development of a model for determining the state of vigor and fatigue in the user, a method was developed for assessing the functional state of a specialist, which is adequate for continuous monitoring and calculations. To implement the model, methods of recording indicators of the electrical activity of the brain were selected. The general view of the model is shown in Fig. 26.

If it is necessary to determine the state of concentration, the unit 5 for determining the state of concentration is made with the possibility of summing the frontal and temporal beta activity, averaging the sum of the leads over the sliding window, determining the standard deviation of the concentration indicator and the global average concentration indicator, determining the state boundary and going beyond it (Fig. 27) ...

Based on the measured data, an index is calculated that characterizes the state of user concentration.

Beta power index is determined in the following way.

The index is calculated based on the signal flow from the electrode at point AF7 (D _af7 ). The signal is processed using the Fast Fourier Transform (FFT):

I _{Baf 7} = FFT (D_af7),

then the power of rhythms is calculated in the frequency band 14-30 Hz.

The computed index is summarized in FIG. 28.

According to the concept of the developed model, after calculating the indices, it is necessary to determine their boundary values, the achievement of which means a certain level of the cognitive state of users (triggers). The following trigger is applied in the model.

The Clarity trigger is generated based on the Beta Power Index (I _{B af7} ). Checked condition:

I _Baf7 > Mean (I _Baf7 ) + 1.5 * std ( I _Baf7 ).

The fulfillment of the condition means that the user is in a state of concentration.

A summary of the trigger applied is shown in FIG. 29.

When the model under consideration is implemented as part of a software package for recording and analyzing electrophysiological parameters of a person and providing biological and optical feedback, a certain response of the PC to the fulfillment of the trigger conditions discussed above is provided. The reaction is expressed by a mark in the database. A total of one mark is provided in the database (Fig. 30).

Concentration status flag is generated if the Tlucidity trigger is executed. At the same time, a note is made in the database that the user is in a state of concentration.

FIG. 31 is a graphical view of the model for determining the state of concentration of the user.

If it is necessary to determine the resource state, the unit 5 for determining the resource state includes a module for determining the state of stress based on the selected left hemispheric and right hemispheric alpha rhythms, a module for determining the state of enthusiasm or monotony based on the selected left hemispheric theta rhythms, right hemispheric alpha and theta rhythms, a module for determining concentration based on the selected left hemispheric beta rhythms (Fig. 32).

FIG. 33 shows a summary of the calculated indices in block 5 for determining the resource state.

Unit 5 for determining the resource state is also configured to normalize the value of each parameter, determine the intermediate value of the resource state, normalize the intermediate value of the resource state, and determine the resource state.

The standardized concentration index uses personalized data on the variability of the concentration index for the user and the current value of the index:

I _concz = (I _Baf7 -mean I _Baf7) / std I _Baf7

The Standardized Stress Index uses personalized data on the user's stress index variability and the current index value:

I _stressz = ( I _Baf7 -mean I _Baf7 ) / std I _Baf7

The Standardized Engagement Index uses personalized information about the variability of the user's Engagement Index and the current value of the index:

I _enthz = ( I _sum -mean I _sum ) / std I _sum

The resource state index uses data on personalized values of the indices of concentration, passion and stress as input and calculates their balance:

I _RSI = I _enthz + I _concz - 2 * I _stressz

FIG. 34 is a block diagram of a model for determining the resource state of a user.

The data output unit 6 is any electronic device capable of outputting information to one of the user's perception channels, configured to provide the user 1 with the results obtained in a text, visual, audio, tactile form, depending on the available peripherals and tasks. A monitor, display, speakers, vibrator, etc. can be connected to it.

The personalization unit 7 of at least one predetermined state is configured to determine the boundaries of the normal state for a given user 1 by statistical analysis of historical values according to the state of user 1, and is a programmable module of computational operations in which the upper limit of the normal predetermined state of the user is calculated based on historical data, the difference between the current value and the state boundary is calculated, and the average value is checked in the state trigger range.

Unit 8 for fixing deviations from the user's normal state is configured to determine whether the user 1 is in a given state, and is a module that checks whether the indicators of the user's current state deviate strongly from the normal state and how stable the deviation is.

All units are powered from the general PC power supply.

Additionally, a block (not shown in the drawing) of interaction with the server can be introduced, which stores historical data on user 1 and to which state data is sent to be stored in the database and presented using web interfaces.

The inventive system implements the following method for determining psychoemotional states based on the EEG signal.

User 1 puts on the neural interface and turns on its bluetooth adapter to transmit the EEG signal to the claimed system. In the case of determining stress based on EDA, user 1 also puts on a bracelet (not shown in the drawing) and includes its bluetooth adapter for transmitting the EEG signal and the signal of electrodermal activity to the inventive system.

An EEG signal is received from the user's neurointerface 1 by means of the signal receiving unit 2, then its primary processing is carried out by spectral decomposition of the received signal into the brain rhythms using the primary signal processing unit 3, if necessary, preliminary smoothing of the received signal is performed.

To determine the state of enthusiasm or monotony, the signal is checked for the presence of interference and the aggregation of primary transformed signals such as left hemispheric theta rhythms, right hemispheric alpha and theta rhythms using block 4 for filtering the primary processed signals. The subsequent determination of the state of entrainment or monotony is carried out by determining the correlate of the state of entrainment or monotony by means of the block 5 for determining the state of entrainment or monotony, configured to perform summation of the primary transformed signals, averaging the sum of leads over a sliding window, determining the standard deviation of the entrainment or monotony indicator and the global average of the entrainment indicator or monotony, defining the state boundary and going beyond it.

To determine the state of cognitive load, the signal is checked for the presence of interference and the aggregation of primary transformed signals, such as left-hemispheric theta rhythms and the number of blinks, using the block 4 for filtering the primary processed signals. The subsequent determination of the state of cognitive load is performed by determining the correlate of the state of cognitive load by means of the block 5 for determining the state of cognitive load, configured to perform the determination of the number of blinks per minute, averaging the sum of leads over a sliding window, determining the standard deviation of the indicator of cognitive load and the global average indicator of cognitive load, determining state boundaries and going beyond it.

To determine the state of stress on the basis of EE and EDA, an EEG signal is additionally obtained from the neurointerface of the user 1 through the signal receiving unit 2. An electrodermal activity signal is obtained from the user's bracelet by means of an electrodermal activity signal receiving unit (not shown in the drawing). Then, primary processing of the received signal of electrodermal activity and primary processing of the EEG signal are carried out by spectral decomposition of the received signal into cerebral rhythms using block 3 of primary signal processing, if necessary, preliminary smoothing of the received signal is performed. Further, the signals are checked for the presence of interference and the aggregation of the primary transformed signals, such as the left hemispheric and right hemispheric alpha rhythms, using the block 4 for filtering the primary processed signals. The subsequent determination of the stress state is carried out by determining the correlate of the stress state by means of the stress state determination unit 5, configured to determine the frontal asymmetry, averaging the sum of leads over a sliding window, determining the standard deviation of the stress indicator and the global average stress indicator, determining the state boundary and going beyond it, evaluating the level of emotional reaction by electrodermal activity, determination of the absolute value of the stress indicator.

To determine the state of fatigue or vigor, the signal is checked for the presence of interference and the aggregation of primary transformed signals such as left hemispheric alpha and theta rhythms using the block 4 of filtering the primary processed signals. The subsequent determination of the state of fatigue or vigor is carried out by determining the correlate of the state of fatigue or vigor by means of the fatigue or vigor state determination unit 5, configured to determine the ratio of alpha rhythms to theta rhythms, averaging the sum of leads over a sliding window, determining the standard deviation of the indicator of fatigue or vigor, and global average indicator of fatigue or vigor, determination of the state boundary and going beyond it.

To determine the state of concentration, the signal is checked for the presence of interference and the aggregation of the primary transformed signals, such as left hemispheric beta rhythms, using the block 4 for filtering the primary processed signals. The subsequent determination of the concentration state is performed by determining the correlate of the concentration state by means of the concentration state determination unit 5, configured to perform the summation of the frontal and temporal beta activity, averaging the sum of the leads over the sliding window, determining the standard deviation of the concentration indicator and the global average of the concentration indicator, determining the state and output boundary for her.

To determine the resource state through the block 5 for determining the resource state, the values of each parameter are also normalized, the intermediate value of the resource state is determined, the intermediate value of the resource state is normalized, and the resource state is determined.

If necessary, after determining at least one predetermined state and before outputting the obtained data, the boundaries of the normal state for this user 1 are determined by statistical analysis of historical values for the state of user 1 using the state personalization unit 7. And after determining the boundaries of the normal state for a given user 1 by statistical analysis of historical values according to the user's state and before outputting the obtained data, it is determined that the user 1 is in a given state using block 8 for recording the deviation from the normal state of user 1.

The final output of the data is carried out by providing the user 1 with the obtained results using the data output unit 6, and recommendations can also be provided.

Three types of recommendations can be used to move to the desired state:

1. Recommendations for users who are confident in psychonetic practices;

2. Recommendations containing a more detailed sequence of actions;

3. Recommendations not related to psychotechnics and work with eidograms.

Recommendations based on psychonetic practices require familiarization with basic psychonetic practices, mastering which will help the user to turn their attention into a flexible tool, first of all, for the most effective transition from undesirable states to the necessary.

To get out of a state of monotony:

1. Breathe deeply for a few minutes. As you inhale, imagine how a stream of energy enters your body through the crown of your head, spreads along the spine to the lower abdomen. Hold your breath for five seconds. As you exhale, imagine how this flow spreads throughout the body. Combine breathing with a walk in the fresh air for 10-15 minutes.

2. Warm up your hands by rubbing them together. Then massage the ears (lobes, tragus, inner part) for 3-5 minutes. Clench and unclench your fists sharply. Massage a point in the groove of the base of your thumb and forefinger.

3. Give your body an intense physical activity in the form of a 10-minute jog or other possible, familiar, suitable physical exercise for you personally.

4. Get distracted for 15-20 minutes on your favorite activity, which always arouses your interest. You will be involved in an intense process and will be able to transfer this energy to further work.

To recover from a state of cognitive load:

1. Have a 10 minute active rest. Remember the simplest exercises from school physical education lessons and do them. Do 20 squats and / or push-ups. Take a run. Active movement will help you return to an active state. After completing the exercises, return to the information that caused the cognitive overload.

2. Try to divide the incoming information into smaller structural elements, and then consistently trace the connections of the elements with each other.

3. Try to draw a diagram representing your understanding of the situation that caused the cognitive overload. The elements of the circuit can be, for example, the functional parts of the object you are considering.

To recover from overvoltage / stress:

1. Close your eyes. Bring your attention to the body. Feel where you have accumulated tension and spread your attention from this part of the body throughout its entire volume. At the same time, imagine a warm stream that washes away your stress.

2. Breathe deeply for a few minutes. As you inhale, imagine how a stream of energy enters your body through the crown of your head, spreads along the spine to the lower abdomen. Hold your breath for five seconds. And as you exhale, imagine how this stream takes away all irritation and tension from you.

3. Get some fresh air - go to the balcony or walk along the street for 10-15 minutes.

4. Try to accompany the described actions with soothing music - the one that you love, or turn it on separately, concentrating only on it.

To get out of a state of fatigue:

1. Take a contrast shower. If this is not possible, then wash yourself with cold water - this will relieve tension from the muscles of the face and improve blood circulation.

2. Exercise for 10-15 minutes will improve blood circulation in the body and relieve fatigue. Use a jog, squat, or any other exercise that suits you.

3. It helps to cheer up and distract from fatigue - laughter. Play funny videos on the Internet or read anecdotes.

4. You can go in the opposite direction and let yourself lie down, relaxing and closing your eyes for 15-20 minutes.

The system and method can be implemented on the basis of a personal computer and can be used in any field and in any type of activity, the main task is to identify a negative state and inform the user about it in order to take timely measures in order to increase labor productivity or training efficiency.

Examples of application of the method and system.

1. To assess the state of enthusiasm or monotony, stress and concentration and resource state in general.

In the children's camp, a pilot study was carried out on the possibilities of neurotechnology to assess the response to various teaching methods and courses.

The pilot study included monitoring the resource state of children in the context of three conditions:

1. Stress. Indicates a negative emotional background.

2. Passion. Indicates interest in the task at hand.

3. Concentration. Indicates an increased consumption of neural resources to complete the task.

Based on these criteria, the Resource State Index (IRS) was calculated, reflecting the psychophysiological balance in the process of activity. IRS allows a quick assessment of whether activity has a positive effect on the well-being of children. The higher the IRS, the greater the positive effect the child receives from the lesson.

The assessment involved 48 children attending three types of studios:

- "Ceramics"

- "Rocket Engineering"

- "Piano"

In 7 children, the data collected are not sufficiently informative due to the peculiarities of age-related changes in the shape of the skull, or too little time for participation in the study. Their data were not included in the analysis.

FIG. 35-37 show the results of evaluating the averaged state for rocket science studios, piano, ceramics, respectively.

FIG. 38, 39 show the results of a comparative analysis of the average state of students for the entire time of the studio.

Unit. -% of time spent in each state relative to the time taken

Study Conclusions:

1. Each of the measured indicators of the state is highly dependent on the studio that the child attends. The difference between the studios cannot be explained by differences in the age of the children, gender differences, or the time of the class. Among the fixed patterns of states, there is not a single participant who has uniformly reduced indicators of stress, enthusiasm, and concentration. This indicates that children are actively involved in the educational process. They may not like what is happening in the studio (this corresponds to high stress levels), but they are not indifferent to the learning process.

2. The best pattern ("flow" - the predominance of enthusiasm and concentration with low stress) of states is observed in all participants of the "Piano" studio. Most likely, this result arises from a combination of two main factors - the competent alternation of heterogeneous activity (independent playing the instrument, choral singing) and the positive effect that music has on the student's brain. The second factor can be used to improve the educational dynamics of other studios by introducing light background relaxing music.

3. The difference between singing and playing an instrument is interesting: choral singing shows a tendency towards decreased concentration and increased stress among participants. This can be interpreted in such a way that singing is a less cognitively difficult activity, but at the same time it is perceived as a more socially responsible activity.

4. The lowest IRS is observed at the Raketostroenie studio. This is most likely due to the use of the quiz format in the classroom, which turned out to be too difficult for most children, which, in turn, caused a strong increase in stress indicators.

5. The format of presentation of educational video has a positive effect on the concentration and enthusiasm of the participants, but the effect does not last long - after 3-4 minutes of viewing the condition of the participants returns to normal. The best condition for children is observed when performing manual actions - folding the origami rocket. Because of this, it can be assumed that in more practice-oriented activities (assembling the rocket model), the state of the children would show the pattern of the "flow" state.

6. Classes at the studio "Ceramics" show a pattern of the state of "flow" for most of the session, however, by the end of the session, there is a decline in indicators and an increase in the level of stress. This is most likely due to the accumulation of manual fatigue and the detection of errors in the products made. It may be justified to make a small switch (5-7 minutes) in the second half of the lesson to an alternative creative activity not related to manual labor. This will maintain a high level of concentration in children.

This data can be used to build a more effective educational process.

2. An example of the application of the system is the receipt by a teacher of notifications about exceeding of cognitive load indicators by mathematicians when mastering topics from linear algebra. Receiving such notifications is a reason for re-explaining the area that caused the difficulty, or organizing a short-term break. The use of timely recommendations makes it possible to increase the assimilation of the educational material, which leads to an increase in the marks for intermediate testing by 0.27. The results of the study, which involved 34 students, are shown in FIG. 40.

3. When determining the state of stress. Evaluation of 5 different videos with advertising content on 3 users in order to detect increased emotional reaction for subsequent adjustment of the advertising offer. The statistical results of the EDA measurements are presented in FIG. 41-43.

The results of changes in the index of asymmetry of the hemispheres in the alpha frequency range are presented in Fig. 44. The growth of the graph means a more pronounced predominance of the right hemisphere activity.

The results of the change in the index of electrodermal activity are shown in FIG. 45. A higher graph means higher electrodermal activity and a more pronounced emotional response.

4. An example of a scenario for the application of the system is the task of controlling drivers' fatigue and falling asleep. When the “Fatigue” trigger is triggered, the driver receives a loud notification with a recommendation to make a stop to normalize the condition, which has a positive effect on reducing accidents on the road. The problem is urgent, since according to Ford research, 32% of Russian drivers have fallen asleep while driving at least once.

Claims (20)

1. The system for determining psychoemotional states based on the EEG signal includes a signal receiving unit configured to receive an EEG signal from a user's neurointerface with which it is connected a signal primary processing unit, configured to spectrally decompose the received signal into the rhythms of the brain, to which it is connected filtering unit for primary processed signals, configured to check the signal for interference and aggregation of primary converted signals with the selection of brain rhythms corresponding to at least one specified state with which it is connected a unit for determining at least one predetermined state, configured to determine the correlate of this state, with which it is connected a unit for personalizing a given state, configured to determine the boundaries of a normal state for a given user by statistical analysis of historical values according to the user's state based on the received EEG signal with which it is connected data output unit configured to provide the user with the results obtained. 2. The system according to claim 1, characterized in that the primary signal processing unit is configured to implement preliminary smoothing of the received signal. 3. The system according to claim 1, characterized in that the block for filtering the primary processed signals allows the selection of brain rhythms corresponding to states of enthusiasm or monotony, cognitive load, stress, vigor or fatigue, concentration, and the block for determining a given state allows determining the correlate of these states ... 4. The system according to claim 1, characterized in that a unit for fixing deviations from the user's normal state is introduced, connected to the state personalization unit and the data output unit and configured to determine the user's being in a given state. 5. A method for determining psychoemotional states based on an EEG signal includes receiving an EEG signal from the user's neurointerface by means of a signal receiving unit, its primary processing by spectral decomposition of the received signal into the rhythms of the brain using a primary signal processing unit, checking the signal for the presence of interference and aggregation of the primary transformed signals with the isolation of the brain rhythms corresponding to at least one given state using the filtering unit of the primary processed signals, determining at least one target state by means of a target state determining unit configured to determine the correlate of this state, determination of the boundaries of the normal state for a given user by statistical analysis of historical values according to the user's state on the basis of the received EEG signal using the personalization unit of the given state, data output by providing the user with the results obtained using the data output block. 6. The method according to claim 5, characterized in that at the stage of primary processing of the EEG signal, preliminary smoothing of the received signal is performed. 7. A method according to claim 5, characterized in that at the stage of brain rhythms aggregation, rhythms corresponding to states of enthusiasm or monotony, cognitive load, stress, vigor or fatigue, concentration are selected, and then correlates of these states are determined. 8. The method according to claim 5, characterized in that after determining the boundaries of the normal state for a given user by statistical analysis of historical values according to the user's state and before outputting the obtained data, the user is determined in a given state using a block for fixing a deviation from the user's normal state.

Images