GB2577882A - Multimodal digital therapy and biometric analysis of biometric signals - Google Patents

Multimodal digital therapy and biometric analysis of biometric signals Download PDF

Info

Publication number
GB2577882A
GB2577882A GB1816383.2A GB201816383A GB2577882A GB 2577882 A GB2577882 A GB 2577882A GB 201816383 A GB201816383 A GB 201816383A GB 2577882 A GB2577882 A GB 2577882A
Authority
GB
United Kingdom
Prior art keywords
user
data
analyser
psychological state
digital therapy
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Withdrawn
Application number
GB1816383.2A
Other versions
GB201816383D0 (en
Inventor
Morelli Davide
Plans David
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Biobeats Group Ltd
Original Assignee
Biobeats Group Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Biobeats Group Ltd filed Critical Biobeats Group Ltd
Priority to GB1816383.2A priority Critical patent/GB2577882A/en
Publication of GB201816383D0 publication Critical patent/GB201816383D0/en
Priority to PCT/GB2019/052846 priority patent/WO2020074878A2/en
Publication of GB2577882A publication Critical patent/GB2577882A/en
Withdrawn legal-status Critical Current

Links

Classifications

    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/70ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mental therapies, e.g. psychological therapy or autogenous training
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024Detecting, measuring or recording pulse rate or heart rate
    • A61B5/02405Determining heart rate variability
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/16Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165Evaluating the state of mind, e.g. depression, anxiety
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4836Diagnosis combined with treatment in closed-loop systems or methods
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4848Monitoring or testing the effects of treatment, e.g. of medication
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/486Bio-feedback
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/30ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to physical therapies or activities, e.g. physiotherapy, acupressure or exercising
    • GPHYSICS
    • G16INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16HHEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/50ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for simulation or modelling of medical disorders
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/0205Simultaneously evaluating both cardiovascular conditions and different types of body conditions, e.g. heart and respiratory condition
    • A61B5/02055Simultaneously evaluating both cardiovascular condition and temperature
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/02Detecting, measuring or recording pulse, heart rate, blood pressure or blood flow; Combined pulse/heart-rate/blood pressure determination; Evaluating a cardiovascular condition not otherwise provided for, e.g. using combinations of techniques provided for in this group with electrocardiography or electroauscultation; Heart catheters for measuring blood pressure
    • A61B5/024Detecting, measuring or recording pulse rate or heart rate
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/24Detecting, measuring or recording bioelectric or biomagnetic signals of the body or parts thereof
    • A61B5/316Modalities, i.e. specific diagnostic methods
    • A61B5/318Heart-related electrical modalities, e.g. electrocardiography [ECG]
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/48Other medical applications
    • A61B5/4806Sleep evaluation
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235Details of waveform analysis
    • A61B5/7264Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems
    • A61B5/7267Classification of physiological signals or data, e.g. using neural networks, statistical classifiers, expert systems or fuzzy systems involving training the classification device
    • AHUMAN NECESSITIES
    • A61MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61BDIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00Measuring for diagnostic purposes; Identification of persons
    • A61B5/72Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7271Specific aspects of physiological measurement analysis
    • A61B5/7275Determining trends in physiological measurement data; Predicting development of a medical condition based on physiological measurements, e.g. determining a risk factor

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Public Health (AREA)
  • Medical Informatics (AREA)
  • General Health & Medical Sciences (AREA)
  • Pathology (AREA)
  • Biomedical Technology (AREA)
  • Biophysics (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • Molecular Biology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Physics & Mathematics (AREA)
  • Veterinary Medicine (AREA)
  • Psychiatry (AREA)
  • Hospice & Palliative Care (AREA)
  • Cardiology (AREA)
  • Developmental Disabilities (AREA)
  • Epidemiology (AREA)
  • Psychology (AREA)
  • Social Psychology (AREA)
  • Primary Health Care (AREA)
  • Child & Adolescent Psychology (AREA)
  • Educational Technology (AREA)
  • Physical Education & Sports Medicine (AREA)
  • Biodiversity & Conservation Biology (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • Physiology (AREA)
  • Measurement Of The Respiration, Hearing Ability, Form, And Blood Characteristics Of Living Organisms (AREA)
  • Medical Treatment And Welfare Office Work (AREA)

Abstract

An apparatus, method and computer program for generating and delivering a multimodal digital therapy signal is disclosed. The apparatus comprises an interface configured to receive at least one biometric signal, a multimodal digital therapy generator comprising an analyser configured to analyse the biometric signal and to generate a multimodal digital therapy based upon it and a desired outcome. An output mechanism is configured to provide the generated multimodal digital therapy. The analyser is configured to monitor changes in the biometric signal in response to the multimodal digital therapy and to determine whether the changes indicate an approach towards the desired outcome, and to modify the multimodal digital therapy in response to changes indicating a movement away from the desired outcome. The interface may be configured to receive signals from the user indicative of mood. The digital therapy may comprise cues for inhaling and exhaling and the changes may include an estimated reduction in stress.

Description

MULTEMODAL DIGITAL THERAPY AND BIOMETRIC ANALYSIS OF BIOMETRIC SIGNALS
FIELD OF THE INVENTION
The field of multimodal digital therapy and analysis of biometric signals.
BACKGROUND
Biometric sensors are being increasingly used to monitor such things as an individual's heartbeat. These devices may analyse changes in heartbeat rate to determine exercise levels and calories burnt. They may perform these estimations in conjunction with other sensors configured to sense a user's movement. Outputs maybe provided to inform a user of their heart beat rate and activity levels and encourage them to be more active where it determines activity levels to be below a threshold.
It would be desirable to be provide additional therapy and analysis from collected data.
SUMMARY
A first aspect provides an apparatus for generating and delivering a multimodal digital therapy signal, the apparatus comprising: an interface configured to receive at least one biometric signal; a multimodal digital therapy generator comprising an analyser configured to analyse said received at least one biometric signal and to generate a multimodal digital therapy based on the analysed received at least one biometric signal and a desired outcome; an output mechanism configured to provide the generated multimodal digital therapy; wherein said analyser is further configured to monitor changes in said received at least one biometric signal in response to said multimodal digital therapy and to determine whether said changes indicate an approach towards said desired outcome and to modify said multimodal digital therapy in response to determining that said changes indicate a movement away from said desired outcome.
The inventors of the present invention recognised that there are an increasing number of sensors and other devices that collect biometric information for individuals. They also recognised that much of this information is continually monitored and provides an indication both of the user's behaviour and their physiology. Thus, they concluded that this information if appropriately collected and analysed could be used to provide an indication of a user's current wellbeing -perhaps their current fitness levels or psychological state. This in turn could be used to determine if a digital therapy might be suitable and what type could be applied to achieve a desired outcome.
Providing such a digital therapy output by an output mechanism will have an effect on the individual and this in turn can be monitored from the collected biometric information. In this way the efficacy of the digital therapy can be determined. The digital therapy can then be modified if required and with the provision of this feedback loop a digital therapy that is effectively tailored to a particular user and updated as required is provided.
In some embodiments, said interface is connected to a biometric sensor configured to measure at least one of a heartbeat, a variation in a heartbeat, and a temperature.
The biometric signal may be a number of things, sensed in a number of ways. In some cases the biometric sensor is one configured to measure a heartbeat, a variation in heartbeat or a temperature. The heartbeat rate and variation in heartbeat rate are both indications of a user's current condition. They may be indicative of activity levels of stress or some other factor. Furthermore, a temperature of the user is also indicative of their physiological condition and tending to rise when they are ill and also to fall when they sleep.
Alternatively and/or additionally, said interface is configured to receive signals indicative of a current time, a user's activity, a user's age, sex and weight.
Other factors that have an effect on measured biometric signals and on the user's wellbeing and may be indicative of what a user is currently doing or how they may react to an event may also be measured. For example, a current time may indicate that a user may currently be in a work environment or currently be asleep and this may be a factor when determining heartrate that may be normal for the user at that time. Furthermore, the user's age sex and weight may also provide an indication of the sort of activity one might expect a user to perform and also their expected heart rate. A user's activity may also be monitored from different sensors on the user or from a separate device perhaps one to which the user provides input and this again is a factor that may affect the heart rate and can be used when changes in heart rate are determined to assess whether they are expected or outside of an expected range and indicative of some unusual condition for which therapy may be appropriate.
In some embodiments said interface is configured to receive signals from a user indicative of a current mood.
In addition to receiving biometric signals that are sent from a user, the interface may also be configured to receive user input and these can be indicative of various things such as a current mood or a current mood and associated activity.
The current mood of a user is also an indication of their current wellbeing and may also affect the other biometric signals being measured.
In some embodiments said interface is configured to receive signals from a user indicative of a current mood and current activity.
Where mood and activity are assessed together analyser may determine a correlation between them and this may be used when determining the user therapy. In this regard, where biometric input and potentially user inputs indicate stress, activities that are associated with lower stress may be indicated as a therapy for the user.
In some embodiments, the generated multimodal digital therapy comprises cues for instructing a user when to inhale and when to exhale, in accordance with the desired outcome.
One digital therapy may include instructions to a user to perform certain functions such as breathing exercises. These may be useful in reducing the stress of a user for example.
In some embodiments, the generated multimodal therapy comprises a display indicating a proposed action or activity for the user to perform.
The multimodal therapy may comprise a display indicating a proposed action for the user to perform. In this regard, the action may be one that has previously been assessed to produce a desired outcome and where the digital therapy is determined to require that outcome then that may be the therapy indicated in the display.
Although the output mechanism may be a number of things in some embodiments it comprises a speaker for outputting an audio component of the generated multimodal digital therapy. Alternatively, the output mechanism may be configured to output a haptic component of the generated multimodal digital therapy by controlling the speaker to provide haptic feedback in the form of low-frequency audio.
In some embodiments, the biometric sensor comprises first and second electrocardiographic ECG electrodes arranged to detect an ECG signal when held by the user, and the interface is communicatively coupled to the first and second ECG electrodes and is configured to receive the ECG signal as the biometric signal.
Additionally and/or alternatively the biometric sensor comprises an optical sensor configured to detect a photoplethysmogram PPG signal, and the interface is communicatively coupled to the optical sensor and is configured to receive the PPG to signal as the biometric signal.
In some embodiments said desired outcome comprises at least one of the following: a reduction in heart rate, a reduction in heart rate variations, a change in an estimated psychological state.
The desired outcome may be a number of things but in each case it is something that is measurable such that whether the therapy tends the user towards the desired outcome or away from it can be determined and the therapy can be modified as required thereby improving the efficacy of the device.
In some embodiments said change in said estimated psychological state comprises an estimated reduction in stress.
One particularly important desired outcome might be a change in the estimated psychological state such as a reduction in stress. Embodiments provide an effective manner of estimating stress and as a reduction in stress is extremely desirable being able to estimate it and provide therapy that achieves such a goal is both desirable and effective. In particular, different people react to different therapies in different ways providing some feedback on the effect of that therapy on a particular user enables the therapy to be tailored to a particular user and an improved effect to be provided.
In some embodiments, said multimodal digital therapy generator is configured to generate a digital therapy in dependence upon said estimated current psychological state.
In some embodiments, said desired outcome comprises a preferred psychological state of a user, said analyser being configured to determine and output an indication of a therapy to said user determined to achieve a change in psychological state towards said preferred psychological sate.
A second aspect provides an analyser for continually analysing at least one biometric signal to determine at least one of a current psychological state and physiological condition of a user, said analyser comprising: an interface configured to continually receive at least one biometric signal and to intermittently receive data from a user at least some of said data being indicative of a user's psychological state; said analyser being configured to analyse and combine said data received from said interface and to compare said combined data with stored data indicative of at least one of a psychological state and physiological condition and to estimate at least one of a current psychological state and physiological condition of said user from said comparison; and to output an indication of said estimated state.
The second aspect recognises that biometric signals provide indications of both psychological and physiological condition of a user. However, such data is difficult to analyse as the different conditions and inputs each contribute towards the measured output. Combining biometric signals with data from a user indicative of their psychological state and comparing the data with stored data, provides an efficient way of determining effects on psychological state and/or physiological condition for a particular user such that an output that provides an indication of the estimated state can be provided.
In some embodiments said analyser is configured to estimate values for at least some of said intermittently received data during periods of time that said data is not received and to analyse and combine both said received data and estimated values.
Some of the data received from the sensors may be received continuously or almost continuously while other data is received intermittently. In some cases in order for the analysis to provide effective models of changes in the state the analyser may estimate values where they are not provided such that variations in data over a continuous time spectrum can be provided and used in the estimation. This may improve the accuracy of the result and allow computationally efficient models to be used.
In some embodiments said analyser is configured to continually estimate and output at said least one of said estimated physiological and psychological state.
Some of the data that is generally only received intermittently is physiological and/or psychological. In particular, psychological state is often determined from inputs from a user and these are received intermittently. Where sufficient of these have been received one can find a pattern and estimate what the psychological state would be where no input is received from a user regarding psychological state but other inputs that can be related to psychological state have been received. In this way, the psychological state can be modelled across time and this will improve the accuracy of any estimation that uses this data.
In some embodiments said analyser is configured to analyse said data from said at least one biometric signal and to correlate changes in said data with changes in said user's indicated psychological states and to generate or update said stored data associated with a psychological state for said user.
In particular, biometric signals which may be received continuously and/or quasi continuously can be correlated to changes in psychological state which data is received intermittently and these can be used to update the stored data to tailor this to a particular user rendering the system more accurate and effective.
In some embodiments said stored data comprises a data pattern indicating both data values and changes in data values over time.
Both the data pattern and changes in the data pattern are indicative of a user's current state and how it is changing and thus, this data may be important in the analysis and may be used as an input for the indication of the estimated state.
In some embodiments said analyser is configured to select one or more of said received data to combine in dependence upon said psychological state being estimated.
The received data may have more or less of an effect on the psychological state and in some embodiments the analyser selects the data that is particularly indicative of psychological state and uses this in its estimation, it may discard other received data as not having an effect.
In some embodiments said analyser is configured to determine an expected rate of change of a mood comprising or associated with a psychological state being estimated and to analyse a rate of change of said received data and where said rate of change is similar to said expected rate of change in said mood to select said data as an input to said analysis.
When trying to determine which data is contributing to a particular state being estimated, one factor that can be useful in the analysis is the rate of change of the signals and where they vary in a similar manner, this is an indication that there is a relationship between them.
In some embodiments, said at least one of said psychological state and physiological condition comprises at least one of stress, workplace stress, resilience, sleep disruption and wellbeing, and said biometric sensor comprises at least one of a heart rate and variable heart rate sensor.
Although different psychological states and physiological conditions can be determined from biometric data some of the more important ones to measure and some that are able to be predicted using biometric data are workplace stress, resilience, sleep disruption and wellbeing. In particular, heart rate and variable heart rate are indicative of these things and thus, this data can be used in the estimation of these states and conditions and can be used in both the generation of appropriate therapy and in the output of data indicative of a user's condition.
In some embodiments, said analyser comprises a data store for storing said estimated current psychological state and to output a summary of changes in psychological state over time as said indication of said estimated state.
The ability to estimate current psychological state and changes in psychological state can be important and the data for storing this over time can be used to determine factors that influence a good psychological state and a bad psychological state. Changes can then be made to the habit of a user or a supervisor overseeing that person can change condition such that better psychological states are generally obtained.
A third aspect provides a method for generating and delivering a multimodal digital therapy signal, the method comprising: receiving at least one biometric signal; analysing said received at least one biometric signal; generating a multimodal digital therapy based on the analysed received at least one biometric signal and a desired outcome; and outputting the generated multimodal digital therapy; and monitoring changes in said received at least one biometric signal in response to said multimodal digital therapy and determining whether said changes indicate an approach towards said desired outcome; and modifying said multimodal digital therapy in response to determining that said changes indicate a movement away from said desired outcome.
In some embodiment, said at least one biometric signal comprises at least one of a heartbeat, a variation in a heartbeat, and a temperature.
In some embodiment, said method further comprises receiving signals indicative of a current time, steps taken, a user's age, sex and weight.
In some embodiment, said method further comprises receiving signals from a user indicative of a current mood.
In some embodiment, said method further comprises receiving signals from a user indicative of a current activity and a current mood.
In some embodiment, said method further comprises instructing a user when to inhale and when to exhale, in accordance with the desired outcome.
In some embodiment, the step of outputting comprises: outputting an audio component of the multimodal digital therapy.
In some embodiment, the step of outputting comprises outputting a haptic component of the multimodal digital therapy by controlling a speaker to provide haptic feedback in 25 the form of low-frequency audio.
In some embodiment, the method comprises receiving said biometric signals from first and second electrocardiographic ECG electrodes arranged to detect an ECG signal when held by the user.
In some embodiments, the method comprises receiving the biometric signal from an optical sensor configured to detect a photoplethysmogram PPG signal.
A fourth aspect provides a method for continually analysing at least one biometric signal to determine at least one of a current psychological state and physiological condition of a user, said analyser comprising: continually receiving at least one biometric signal and intermittently receiving data from a user, at least some of said data being indicative of a user's psychological state; analysing and combining said data received from said interface and comparing said combined data with stored data indicative of at least one of a psychological state and physiological condition; estimating at least one of a current psychological state and physiological condition of said user from said comparison; and outputting an indication of said estimated state.
The inventors of the present invention recognised that biometric signals that are continually monitored such as heart rate can be indicative not only of the direct biometric function measured but also indirectly of other things such as a psychological state and/or physiological condition. In particular, if a continually measured biometric signal is monitored in conjunction with intermittently received data from a user, then not only can a current state of the user be estimated by comparing the received data with stored data, but also correlations between changes in psychological state derived from the user input and biometric data can be studied and the stored data updated with new information as it is derived. In this way an effective estimate of a user's state that is tailored to that user is provided.
In some embodiments, said analyser is configured to estimate values for at least some of said intermittently received data during periods of time that said data is not received and to analyse and combine both said received data and estimated values.
In some embodiments, said method comprises continually estimating and outputting at said least one of said estimated physiological and psychological state.
In some embodiments, said method comprises analysing said data from said at least one biometric signal and correlating changes in said data with changes in said user's indicated psychological states and generating or updating said stored data associated with a psychological state for said user.
In some embodiments, said stored data comprises a data pattern indicating both data values and changes in data values over time.
In some embodiments, said method comprises selecting one or more of said received data to combine in dependence upon said psychological state being estimated.
In some embodiments, said method comprises determining an expected rate of change of a mood comprising or associated with a psychological state being estimated and analysing a rate of change of said received data and where said rate of change is similar to said expected rate of change in said mood selecting said data as an input to said analysis.
In some embodiments, said at least one of said psychological state and physiological condition comprises at least one of stress, workplace stress, resilience, sleep disruption and wellbeing, and said biometric sensor comprises at least one of a heart rate and variable heart rate sensor.
In some embodiments, said method comprises storing said estimated current psychological state and outputting a summary of changes in psychological state over time as said indication of said estimated state.
A fifth aspect comprises an apparatus according to a first aspect, wherein said analyser of said multimodal digital therapy generator comprises an analyser according to a second aspect.
A sixth aspect provides a computer program which when executed by a processor is operable to control said processor to perform a method according to a third or fourth aspect.
Further particular and preferred aspects are set out in the accompanying independent and dependent claims. Features of the dependent claims may be combined with features of the independent claims as appropriate, and in combinations other than those explicitly set out in the claims.
Where an apparatus feature is described as being operable to provide a function, it will be appreciated that this includes an apparatus feature which provides that function or which is adapted or configured to provide that function.
BRIEF DESCRIPTION OF THE DRAWINGS
Embodiments of the present invention will now be described further, with reference to the accompanying drawings, in which: Figure 1 provides a legend indicating the different notations used in the flow diagrams; Figure 2 schematically shows the basic blocks of a computational model according to an embodiment; Figure 3 schematically shows the different blocks of a model according to an embodiment; Figure 4 schematically shows an example of the model according to an embodiment; Figure 5 shows a flow diagram schematically illustrating steps in a method according to 5 an embodiment; Figure 6 shows how events and moods may be inter-related over time; and Figure 7 schematically shows one example of how a user may input both activities and their associated moods.
to DESCRIPTION OF THE EMBODIMENTS
Before discussing the example embodiments in any more detail, first an overview will be provided.
Embodiments provide a device and method that monitor a user, analyse the data and provide therapy in the form of coaching, information and/or exercises to the user based on the analysis and on a desired outcome. Embodiments also provide a computational model that analyses the received data and determines expected psychological states from the data and outputs the information or again provides some sort of therapy. Embodiments provide a model that can be tailored to the user and continually updated by using user input in addition to biometric data to monitor changes in user's psychological states where available and correlating with the received data and updating the model.
In effect a framework capable of passively and continuously collecting data from devices such as smartphones and wearables is provided. The data provides an indication regarding the user's behaviour and physiology. By determining the appropriate data to collect and a suitable manner to analyse and combine the data estimations of hidden variables such as "stress", "wellbeing" "workplace stress", sleep disruption" and "resilience" can be determined. This provides both diagnostic and therapeutic tools for addressing these estimated conditions.
The data that is collected may be collected actively by user's providing inputs in response to requests, and passively from data collected from sensors that the user may wear. The data is then analysed and an estimation of the user's physiological and psychological state is determined. This information may be used to personalise interventions and to tailor content displayed to the user.
Embodiments collect behavioural (e.g user actions), physiological (e.g. steps heart rate HR, sleep) and psychological (e.g questionnaire answers, EMA (Ecological Momentary Assessment), executive function tests).
These data inputs are non-evenly sampled and some data may be missing due to noise. Data points that are missing may be estimated and added to the data that the model analyses. In way mathematical tools that require data sampled at a constant rate, i.e. evenly timed samples, e.g. every 5 minutes, or every lo minutes can be used.
The output from the analysis is in some embodiments a continuous or continual estimate of physiological and /or psychological variables for a user.
In response to this information, interventions are offered in the form of coaching and/or exercises, in particular ones that fight stress, foster wellbeing and resilience.
Feedback and machine learning may be used such that the model is continually updated and customised for the user so that interventions that are preferred by and more effective for the user may be offered.
The analysis of the data may be done using a computational model that in some zo embodiments unifies the expected influences of stress on physiological data and psychological data. The aim of the model is to be able to estimate the true stress level of a user over time using measures of HR (heart rate) and user inputs combined in some cases with psychological tests.
Standardized psychological tools can be used to: * Estimate the stress level of the user * Estimate the causes of stress, and the coping strategies However, in an ecological context (real life), psychological measures can only be taken sporadically. Therefore we will not be able to have a continuous, real-time estimation of the status of the user.
Wearable devices offer the opportunity to monitor continuously the user, passively, 24/7. Several models of "stress" have been proposed that use continuous HRV (heart rate variation) data. "Stress" in this context is different than "stress" as measured by psychological measures.
Thus, if a correlation between the biometric data continuously collected and the psychological state of the user can be found, the psychological state of the user could be continuously estimated using the continuously collected data.
A unified computational model for physiological and psychological stress is proposed.
Such a computational model is useful because it offers an interpretation of the raw data collected from a multitude of sources, offering realtime continuous estimates of the user state.
The output of the model could be immediately used to inform the user, or as an input for other systems to implement automatic, customized, realtime wellness programs.
Overview of the model The model follows a Bayesian approach. Some of the variables may be measured over time, some of the variables are inherently hidden.
Figure 1 schematically shows a legend indicating the different notations that are used in the flow diagrams.
It may not in general, be possible to obtain measures of the sensors at arbitrary times, and therefore hidden variables are introduced for most of the sensors, and these provide the most likely value that it is estimated this sensor would have if it was measuring. This estimation is updated to be user specific. In one example a prior over the population for that measure is constnicted, and is changed using the measures we have from that user. The most important sensors are HR and HRV (heart rate variability), as they are influenced by both physiological and psychological factors, and an accurate model will let us discriminate what effects different states have on the measures we see.
Relationships are sometimes explicit models obtained from medical literature, sometimes black box tools, trained on data, like Neural Networks, or other regression tools. Every arrow present in any diagram of this document represents a relationship, that is implemented with mathematical functions or Machine Learning models. The implementation allows the algorithms of the computational model to be updated in a modular way. The important point is the recognition that there is a relationship between these features that can be modelled mathematically.
Figure 2 schematically shows the basic blocks of a computational model according to an embodiment. The Computational Model estimates states that cannot be measured using the values obtained from Measured Variables and the functions defined by the Relationships. These states are the output of the Computational Model.
Embodiments of the overall model are made of the following sections: * The Psychological model: this model captures the interaction between personality, external stressors, good and bad stress, workplace pressure and 10 environment, and HR/HRV * The Physiological model: this model captures the interaction between physical activity, circadian rhythm, physiological conditions, and HR/HRV * Sleep is influenced by the psychological model (stress) and has an influence over the physiological model (circadian rhythm) * Both the Psychological and the Physiological model have an influence on the ANS (automatic nervous system) demand (HR/HRV) Physiological model The physiological part of the computational model attempts to explain heart activity (HR and HRV) as a function of circadian rhythm (therefore, as a function of time), and as a function of physical activity.
Figure 3 schematically shows the different blocks of a model according to an embodiment the arrows representing mathematical relationships between the different 25 parts.
The model has the following structure: * Measurable variables: o Time: time is always available and can be sampled at will o Sex: the sex of the user o Age: the age of the user Steps: the number of steps can be obtained from sources like BioBeam, HealthKit, or other fitness frameworks BM1: the Body Mass Index of the user c HR: if the user owns a wrist worn fitness wearable device we can obtain (mostly) continuous HR measures 24/7. We obtain this data directly from the device.
HRV: HRV can be obtained in a number of ways such as from a user's phone camera with an active action of the user (body stress check) or before a breathing exercise. If the user has a BioBeam device we can collect HRV data 24/7. However, the quality of HRV data depends on the presence of motion artifacts, so only data collected at night, or when we know that the user was still, should be considered reliable.
* Latent variables: o Circadian: the circadian rhythm, this is encoded as average, phase and amplitude.
o Physiological demand: the energy expenditure caused by physical activity.
to o Physiological conditions: a change in the general physiological wellbeing, such as a cold or a flu, or other physiological conditions (sickness), but also the overall status (is the user fit).
* Relationships: o Time, Sex and Age on Circadian: circadian rhythm is primarily determined by 15 the time of the day, but we can adjust the expected values depending on Sex and Age.
o Circadian on HR and HRV: we know from medical literature that Circadian rhythm is visible in HR and HRV. From experience we know that the effect on HRV is not certain. The method used to implement this relationship is described in the co-pending application entitled Resting heart Rate Estimation, in the name of Biobeats, that is filed on the same day as this application, the entire contents of which are incorporated herein by reference.
o Steps, Sex and Age on Physiological demand: the number of steps recently taken (last few minutes) are strongly correlated to the Physiological Demand. We expect that the same activity will result in a different Physiological Demand depending on Sex and Age (and Physiological Conditions) and thus, providing these details as an input allows the model to be appropriately modified to tailor it to a specific user.
o Sex and Age on Physiological conditions: the probability of suffering from physiological conditions changes with age. This relationship is implemented with Neural Networks.
c Physiological conditions on Physiological demand: the way the body reacts to external demand changes in presence of health problems. The Physiological Conditions variable also captures the general wellbeing of the user, therefore being e.g. fit vs fat will change the recovery after physical activity. This relationship is implemented with Neural Networks.
c Physiological conditions on Circadian rhythm: the presence of conditions might have an effect on the expected Circadian rhythm. The Physiological Conditions variable also captures the general wellbeing of the user, therefore it surely has an effect on resting heart rate. This relationship is implemented with Neural Networks.
Circadian rhythm on Physiological demand: the physiological demand could change in accordance with the circadian rhythm related to people's chronotype (i.e., in accordance with the time of the day when a task is performed). In particular, people perceive a higher effort when performing a task not in accordance with their chronotype. This relationship is implemented with Neural Networks.
Prolonged Stress on Physiological Conditions: we know from medical literature that chronic stress has co-morbidities with health conditions. This relationship is to implemented with Neural Networks.
o Circadian, Physiological Conditions, and Physiological demand with themselves: we expect those variables to change more or less slowly through time, the status of a variable at time x has an influence on its status at time x+1. For example, we update the expected circadian rhythm analysing data of the user; the presence of a cold in the morning makes it very likely to also have a cold in the afternoon. The way Physiological demand changes over time depends on Steps, Physiological Conditions, but also the time passed since the demand stopped, so status at time x depends on the status at the previous time. This relationship is implemented with Neural Networks.
zo Psychological model The model is based on the concepts of Belief-Belief-Comparator (BBC) and BeliefDesire-Comparator (BDC) to model emotions, after the CBDIE (computational model of the belief desire theory) model.
An increase BBC corresponds to an unexpected event that changed the beliefs of the user. An increase in BDC corresponds to an event that increased the distance between beliefs and desires (the user thinks that her goals are more distant), a decrease in BDC corresponds to an event that decrease the distance (the user thinks that her goals are closer).
The user's personality traits and their tendency to rumination change the prior of the 30 status of BBC and BDC.
BBC has an effect on Heart Rate Variability: the need to adapt to a change is the definition of sympathetic activation (physiological stress). The quality of stress is not relevant for HRV (eustress vs distress).
BDC and BBC are the cause of emotions in CBDTE, therefore they have an influence on 35 EMA.
From the CATS model we notice that stress is the effect of the lack of coping strategies after an unexpected event, whereas having coping strategies results in a positive experience (self efficacy). We merge the CATS and CBDTE models noticing that the unexpected events in CATS can be modelled with non zero values of BBC, and the valence of the event with the value of BDC (is the event making my beliefs closer or further from my goals?).
We encode BBC and BDC as Valence and Arousal.
Figure 4 schematically shows an example of the model. The model has measured variables (indicated with filled boxes) and unknown/latent variables (indicated with white boxes). The real personality traits of the user are unknown, but we may have 10 some measures of them through personality tests.
The model is divided in 4 areas: 1. Personality traits: variables that are expected to change very slowly (months), or not change at all 2. Emotional state: the user is assumed to continuously change emotion. We model these changes with a hidden markov model. Each emotion/mood has a different coordinate in a dominance/situation/valence/arousal dimensional state. E.g. the "Angry" emotion will have low valence, high arousal, high dominance 3. Work related: measures and variables that express the workplace environment 20 in terms of work demand (how much work pressure the user goes through), resources (amount of help from co-workers and employer), and control (the perception of being able to cope with challenges).
4. Effects of emotions: stress is modelled as an effect of emotions over time. EMA (Ecological Momentary Assessment) are momentary measures of emotions. We assume that arousal will have an effect on the Autonomic Nervous System: high arousal levels will increase the sympathetic activation.
Personality Traits All the variables that are assumed to change slowly, or not change at all, are handled as personality traits. The true values of the personality traits of the user are unknown, but we might have measures that help us estimate them (personality tests). Some personality traits are particularly interesting: * Rumination: we have specific tools to measure it, and we have a coaching program designed to change this personality trait * Coping strategies: coaching program and other tools can be designed to teach the user how to change her coping strategies.
We use two different classes of trait to evaluate these individual characteristics depending on the stability and range of the user's behaviour in real world described by single trait.
1. Some traits are cardinal and rooted to biological and pre-dispositional ways in which subject interacts with external stimuli and in social context, they are lifelong characteristics that influence transversal aspects of personal life; 2. Some traits could change in the long period (e.g. months), these attributes may be isolated characteristic composed of multiple facets and owned at different degree by each person (e.g. resiliency) or constitute a set of possible and different typology of responses to the world linked to a unique trait (e.g. coping style) that change according to situation. We can measure the relative degree of each of the facets related to a trait or the predominant typology of responses related to a set of behaviours. Over a longer period we can hypothesize the change of degree of one of these attribute or the variation of respective amounts of one of them compared to others. For example a subject may have a predominant coping style toward social situation but could show also a small amount of other coping style in this class of situation or a complete different favourite coping style for example in a "challenge situation".
zo We may define every time which is the class of the personality traits that we are interested in, taking into account that our devices and coaching program could help the users to change only the second class of traits, differently the first class represents a filter to understand how the subject think and act in the reality or how he/she interact to our devices.
How to measure: For a single personality trait (indifferently on class 1 or 2) we can assign an integer discrete number to the subject that represent the degree of that characteristic in his/her life and how much that trait shapes his/her behaviour. For the class 2, when we measure a trait in which are presented multiple facets or typology of response we assign an integer discrete number to each facets or typology of responses.
That number indicate the respective amount of each facet compare to other or the frequency with which a subject show a specific typology of response.
Currently we Measure: 1) A personality profile of BISBAS constitutes by three facets: (1) BIS (anxiety) (2) BAS (fun seeking) (3) BAS (Drive) (4) BAS (Reward-Responsiveness) 2) Neuroticism (Neu) 3) A measure of Coping and Personal Resources (1) Resourcefulness Skills Scale (RSS) (2) Ways of Coping This questionnaire measure coping during or after a specific situation Other measures of general Health Status: 1. HRQ0L-14 -Healthy Days Measure Emotional state We model the emotional state of the user as a continuous time hidden markov model. 15 The user is in some emotional state in every moment in time, even if we don't measure it. Every emotional state has an associated value for valence, arousal and dominance. Arousal: a real value between o and 1. High arousal values imply excitement.
Valence: a real value between o and 1. High valence values imply positive mood, low values negative mood.
Dominance: a real value between 0 and 1. Low values imply submissive attitude.
The situational state of the user is independent from the arousal/valence/dominance space and indicates in what context the user is currently. E.g. at work, at home, etc. The emotional state and its evolution over time is implemented using Hidden Markov Models (HMM). The HMM models the probability of each emotional state to evolve over time into other states, as described in the following section (sojourn times). HMM allows us to estimate the most likely evolution over time of the true emotional state of the user, even in presence of missing data (i.e. we miss emotion declarations).
Expected Sojourn times Each of the 5 Mood corresponds to a different sojourn time in which subject could transit before he/she returns to the basic Mood that is CALM mood.
When a user reports SAD mood, he/she stays in that mood a period ranged from 1 hour to 36 hours depending by the amount of valence associated to specific emotion in this category, lower is the valence longer is the time of permanence of the subject, with a direct proportionality. The fluctuation of the emotion/mood reported returns gradually to higher value of CALM mood following a Gaussian pattern, where the EMA reported is in the middle section of the curve. Lower is the valence of SAD category of emotion, closer in time is the event that provokes that affective state.
When a user reports WORRY mood he/she stays in that mood a period ranged from 1 hour to 4 hour. When WORRY EMA is reported the subject is in the crescent part of a Gaussian curve that represents the time decorse of the emotion, the middle section and the average point of the curve represents the hypothetical peak of WORRY emotion to connected to a specific event and determines the very next possibility of a reported EMA in SAD or Angry category. So the lower the valence reported by the subject in this category the closer in time he/she is to the EVENT that provokes the emotional state and the faster will be the gradual return to CALM condition. Conversely the higher the valence values reported in this category, the higher the possibility that this emotion increases in the next hours before it could decrease to return to CALM condition.
When a user reports ANNOYED mood he/she stays in that mood a period of maximum 2 hours. ANNOYED category follows a Gaussian pattern of decorse in time where the subject is in the middle section of the curve when he/she reports the emotion/mood.
The lower the ANNOYED EMA valence reported, the higher the possibility in the next days that the subject reports a EMA in the ANNOYED category, in particular in the same situational cues described in the primary EMA.
When a user reports HAPPY mood he/she stays in that mood a period of maximum 3 hour. HAPPY category follows a Gaussian pattern of decorse in time where the subject is in the middle section of the curve when he/she reports the emotion. The higher the valence the longer the period of permanence in the HAPPY condition before the subject returns to CALM condition.
Relationships between Personality traits and emotional state This section lists the effects that personality traits have on the emotional state. This is technically implemented changing the transition matrix of the HMM underlying the emotional state of the user.
* Neuroticism (Neu) is a measure of emotion lability: the higher the value of this trait the higher the possibility that a user frequently reports emotion with lower (and sometimes higher) valence values compared to the average population and the possibility that he/she changes category of emotion during the day.
* BIS (anxiety) is a measure of general susceptibility of the user to worries. A user with high value in this scale will report more frequently EMA in WORRY and SAD categories (30%) of mood/emotions and generally choosing the label with higher values compare to average population.
* BAS (fun-seeking) this user loves intense emotion. The higher the value in this trait the higher the possibility that a user enjoys new experiences and risky situations. He/She will report more frequently EMA in HAPPY mood/emotion category close to intense and emotional events. He/she is a little more resistant to stressful events and loves to put himself/herself in unpredictable situations, but he/she is also more prone to feeling bored and shifts more frequently to lower values in SAD and ANNOYED categories during a long period of inactivity.
* BAS (Drive) a higher value describes a user that loves to accept challenges and endures following a task, so they are more reliable in following coaching programs or exercise to reach a goal.
* BAS (Reward Responsiveness) Low values corresponds to a user that is very prone to feel and fall in a depressive state and could be less reactive to positive events. These users will report more frequently EMA in SAD mood/emotions (15%) category and less frequently EMA in HAPPY mood/emotions category. Higher values corresponds to a major sensitivity to frustrations caused by a missed goal or failed personal standard, with a higher probability to report ANNOYED EMA (zo%).
* Rumination: Higher values in this section corresponds to a more frequent report of EMA with a low valence score without a specific trigger event in particular on 25 ANNOYED (20%) and SAD category * Resourcefulness Skills Scale (RSS) higher values correspond to a user that has many resources to resist or to face potential stressful situation. These users will report low valence EMA less frequently or fall in negative mood/emotion and consequent stress state for less time or feel less effect to potential stressful events.
Figure 5 shows a flow diagram illustrating steps of a method according to an embodiment. In this embodiment, one or more biometric signals are received and analysed. A multimodal digital therapy is then generated based on the analysed data and a desired outcome. This may involve breathing exercises for a user or advice to perform some activity or some other coaching. Changes in the biometric signals are then monitored and it is determined whether the desired outcome is being achieved. if it is, no change is made to the therapy, if it is not, the therapy may be modified and the biometric signals monitored again. If this change in therapy is deemed successful, details regarding this updated therapy may be stored.
An example is provided below of how relationships between emotional state, stress and 5 ANS activation are analysed and determined.
Stress is defined as a product of the emotional state of the user over time (e.g. low valence high arousal). Stress is directly proportional to the amount of low valence report provided by the subject in a week or in a month.
An assumption is made that high arousal implies ANS activation. In turn, ANS activation implies elevated HR that is measured by sensors and also used by the Physiological model. ANS Activation is therefore the main link between the Physiological and Psychological models.
The Ecological Momentary Assessment (EMA) tool derives from emotional theories based on appraisal processes and represents a synthetic measure of a main determinant factor for an effective state: emotional causal component, event trigger and situational context. Different tools can however be used, the importance is the recognition of how quickly these emotions are liable to change, what triggers them and the effect this has on measurables such as heart rate. This allows changes in things such as heart rate to be mapped to changes in emotions and an output of emotions and means for coping with these can be provided.
Appraisal is described by various authors as a primitive process that generates or contributes to an affective response. We propose a causal model of appraisal where we label each emotion according to major dimensions as shared in Lazarus, Frijd a and Scherer appraisal models (Ellsworth, 2013). These Four dimensions are indicated by a value and link the affective state (further defined as MOOD or EMOTION) to an EVENT.
Novelty arousal Implication-goal relevance valence Agent implication self/other target Coping Potential -> dominance Users choose different emotional terms characterized primarily by an arousal and valence values and previously divided into different categories derived from the validation procedure of Discrete Emotional Questionnaire (DEQ) (Harmon-Jones, 2016). The model uses 5 categories to create a self-reported measure of current emotional state towards in Present Event and 2 categories to measure emotional expectancy towards Future Event.
These categories correspond to Core Relational themes from Smith and Lazarus (1993) appraisal theory and are conceptualized and tested as different dimension of person-environment relationship or Core Effect in Russell circumplex model (Posner et al., 2005). For each emotion a first term refers to an arousal value while a second term refers to valence values. Arousal corresponds to the saliency of an event that a person is living (in our general model a discrepancy between belief of event as it has prospected and understanding of how it is really happening); while valence corresponds to the importance of this event compared to individual goals. These values range from 1 to 7: higher value for valence represents an unexpected event, higher values of valence a negative result for person's goals.
We propose a scale of 9 points for the classification of valence and arousal from Hepach et al., 2011 to select single emotions values, refining the MOOD categories border through an adaptation of of Yik et al., 2011 circumflex model. First value is for arousal second for valence.
Terms to the left refer to items present in the first step watched by a user, they represent average emotional words corresponding to what we define as MOOD subscale dimension. We choose average states to depict emotional categories because they could be used also as more frequent response for our general question "how do you feel now?". User could refine their state choosing a more convenient emotion in the second step. In this second step we also list a pool of emotions include in original DEQ and connotated by similar but not identical arousal and valence score. In future implementation we will define each emotion by dominance and person value. Each emotion with arousal below the MOOD values is included as MOOD -/+ report, if the arousal is higher they are labeled as EMOTION -/+.
The main difference between MOOD category is related to the positive or negative valence. Calm and Happy category is placed in a continuum level, where Happy has major arousal and valence than calm. For negative MOOD categories, Sad represents a low arousal high valence and low dominance category (with exception of very high valence that enhance arousal value -as for Despair EMOTION) , while Annoyed category has a high arousal, high valence, high dominance value and Worried has a high value, high valence, low dominance values.
Coping note: low dominance MOOD and EMOTION bring to emotional form of coping as well as high dominance ones bring to action form of coping.
Emotion for Present Event The first column contains the moods presented in the first screen (when all the moods are collapsed). When the user taps on a mood we assign that mood as the selected value with its relative arousal and valence. We also expand the section relative to that mood with the emotions contained in the second column. If the user selects an emotion contained in the expanded pane the selection is replaced with the selected emotion and its valence and arousal. Therefore the user can select 1 mood or 1 emotion, no multiple selection is possible.
In the expanded pane, below the motions, there is also a list of situational tags that the user can select (this time multiple selection is possible): zo Emotion for Future Event Wanting 3.96 2.54 Desire 2.95 2.57 Craving 5.69 2.21 Scared 6.27 6.6 Worried 6.27 5.67 Fear 6.69 6.68 Terror 8.26 7.78 Label for Dashboard in the Card In the dashboard of an EMA Card we show the principal emotions or moods reported by users during a specific amount of time (e.g. last week), and below this 'emotional summary' we report by two small bars the average valence and arousal of the subject, accompanied by a specific label. Users can choose the specific period of time showed by these indexes (e.g. last week, last month, yesterday).
Valence and arousal in the bars will be calculated by the arithmetic average of all EMA values reported during the current specific period.
Below are listed the labels that we could assign to user index reported in the bars: Extreme Activation 7t0 9 Positive emotional tone 1 to 3.5 High Activation 6 to 7 Neutral emotional tone 3.5 to 5.5 Medium Activation 4.5 to 5.9 Negative emotional tone 5.5 to 7 Low Activation 1 to 4.5 Extremely Negative emotional tone 7 to 9 Temporal dimension of Negative Emotion and Mood To clarify the relationship between our mood and emotion we postulate, according to scientific literature, a dynamic dimension among these affective states, that explain the passage from one specific state to another.
An important distinction between MOOD and EMOTION states is their time duration in relation to an EVENT: MOOD states have in general an average value for valence and arousal (in particular considering the EMOTION related to each appropriate MOOD category) but have a longer permanence. Negative emotion in the PRESENT EVENT could be a different connection with a trigger EVENT: * Worried MOOD category (similar to Scared) and relate EMOTION indicates a long lasting negative affect that begins in time close to the moment of tag and is prolonged to a future hypothetical EVENT; * Sad MOOD category and relate EMOTION indicates a negative affect refers to a 25 current or recent past EVENT; * Annoyed category and relate EMOTION indicates a current negative affect refers to a close EVENT with a shorter permanence compare to worried category.
Figure 6 schematically illustrates the link between these.
Sad category is referred to a past event but his duration could be longer than other categories, while worried category generally is referred to a future situation but its intensity became higher next to the trigger EVENT. Annoyed category is focused on a present event but its influence could be recurrent in the future.
All MOOD and EMOTION decrease in intensity with the augmentation of temporal distance from the trigger EVENT, Sad category instead has a slower slope of decreasing and the proximity of trigger event could be estimated from the initial valence of the first sad log (higher is the valence closer is the event).
Situation cue Situational cue can be used to describe in the third step the situation connected to an EVENT that users are experiencing during EMA logging: they could select more items from each cue dimensions. Situational description is incredibly useful to determine the psychological environment in which subject acts: the correspondence between frequent emotions felt in a class of situation constitutes an extensive description of individual reaction that can be crossed with subject personality trait. Personality reported traits through separate questionnaires and frequently adopted behaviours in a set of typical situation are the main factors to predict future individual conduct. Exploring users' emotions during a current situation we understand their perception and response to a series of event generated into a specific frame and predict their subsequent behaviour in similar context.
Figure 7 shows an input screen where users may input their emotions and how they relate to activities.
Definition of stress response and emotion A starting point of the model is a concise definition of both stress and emotion: The framework of our computational model of stress response and associated emotion combines the 'Cognitive Activation Theory of Stress' (CATS) by Ursin & Eriksen (2004) and the GUTS (Generalized Unsafe Theory of Stress) from Brosschot et al., (2016) in the definition of stress physiological processes and the concept of emotion that combine the main features shared to the appraisal theories of emotion with the schematisation offered in the 'Computational Belief and Desire Theory of Emotion' (CBDTE) of Reisenzein (2009).
The bearing structure of our model combine CATS and CBTE theories and lays on the central role that expectancy plays in both models: to generate a specific emotion in CBTE, taking into account the instant nature of this reaction and to establish eventually a stress condition and response in the CATS.
Following CATS approach instead "the effects of stress are manifest in four distinct domains; physiology, behaviour, subjective experience, and cognitive function (Levine and Ursin 1991, Steptoe et al 2008). The term "stress" is used for four aspects of "stress"; stress stimuli, stress experience, the non-specific general stress response, and the experience of the stress response (Levine and Ursin 1991). The stress response is conceptualized as a general alarm in a homeostatic system, producing general and unspecific neurophysiological activation from one level of arousal to a higher level of arousal (Ursin & Eriksen, 2010)". "CATS is a cognitive theory since the physiological and psychological consequences of "stress" all depend on cognitive evaluations of the situation and the consequences. A crucial concept is "expectancy", the knowledge we acquire when dealing with challenging situations. CATS is also a psychobiological theory, the psychobiological consequences of the cognitive activity is explained by increases in brain activation (arousal -wakefulness), and the psychological and physiological concomitants of arousal. Despite the neurophysiological basis of CATS and its distinct identity compares to other theories, the integrative nature of this approach allows to conjunge this vision of stress with other model more organizational and job oriented, to extend and refine its application on workplace environment (Meurs"J. A., & Perre'ive. P. L., 2011). In particular to extend the validity of our model in workplace situation we mutated same concept and relationship from the 'demand-control model' of Karasek & Theorell (1996) and job demand resource model (Schaufeli and Bakker, 2004) that take into account the objective contextual factors that may produce stress strengthen the subjective perception provides from CATS approach in In this prospective of stress process, arousal is an implicit measure derived from a physiological parameter that we could estimate directly by Heart Rate and Heart Rate Variability. Stress response in particular corresponds to a specific and constant pattern of arousal activity establish through learning mechanism that links a set of stimuli to a specific consequence. These S-R bond for the perpetuation of hopelessness and helplessness state.
The affective nature of Stress could be investigated by a deeper description of emotional response and its antecedents.
CBDTE provides a perfect explanation to understand emotion phenomenology that is also computational based. Current theories of emotion in psycholog-y, in particular on the cognitive science approach, grounded the differentiation and the origin of emotion upon the concept of appraisal (Arnold, 1960), i.e. a basic antecedent of emotion that define all the core aspect of this mental state during its occurrence.
While most appraisal theories quote the evaluative nature of this process, they often don't clarify how this evaluation proceeds and if it is conscious. CBDTE similar to Frijda (1993) and Scherer (2005) or Lazarus (2004) conceptualization arise the motivational and goal-based roots of emotion.
In this notion of emotion their characterization are based on the difference between the idea external world that an individual sketch out in his/her mind (Called beliefs) and the actual occurrence of a certain stimulus or event. This difference is the basic functions of appraisal and corresponds indeed of an expectation process.
The two fundamental components of appraisal are the belief and belief comparators (BBC), that is deputy to the comparison of what a person thinks about an event (belief) and what is happening in the actual moment (belief). While the belief and desire comparators (BDC) gives a positive or negative evaluative connotation of this discrepancy, based on the specific goal of the subjects.
It is important to note that, in this vision, emotions are completely originated by the action of BBC and BCD that could be formally described even in computational format, but in theoretically form emotion are the result of a constant comparison of our internal vision or the world and the real events, evaluated from the perspective of our goals and ambitions. More importantly both BBC and BCD are not necessarily conscious as well as the definition of precise goal in our life or in a particular situation, while the emotional experience by the subjects could be reported.
The conciliation of CATS Model and CBDTE derives from the observation of the key role of expectancy in both models, in particular we can link the arousal fluctuation described in CATS that brings to a stress pattern with the constant and sudden action of BBC, while the negative evaluation of a specific stimulus of event are described by BCD. The establishment of a protracted discrepancy in BBC forecasts and a BCD negative response leads to a stress response over time.
While the BBC action could be detected, even implicitly by HR and HRV measure, BCD results are described directly by subject via emotion description with Ecological Momentary Assessment (EMA), for the discrete emotion felt as well as the Goals could be reported by subject in the Journal.
Interestingly the values of BBC and BCD could be inferred following the evaluation gave by subjects through a specific emotion that is codified with a specific BBC and BCD coordinates. IN particular they could be inferred by EMA values describing BBC discrepancy as an arousal dimension of emotion and valence dimension as a BDC correspondent measure.
Subject's Goal or Desire could be inferred asking directly with a EMA subscales of desire for a target event or stimulus with a scale labelling emotions and mood according to various dimensions and by individual personality traits.
The complex model derived from these two visions lacks any-ways of a fundamental component that is the transversal influence of human perception of own capability to face a situation, i.e. the subject resource and their influence in create a coping response to a stressful situation. This third face of our model is described through the adoption of a third dimension in emotion labelling, defined dominance, that represents a measure of both emotional reaction generation and the individual perception of dangerous and stressful situation. This determined emotional feature and its influence in conjoint emotional causality, possible distress response and consequent coping is well explained in Lazarus and Folkaman Transteoretical Theory. We will refers to this wide conceptualization just as a link to understand the connection between emotion generation and regulation elements (the appraisal dimension) and their role in facing a stressful situation through a different form of coping strategies, but the connection between dominance dimension of emotions and a chronic stress establishment is better fitted in GUTS model. GUTS extends the explanation power of CATS and the importance of expectancy with the notion of unsafety situation. While expectancy in CATS include a sort of reference to dangerous and harmful dimension of stress, where a more unexpected event is synonymous of a threat, CUTS model make explicit this dimension describing the individual challenge of a situation in terms of physiological processes prolonged in time.
CATGUT of Stress Emotion generation (EMA) CHRONIC stress Individual coping and stress response (Journal) (physiology) (HR/HRV) CBDTE model (valence, arousal) CATS model (arousal) Lazarus coping (dominance-resources) Appraisal features and Lazarus model (+dominance) GUTS model (dominance) GuTS model and rumination Demand-Induced Strain Compensation Recovery (DISC-R) factor demand resource outcome-recovery cognitive Cognitive demand (tasks) CATS model (expectancy) Cognitive load emotional (Negative emotions) CBTE and Lazarus appraisal Lazarus coping GUTS model (safety) time/situation Affect event theory GUTS model Rumination Although illustrative embodiments of the invention have been disclosed in detail herein, with reference to the accompanying drawings, it is understood that the invention is not limited to the precise embodiment and that various changes and modifications can be effected therein by one skilled in the art without departing from the scope of the invention as defined by the appended claims and their equivalents.
A person of skill in the art would readily recognize that steps of various above-described methods can be performed by programmed computers. Herein, some embodiments are also intended to cover program storage devices, e.g., digital data storage media, which are machine or computer readable and encode machine-executable or computer-executable programs of instructions, wherein said instructions perform some or all of the steps of said above-described methods. The program storage devices maybe, e.g., digital memories, magnetic storage media such as a magnetic disks and magnetic tapes, hard drives, or optically readable digital data storage media. The embodiments are also intended to cover computers programmed to perform said steps of the above-described methods.
Features described in the preceding description may be used in combinations other than the combinations explicitly described.

Claims (25)

  1. CLAIMS1. An apparatus for generating and delivering a multimodal digital therapy signal, the apparatus comprising: an interface configured to receive at least one biometric signal; a multimodal digital therapy generator comprising an analyser configured to analyse said received at least one biometric signal and to generate a multimodal digital therapy based on the analysed received at least one biometric signal and a desired outcome; an output mechanism configured to provide the generated multimodal digital therapy; wherein said analyser is further configured to monitor changes in said received at least one biometric signal in response to said multimodal digital therapy and to determine whether said changes indicate an approach towards said desired outcome and to modify said multimodal digital therapy in response to determining that said changes indicate a movement away from said desired outcome.
  2. 2. An apparatus according to claim 1, wherein said interface is connected to a biometric sensor configured to measure at least one of a heartbeat, a variation in a heartbeat, and a temperature.
  3. 3. An apparatus according to claim 1 or 2, wherein said interface is configured to receive signals indicative of a current time, a user's activity, a user's age, sex and weight.
  4. 4. An apparatus according to any preceding claim, wherein said interface is configured to receive signals from a user indicative of a current mood.
  5. 5. An apparatus according to any preceding claim, wherein said interface is 30 configured to receive signals from a user indicative of a current mood and current activity.
  6. 6. An apparatus according to claim 5, wherein said analyser is configured to determine a correlation between mood and activity for a user
  7. 7. An apparatus according to any preceding claim, wherein the generated multimodal digital therapy comprises cues for instructing a user when to inhale and when to exhale, in accordance with the desired outcome.
  8. 8. An apparatus according to any preceding claim, wherein the output mechanism comprises at least one of: a display configured to display a proposed action or activity for the user to perform, said proposed action or activity comprising said generated multimodal therapy; a speaker for outputting an audio component of the generated multimodal to digital therapy; and an output mechanism configured to output a haptic component of the generated multimodal digital therapy by controlling a speaker to provide haptic feedback in the form of low-frequency audio.
  9. 9. An apparatus according to any preceding claim, wherein said desired outcome comprises at least one of the following: a reduction in heart rate, a reduction in heart rate variations, a change in an estimated psychological state.to.
  10. An apparatus according to any preceding claim, wherein said change in said estimated psychological state comprises an estimated reduction in stress it.
  11. A method for generating and delivering a multimodal digital therapy signal, the method comprising: receiving at least one biometric signal; analysing said received at least one biometric signal; generating a multimodal digital therapy based on the analysed received at least one biometric signal and a desired outcome; and outputting the generated multimodal digital therapy; and monitoring changes in said received at least one biometric signal in response to 30 said multimodal digital therapy and determining whether said changes indicate an approach towards said desired outcome; and modifying said multimodal digital therapy in response to determining that said changes indicate a movement away from said desired outcome.
  12. 12. An analyser for continually analysing at least one biometric signal to determine at least one of a current psychological state and physiological condition of a user, said analyser comprising: an interface configured to continually receive at least one biometric signal and to intermittently receive data from a user at least some of said data being indicative of a user's psychological state; said analyser being configured to analyse and combine said data received from said interface and to compare said combined data with stored data indicative of at least one of a psychological state and physiological condition and to estimate at least one of a current psychological state and physiological condition of said user from said comparison; and to output an indication of said estimated state.
  13. 13. An analyser according to claim 12, wherein said analyser is configured to estimate values for at least some of said intermittently received data during periods of time that said data is not received and to analyse and combine both said received data and estimated values.
  14. 14. An analyser according to claim 12 or 13, wherein said analyser is configured to continually estimate and output at said least one of said estimated physiological and psychological state.
  15. 15. An analyser according to any one of claims 12 to 14, wherein said analyser is configured to analyse said data from said at least one biometric signal and to correlate changes in said data with changes in said user's indicated psychological states and to generate or update said stored data associated with a psychological state for said user.
  16. 16. An analyser according to any one of claims 12 YO 15, wherein said stored data comprises a data pattern indicating both data values and changes in data values over time.
  17. 17. An analyser according to any one of claims 12 YO 16, wherein said analyser is 30 configured to select one or more of said received data to combine in dependence upon said psychological state being estimated.
  18. 18. An analyser according to any one of claims 12 to 17, wherein said analyser is configured to determine an expected rate of change of a mood comprising or associated with a psychological state being estimated and to analyse a rate of change of said received data and where said rate of change is similar to said expected rate of change in said mood to select said data as an input to said analysis.
  19. 19. An analyser according to any one of claims 12 to 18,wherein said at least one of said psychological state and physiological condition comprises at least one of stress, workplace stress, resilience, sleep disruption and wellbeing, and said biometric sensor comprises at least one of a heart rate and variable heart rate sensor.zo.
  20. An analyser according to any one of claims 12 to 19, wherein said analyser comprises a data store for storing said estimated current psychological state and to output a summary of changes in psychological state over time as said indication of said estimated state.
  21. 21. A method for continually analysing at least one biometric signal to determine at least one of a current psychological state and physiological condition of a user, said analyser comprising: continually receiving at least one biometric signal and intermittently receiving data from a user at least some of said data being indicative of a user's psychological state; analysing and combining said data received from said interface and comparing said combined data with stored data indicative of at least one of a psychological state and physiological condition; estimating at least one of a current psychological state and physiological condition of said user from said comparison; and outputting an indication of said estimated state.
  22. 22. An apparatus according to any one of claims ito up, wherein said analyser of said multimodal digital therapy generator comprises an analyser according to claim 12 10 20.
  23. 23. An apparatus according to 22, wherein said multimodal digital therapy generator is configured to generate a digital therapy in dependence upon said estimated current psychological state.
  24. 24. An apparatus according to 22 or 23, wherein said desired outcome comprises a preferred psychological state of a user, said analyser being configured to determine and output an indication of a therapy to said user determined to achieve a change in psychological state towards said preferred psychological sate.
  25. 25. A computer program product configured to perform a method according to claim 11 or 21.
GB1816383.2A 2018-10-08 2018-10-08 Multimodal digital therapy and biometric analysis of biometric signals Withdrawn GB2577882A (en)

Priority Applications (2)

Application Number Priority Date Filing Date Title
GB1816383.2A GB2577882A (en) 2018-10-08 2018-10-08 Multimodal digital therapy and biometric analysis of biometric signals
PCT/GB2019/052846 WO2020074878A2 (en) 2018-10-08 2019-10-08 Multimodal digital therapy and biometric analysis of biometric signals

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
GB1816383.2A GB2577882A (en) 2018-10-08 2018-10-08 Multimodal digital therapy and biometric analysis of biometric signals

Publications (2)

Publication Number Publication Date
GB201816383D0 GB201816383D0 (en) 2018-11-28
GB2577882A true GB2577882A (en) 2020-04-15

Family

ID=64394845

Family Applications (1)

Application Number Title Priority Date Filing Date
GB1816383.2A Withdrawn GB2577882A (en) 2018-10-08 2018-10-08 Multimodal digital therapy and biometric analysis of biometric signals

Country Status (2)

Country Link
GB (1) GB2577882A (en)
WO (1) WO2020074878A2 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN113255635B (en) * 2021-07-19 2021-10-15 中国科学院自动化研究所 Multi-mode fused psychological stress analysis method
WO2023135632A1 (en) * 2022-01-11 2023-07-20 日本電気株式会社 Stress estimation device, stress estimation method, and storage medium

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020083122A1 (en) * 2000-12-21 2002-06-27 Lemchen Marc S. Method and apparatus for the use of a network system for biofeedback stress reduction
US20080214944A1 (en) * 2007-02-09 2008-09-04 Morris Margaret E System, apparatus and method for mobile real-time feedback based on changes in the heart to enhance cognitive behavioral therapy for anger or stress reduction
US20100069774A1 (en) * 2005-06-13 2010-03-18 University Of Vermont And State Agricultural College Breath Biofeedback System and Method
US20110183305A1 (en) * 2008-05-28 2011-07-28 Health-Smart Limited Behaviour Modification
WO2016202442A1 (en) * 2015-06-17 2016-12-22 L4 Method and product for determining a state value, a value representing the state of a subject
US20180279899A1 (en) * 2017-04-03 2018-10-04 International Business Machines Corporation System, apparatus, and methods for achieving flow state using biofeedback

Family Cites Families (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2011109716A2 (en) * 2010-03-04 2011-09-09 Neumitra LLC Devices and methods for treating psychological disorders
US8529447B2 (en) * 2011-05-13 2013-09-10 Fujitsu Limited Creating a personalized stress profile using renal doppler sonography
EP2774533A1 (en) * 2013-03-06 2014-09-10 Machine Perception Technologies, Inc. Apparatuses and method for determining and using heart rate variability
US9801553B2 (en) * 2014-09-26 2017-10-31 Design Interactive, Inc. System, method, and computer program product for the real-time mobile evaluation of physiological stress

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20020083122A1 (en) * 2000-12-21 2002-06-27 Lemchen Marc S. Method and apparatus for the use of a network system for biofeedback stress reduction
US20100069774A1 (en) * 2005-06-13 2010-03-18 University Of Vermont And State Agricultural College Breath Biofeedback System and Method
US20080214944A1 (en) * 2007-02-09 2008-09-04 Morris Margaret E System, apparatus and method for mobile real-time feedback based on changes in the heart to enhance cognitive behavioral therapy for anger or stress reduction
US20110183305A1 (en) * 2008-05-28 2011-07-28 Health-Smart Limited Behaviour Modification
WO2016202442A1 (en) * 2015-06-17 2016-12-22 L4 Method and product for determining a state value, a value representing the state of a subject
US20180279899A1 (en) * 2017-04-03 2018-10-04 International Business Machines Corporation System, apparatus, and methods for achieving flow state using biofeedback

Also Published As

Publication number Publication date
GB201816383D0 (en) 2018-11-28
WO2020074878A3 (en) 2020-06-25
WO2020074878A2 (en) 2020-04-16

Similar Documents

Publication Publication Date Title
Christopoulos et al. The body and the brain: Measuring skin conductance responses to understand the emotional experience
Adão Martins et al. Fatigue monitoring through wearables: a state-of-the-art review
Hopkins et al. Electrodermal measurement: Particularly effective for forecasting message influence on sales appeal
US11013449B2 (en) Methods and systems for decoding, inducing, and training peak mind/body states via multi-modal technologies
Sharma et al. Objective measures, sensors and computational techniques for stress recognition and classification: A survey
Nacke An introduction to physiological player metrics for evaluating games
Conner et al. Conscientiousness and the intention–behavior relationship: Predicting exercise behavior
US10783801B1 (en) Simulation based training system for measurement of team cognitive load to automatically customize simulation content
Al Osman et al. U-biofeedback: a multimedia-based reference model for ubiquitous biofeedback systems
Hussain et al. Automatic cognitive load detection from face, physiology, task performance and fusion during affective interference
Baethge et al. Coworker support and its relationship to allostasis during a workday: A diary study on trajectories of heart rate variability during work.
Maier et al. Pupil dilation predicts individual self-regulation success across domains
Cohen et al. Effects of different real-time feedback types on human performance in high-demanding work conditions
Sinha et al. Dynamic assessment of learners' mental state for an improved learning experience
Patt et al. Disentangling working memory processes during spatial span assessment: A modeling analysis of preferred eye movement strategies
WO2020074878A2 (en) Multimodal digital therapy and biometric analysis of biometric signals
Foglia et al. Towards relating physiological signals to usability metrics: a case study with a web avatar
DiDomenico An investigation on subjective assessments of workload and postural stability under conditions of joint mental and physical demands
US10085690B2 (en) System and method for feedback of dynamically weighted values
Wang et al. Pervasive persuasion for stress self-regulation
Talukdar et al. Estimation of mental fatigue during EEG based motor imagery
RU2740256C1 (en) System and method for determining psychoemotional states based on biometric eeg signal
Bläsing et al. Influence of complexity and noise on mental workload during a manual assembly task
Gontier How to prevent mind-wandering during an EVA? Presentation of a mind-wandering detection method using ECG technology in a Mars-analog environment
Stevens et al. Intermediate neurodynamic representations: a pathway towards quantitative measurements of teamwork?

Legal Events

Date Code Title Description
732E Amendments to the register in respect of changes of name or changes affecting rights (sect. 32/1977)

Free format text: REGISTERED BETWEEN 20201126 AND 20201202

WAP Application withdrawn, taken to be withdrawn or refused ** after publication under section 16(1)