WO2012143834A1 - Emotion guidance device and method - Google Patents

Emotion guidance device and method Download PDF

Info

Publication number
WO2012143834A1
WO2012143834A1 PCT/IB2012/051817 IB2012051817W WO2012143834A1 WO 2012143834 A1 WO2012143834 A1 WO 2012143834A1 IB 2012051817 W IB2012051817 W IB 2012051817W WO 2012143834 A1 WO2012143834 A1 WO 2012143834A1
Authority
WO
WIPO (PCT)
Prior art keywords
signal
physiological
biofeedback
synchronization
user
Prior art date
Application number
PCT/IB2012/051817
Other languages
French (fr)
Inventor
Joris Hendrik Janssen
Joanne Henriëtte Desirée Monique WESTERINK
Wijnand Anton IJSSELSTEIJN
Martin Ouwerkerk
Original Assignee
Koninklijke Philips Electronics N.V.
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Koninklijke Philips Electronics N.V. filed Critical Koninklijke Philips Electronics N.V.
Publication of WO2012143834A1 publication Critical patent/WO2012143834A1/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/16—Devices for psychotechnics; Testing reaction times ; Devices for evaluating the psychological state
    • A61B5/165—Evaluating the state of mind, e.g. depression, anxiety
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/48—Other medical applications
    • A61B5/486—Bio-feedback
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/68—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient
    • A61B5/6801—Arrangements of detecting, measuring or recording means, e.g. sensors, in relation to patient specially adapted to be attached to or worn on the body surface
    • A61B5/6802—Sensor mounted on worn items
    • A61B5/681—Wristwatch-type devices
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/015—Input arrangements based on nervous system activity detection, e.g. brain waves [EEG] detection, electromyograms [EMG] detection, electrodermal response detection
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M1/00—Substation equipment, e.g. for use by subscribers
    • H04M1/72—Mobile telephones; Cordless telephones, i.e. devices for establishing wireless links to base stations without route selection
    • H04M1/724—User interfaces specially adapted for cordless or mobile telephones
    • H04M1/72403—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality
    • H04M1/72409—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories
    • H04M1/72412—User interfaces specially adapted for cordless or mobile telephones with means for local support of applications that increase the functionality by interfacing with external accessories using two-way short-range wireless interfaces
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/0002—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network
    • A61B5/0004—Remote monitoring of patients using telemetry, e.g. transmission of vital signals via a communication network characterised by the type of physiological signal transmitted
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/01—Measuring temperature of body parts ; Diagnostic temperature sensing, e.g. for malignant or inflamed tissue
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/05—Detecting, measuring or recording for diagnosis by means of electric currents or magnetic fields; Measuring using microwaves or radio waves 
    • A61B5/053—Measuring electrical impedance or conductance of a portion of the body
    • A61B5/0531—Measuring skin impedance
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/08—Detecting, measuring or recording devices for evaluating the respiratory organs
    • A61B5/0816—Measuring devices for examining respiratory frequency
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B5/00—Measuring for diagnostic purposes; Identification of persons
    • A61B5/72—Signal processing specially adapted for physiological signals or for diagnostic purposes
    • A61B5/7235—Details of waveform analysis
    • A61B5/7239—Details of waveform analysis using differentiation including higher order derivatives
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F2203/00—Indexing scheme relating to G06F3/00 - G06F3/048
    • G06F2203/01—Indexing scheme relating to G06F3/01
    • G06F2203/011—Emotion or mood input determined on the basis of sensed human body parameters such as pulse, heart rate or beat, temperature of skin, facial expressions, iris, voice pitch, brain activity patterns
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04M—TELEPHONIC COMMUNICATION
    • H04M2250/00—Details of telephonic subscriber devices
    • H04M2250/12—Details of telephonic subscriber devices including a sensor for measuring a physical value, e.g. temperature or motion
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42201—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS] biosensors, e.g. heat sensor for presence detection, EEG sensors or any limb activity sensors worn by the user

Definitions

  • the present invention relates to an emotion guidance device and method for guiding emotions of at least a first user and a second user towards synchronization or de- synchronization.
  • the present invention also relates to a wearable device, a biofeedback device, and an emotion guidance system, each comprising such an emotion guidance device.
  • the present invention also relates to a computer program for implementing such a method.
  • WO 2007/042947 Al discloses a handheld device comprising a heartbeat sensor for monitoring a first heartbeat signal of a user of the handheld device, a
  • the communication unit for receiving a second heartbeat signal of a user of another handheld device, a comparison unit for comparing the first and second heartbeat signals so as to determine a level of coherence between them, and a decision unit for indicating a potential lover if the level of coherence is above a given threshold.
  • the indication can be delivered to the user via a specific ringing tone or via a given message on the screen. In this way, the handheld device indicates a potential lover to the user of the handheld device.
  • this functionality is very specialized.
  • the present invention provides an emotion guidance device and method for guiding emotions of at least a first user and a second user towards synchronization or de-synchronization, in particular in a repeated or continuous manner.
  • an emotion guidance device for guiding emotions of at least a first user and a second user towards synchronization or de- synchronization is presented.
  • the device comprises an input interface for receiving a first physiological signal of the first user from a first physiological sensor and a second physiological signal of the second user from a second physiological sensor.
  • the device further comprises a calculating unit for calculating a synchronization index based on the first physiological signal and the second physiological signal, the synchronization index representing the level of synchronization of emotions of the users.
  • the device further comprises a control unit for controlling at least one biofeedback signal depending on the calculated synchronization index, and an output interface for repeatedly or continuously outputting the at least one biofeedback signal to at least one biofeedback device.
  • a wearable device wearable by a user is presented, in particular a wristband.
  • the wearable device comprises the emotion guidance device and the physiological sensor for sensing the physiological signal of the user.
  • a biofeedback device comprising the emotion guidance device is presented.
  • the biofeedback device is adapted for providing biofeedback based on the biofeedback signal.
  • an emotion guidance system comprising the emotion guidance device, the first physiological sensor for sensing the first physiological signal of the first user, the second physiological sensor for sensing the second physiological signal of the second user, and the at least one biofeedback device for providing biofeedback based on the at least one biofeedback signal.
  • an emotion guidance method for guiding emotions of at least a first user and a second user towards synchronization or de- synchronization.
  • the method comprises receiving a first physiological signal of the first user from a first physiological sensor, and receiving a second physiological signal of the second user from a second physiological sensor.
  • the method further comprises calculating a synchronization index based on the first physiological signal and second physiological signal, the synchronization index representing the level of synchronization of the users.
  • the method further comprises controlling at least one biofeedback signal depending on the calculated synchronization index, and repeatedly or continuously outputting the at least one biofeedback signal to at least one biofeedback device.
  • a computer program comprising program code means for causing a computer to carry out the steps of the emotion guidance method when said computer program is carried out on the computer is presented.
  • the basic idea of the invention is to calculate a synchronization index based on the first physiological signal and the second physiological signal, the synchronization index representing the level of synchronization of emotions of the users, and to control the repeated or continuous biofeedback signal output depending on that calculated
  • the output can be repeated at preset time intervals (e.g. every minute).
  • the preset time interval can be between 1 second and 10 minutes, in particular between 5 seconds and 5 minutes, in particular between 30 seconds and 3 minutes, in particular between 1 minute and 2 minutes.
  • the time intervals can be constant or non-constant.
  • Continuously output can be at very short time intervals (e.g. such that the output appears continuous to the user) or without interruption, for example in real-time.
  • the output can be provided at the same rate as the synchronization index is calculated.
  • the users can enhance their synchronization.
  • the users can then gradually get in tune with each other.
  • the users can worsen their synchronization (de-synchronization).
  • the users can then gradually get out of tune with each other.
  • the idea is that the users can (repeatedly or continuously) track their synchronization level with the other user and therefore adapt their emotions.
  • the users will be primed to enhance (or worsen) their synchronization of emotions and the users see the direct effects of their efforts.
  • users can control their physiological state by cognitive strategies like changing their own emotional state, trying to emphasize or focus more on the other person, or by physiological strategies like changing their breathing. All these strategies influence the synchronization of emotions with the other person.
  • the biofeedback based on the biofeedback signal could also be presented in a less conscious way.
  • the biofeedback could be provided through varying or changing the quality of an audio output, such as background music. For example, if the synchronization index is low, the quality of the audio output could be relatively bad. The user would then gradually learn that enhancing his empathy with the other user (e.g. his partner), will result in better quality audio output, since the synchronization index will go up. Thus, the users are gently guided towards a higher synchronization, without their explicit awareness of the synchronization index or synchronization state as a necessary prerequisite.
  • the synchronization index is calculated using at least one measure selected from the group comprising an absolute difference between the first and second physiological signals, the difference between the derivatives of the first and second physiological signals, the correlation of the first and second physiological signals, the correlation of the derivatives of the first and second physiological signals, the coherence of the first and second physiological signals, the coherence of the derivatives of the first and second physiological signals, and the difference between the directions of the first and second physiological signals.
  • Using the measure of the absolute difference between the first and the second physiological signals provides for an easy implementation of calculating the synchronization index.
  • the measures of the difference between the derivatives of the first and second physiological signals and the measure of the correlation or coherence of the first and second physiological signals or their derivatives provides for a more advanced calculation of the synchronization index.
  • Using the measure of the difference between the directions of the first and second physiological signals is another approach.
  • the input interface is further adapted to receive a third physiological signal of a third user from a third physiological sensor, and the calculating unit adapted to calculate a first intermediary synchronization index based on the first physiological signal and the second physiological signal and a second intermediary synchronization index based on the first physiological signal and third physiological signal.
  • the calculating unit is further adapted to also calculate a third intermediary synchronization index based on the second physiological signal and the third physiological signal. In this way, the physiological signal of each user in the group is compared to each of the other users' physiological signals. An intermediary synchronization index for each possible user pair is calculated. This variant is in particular useful if all users (the first, second and third user) are equally important.
  • the calculating unit is adapted to calculate a final synchronization index by calculating an average of the first intermediary synchronization index and the second synchronization intermediary index.
  • the synchronization index can be calculated by calculating a weighted average, so as to give certain users more weight.
  • the calculating unit in combination with the variant of calculating a third synchronization index, is adapted to calculate a final
  • the final synchronization index is based on all of the calculated intermediary synchronization indices. This variant is in particular useful if all users (the first, second and third user) are equally important.
  • the emotion guidance device further comprises a preprocessing unit for pre-processing each of the first physiological signal and the second physiological signal, wherein the synchronization index is calculated based on the pre- processed first physiological signal and the pre-processed second physiological signal. This improves the quality of the signal.
  • the physiological signal in particular the first physiological signal, the second physiological signal and/or the third physiological signal, comprises or is at least one signal selected from the group comprising a skin conductance signal, a skin temperature signal and a respiration signal.
  • These physiological signals are particularly suitable for representing the emotion of the user.
  • These physiological signals are known to be coupled to emotions in general.
  • skin conductance is very strongly coupled to emotions and has relatively fast response times. The skin conductance rises sharply within a few seconds after the experience of a stressor.
  • skin conductance is very strongly coupled to emotions and has relatively fast response times. The skin conductance rises sharply within a few seconds after the experience of a stressor.
  • the at least one biofeedback signal is a signal for providing biofeedback to the user by changing color or intensity of light output and/or changing type or quality of audio output.
  • the at least one biofeedback signal is a signal for increasing a pleasant biofeedback output and/or decreasing an unpleasant biofeedback output, when the synchronization index increases. This enables the users to easily
  • the emotion guidance device further comprises a wireless communication link for receiving the first physiological signal and/or second physiological signal, and/or a wireless communication link for transmitting the biofeedback signal.
  • a wireless communication link for receiving the first physiological signal and/or second physiological signal, and/or a wireless communication link for transmitting the biofeedback signal.
  • the emotion guidance device further comprises at least one memory unit for storing the calculated synchronization indices over time. This enables that the calculated synchronization indices over time, thus the synchronization data, can be used after the emotion guidance session to evaluate the process.
  • the emotion guidance device further comprises at least one memory unit for storing the calculated synchronization indices over time.
  • the memory unit stores the first physiological signal and second physiological signal over time. This enables to calculate the synchronization index based on the stored data over time, which improves accuracy.
  • Fig. 1 shows a schematic diagram of an emotion guidance system and device according to an embodiment
  • Fig. 2 shows a perspective view of a biofeedback device according to an embodiment
  • Fig. 3 shows a perspective view of a wearable device according to an embodiment
  • Fig. 4a to Fig. 4c each shows a schematic diagram of the emotion guidance system and device according to different embodiments.
  • Fig. 1 shows a schematic diagram of an emotion guidance system 100 and an emotion guidance device 10 according to an embodiment.
  • the emotion guidance system 100 and emotion guidance device 10 guide emotions of at least a first user 1 and a second user 2 towards synchronization or de-synchronization.
  • the emotion guidance system 100 comprises the emotion guidance device 10, a first physiological sensor 22 for sensing a first
  • the emotion guidance device 10 is a separate part. It will be understood that the emotion guidance device 10 can also be incorporated into any other part or entity, such as for example the biofeedback device 26, 28 or a wearable device 30, as will be explained.
  • the first physiological sensor 22 and/or the second physiological sensor 24 can comprise at least one sensor selected from the group comprising a skin conductance sensor, a skin temperature sensor and a respiration sensor.
  • the first physiological signal 16 and/or the second physiological signal 18 can comprise at least one signal selected from the group comprising a skin conductance signal, a skin temperature signal and a respiration signal.
  • These physiological signals are known to be coupled to emotions in general.
  • skin conductance is very strongly coupled to emotions and has relatively fast response times.
  • the most relevant sensor and signal for determining the synchronization of emotions is an electrodermal activity (EDA), or skin conductance sensor and signal.
  • EDA electrodermal activity
  • the skin conductance rises sharply within a few seconds after the experience of a stressor.
  • the amount of skin conductance level change between an onset point and a peak point is correlated with the intensity of arousal.
  • Electrodermal activity is normally measured between two electrodes which are placed on the skin of the user 1. In particular, it can be measured unobtrusively on the wrist, ankle, feet, or hands.
  • Other sensors and signals from which emotional synchronization can be determined are a skin temperature sensor and signal and a respiration sensor and signal.
  • skin temperature is measured using a skin temperature sensor placed on the skin.
  • skin temperature can also be unobtrusively measured.
  • skin temperature reacts slowly to arousal (rising) and relaxation (dropping). Thus, skin temperature changes more gradually and is therefore better suited for longer term measurements.
  • Respiration can be measured using a respiration sensor, such as an accelerometer or gauge device, for example around the chest or waist of the user.
  • respiration measurement by means of video, for example by extracting the chest/abdomen movement from a video stream.
  • respiration reacts fast to startle (e.g. holding breath for a short time) and shows relaxation by slow regular abdominal breathing (e.g. 6-8 times/minute).
  • startle e.g. holding breath for a short time
  • slow regular abdominal breathing e.g. 6-8 times/minute
  • Respiration can be under conscious control of the user.
  • it can be easier for the users to synchronize, especially when the goal of
  • the emotion guidance device 10 will now be explained in more detail.
  • Emotion guidance device 10 comprises an input interface 11 for receiving the first physiological signal 16 of the first user 1 from the first physiological sensor 22 and the second physiological signal 18 of the second user 2 from the second physiological sensor 24.
  • the emotion guidance device further comprises a pre-processing unit 12 for pre-processing each of the first physiological signal 16 and the second
  • the pre-processing can for example comprise at least one operation selected from the group comprising smoothing of the signal, frequency filtering of the signal, and normalization of the signal.
  • normalization of the signal can for example comprise mean subtraction or division by standard deviation.
  • the emotion guidance device 10 further comprises a calculating unit 13 for calculating a synchronization index I based on the pre-processed first physiological signal 16 and the pre-processed second physiological signal 18.
  • the synchronization index I represents the level of synchronization of the emotions of the first user 1 and the second user 2.
  • the synchronization index I is calculated using at least one measure selected from the group comprising an absolute difference between the first and second physiological signals 16, 18, the difference between the derivatives of the first and second physiological signals 16, 18, the correlation of the first and second physiological signals 16, 18, the correlation of the derivatives of the first and second physiological signals 16, 18, the coherence of the first and second physiological signals 16, 18, the coherence of the derivatives of the first and second physiological signals 16, 18, and the difference between the directions of the first and second physiological signals 16, 18.
  • Calculating the absolute difference between the signals can in particular be used after normalization has been performed by the pre-processing unit 12.
  • Using the difference between the derivatives of the signals uses the differences of the slope of the signals, and thus the shape of the signal.
  • Each of the above-mentioned calculation measures can provide a synchronization index I which is number from 0 to 1.
  • a synchronization index I of 0 represents a minimal level of synchronization and a synchronization index I of 1 represents a maximal level of synchronization, or vice versa.
  • the first and second physiological signals 16, 18 can for example be compared at different times or lags. Thus, the timing between the first and second physiological signals 16, 18 is shifted.
  • the lag can for example be zero. However, the lag can also be a positive number, thus larger than zero, to take into account the time it takes for an emotion to get the other user (e.g. 5 seconds).
  • the emotion guidance device 10 further comprises a control unit 14 for controlling at least one biofeedback signal 19 depending on the calculated synchronization index I.
  • the emotion guidance device 10 further comprises an output interface 15 for repeatedly or continuously outputting the at least one biofeedback signal 19 to at least one biofeedback device 26, 28. In this way a (more or less) continuous feedback loop through the user is provided.
  • the control unit 14 can repeatedly or continuously control the at least one biofeedback signal 19, the calculating unit 13 can repeatedly or continuously calculate the synchronization index I, and/or the input interface 11 can repeatedly or continuously receive the first physiological signal and the second physiological signal 18.
  • the emotion guidance device 10 in the embodiment of Fig. 1 further comprises a first memory unit 17 for storing the calculated synchronization indices I over time. This synchronization data can be used afterwards to evaluate the process. Thus, an overview or summary is provided which can be evaluated, and the users may know how to change their behavior.
  • the physiological sensors 22, 24 can repeatedly or continuously sense the physiological signals 16, 18. All the sensed signal can be further processed (repeatedly or continuously) or only the sensed signal for a specific time period can be used, for example at least ten seconds, in particular between one minute and ten minutes.
  • the emotion guidance device 10 can comprise a second memory unit 17a for storing the first physiological signal 16 and the second physiological signal 18 over time, as can be seen in Fig. 1. This stored data can then be transmitted to the calculating unit 13, or first to the pre-processing unit 12. For example, a running window can be used over this data. Updates of this data can be made at high frequency, so that changes are reflected immediately to the user.
  • any number of units selected from the group comprising the input interface 11, pre-processing unit 12, calculating unit 13, control unit 14 and output interface 15 can be implemented in one single processing device, such as for example a processor (e.g. microprocessor) or any other suitable processing device.
  • the calculating unit 13 and the control unit 14 (and optionally the pre-processing unit 12) can be implemented in one single processing device.
  • the first memory unit 17 and the second memory unit 17a can be separate entities, or can be implemented in one single memory device.
  • the emotion guidance device 10 can further comprise a wireless communication link for receiving the first physiological signal 16 and/or the second physiological signal 18, as for example illustrated in Figs. 4a to 4c.
  • the physiological signals 16, 18 can be transmitted using a bluetooth communication link.
  • the emotion guidance device 10 can further comprise a wireless communication link for transmitting the biofeedback signal 19.
  • the biofeedback device 26, 28 provides an indication of the level of synchronization of emotions of the users 1, 2 to the users 1, 2.
  • the biofeedback signal 19 is output to a single biofeedback device 26 (solid line in Fig. 1).
  • the first biofeedback device 26 is here in the environment El of the first user 1. In this case, the biofeedback is provided to only the first user 1, not to the second user.
  • the first biofeedback device 26 can be in the environment of both the first user 1 and the second user 2. In this case, the biofeedback is provided to both the first user 1 and the second user 2.
  • the biofeedback signal 19 can also be output to a second biofeedback device 28 in the environment E2 of the second user. Again, in this case, the biofeedback is provided to both the first user 1 and the second user 2.
  • the biofeedback device can be in the environment of another person which is not taking part in the emotion guidance, for example an impartial facilitator or mediator or a therapist.
  • the biofeedback device 26, 28 can be adapted for haptic, visual and/or audio biofeedback.
  • the at least one biofeedback signal 19 can be a signal for providing biofeedback to the user by changing color or intensity of light output.
  • the biofeedback device 26, 28 can be adapted to change color or intensity of light output.
  • a perspective view of an embodiment of such a biofeedback device 26, 28 is shown in Fig. 2.
  • the biofeedback device 26, 28 of Fig. 2 is a lamp, in particular an LED lamp, for outputting multiple colors, such as for example "LivingColors" by Philips.
  • a remote control 29 can also be used as an extra control mechanism/device for the users (e.g. to switch the light on or off).
  • the skin conductance signal 16, 18 can be linked in software to the lamp 26, 28 by means of an address of the remote control 29, e.g. using a wireless PC dongle normally used for software firmware upload to the device.
  • the biofeedback signal 19 can be a signal for changing type or quality of audio output.
  • the type of audio output such as music
  • the quality of music can for example be changed.
  • the at least one biofeedback signal 19 can be a signal for increasing a pleasant biofeedback output and/or decreasing an unpleasant biofeedback output, when the synchronization index I increases.
  • the at least one biofeedback signal 19 can be a signal for decreasing a pleasant biofeedback output and/or increasing an unpleasant biofeedback output, when the
  • the synchronization index I decreases.
  • the lighting type such as color or intensity of the light
  • the lighting type such as color or intensity of the light
  • the users can be made more pleasant, thus improving the lighting atmosphere, or the quality of music to which the users listen can be made better.
  • the users have a natural tendency to prefer a more pleasant biofeedback, such as more pleasant lighting type or better quality music
  • the users will adjust their emotions towards synchronization, in order to achieve the more pleasant biofeedback.
  • the first user can be in a very negative emotional state
  • the second user can be in the opposite, thus positive, emotional state.
  • the second user can try to cheer the first user up.
  • the first user will then try to guide his/her emotions towards synchronization with the second user.
  • At least one biofeedback signal 19 can be a signal for increasing an unpleasant biofeedback output and/or decreasing a pleasant biofeedback output, when the synchronization index I increases.
  • the at least one biofeedback signal 19 can be a signal for decreasing an unpleasant biofeedback output and/or increasing a pleasant biofeedback output, when the
  • the synchronization index I decreases.
  • the lighting type such as color or intensity of the light
  • the lighting type such as color or intensity of the light
  • the users can be made less pleasant, thus worsen the lighting atmosphere, or the quality of music to which the users listen can be made worse.
  • the users will then adjust their emotions towards de-synchronization, in order to achieve the worse biofeedback and become thus more aggressive.
  • Fig. 3 shows a perspective view of an embodiment of a wearable device 30 wearable by a user.
  • the wearable device 30 is in form of a wristband comprising a wristband material part 33 and a casing 34. It will be understood that the wearable device 30 could also be worn around any other suitable body part, such as the ankle, foot or hand.
  • two skin conductance electrodes 31, 32 are integrated into the wrist band material 33. The skin conductance electrodes 31, 32 are used for sensing the skin conductance of the user.
  • the wearable device 30 comprises the first physiological sensor 22 or the second physiological sensor 24.
  • the physiological sensor 22, 24 is a skin conductance sensor.
  • the physiological sensor 22, 24 can also be any other suitable sensor, such as a skin temperature sensor.
  • the wearable device 30 can for example comprise a respiration sensor and be in the form of a respiration belt.
  • the wearable device 30 shown in Fig. 3 can for instance comprise the emotion guidance device 10, for example the emotion guidance device described with reference to Fig. 1.
  • the emotion guidance device 10 can be integrated into the casing 34 of the wearable device 30.
  • the wearable device 30 can further comprise a transmitter for wirelessly transmitting data over a wireless communication link, such as the biofeedback signal 19.
  • the emotion guidance device 10 shown in the embodiment of Fig. 1 is a central emotion guidance device 10 which is remotely located from the users 1, 2.
  • the emotion guidance device 10 can also be integrated into the first
  • the wearable device 30 can comprise the emotion guidance device 10.
  • the emotion guidance device 10 can be integrated into the biofeedback device 26, 28.
  • the emotion guidance device 10 can be integrated into the lamp shown in Fig. 2.
  • the emotion guidance device 10 can be a static device.
  • Fig. 4a to 4c show schematic diagrams of the emotion guidance system 100 and device 10 according to different embodiments.
  • Fig. 4a shows a first user 1 and a second user 2 in the same environment, for example in the same room.
  • the first physiological signal 16 of the first user 1 from the first physiological sensor 22 and the second physiological signal 18 of the second user 2 from the second physiological sensor 24 are wirelessly transmitted and received at the emotion guidance device 10 over a wireless transmission link.
  • the first physiological sensor 22 and/or the second physiological sensor 24 can each be integrated into a wearable device 30 wearable by the user.
  • the emotion guidance device 10 can be a separate part, as shown in Fig. 4a.
  • the biofeedback device 26, 28 can be a separate part or can be incorporated into the emotion guidance device 10 or the wearable device 30.
  • Fig. 4a One example for the embodiment of Fig. 4a is mother-child-care.
  • the emotion guidance system 100 and device 10 can enable that a mother, such as the first user 1 shown in Fig. 4a, is better in tune with her child, such as the second user 2 shown in Fig. 4a.
  • the mother 1 can better understand the child's 2 needs and can take better care of the child 2.
  • the emotion guidance device 10 of Fig. 4a can be incorporated into a baby phone.
  • the feedback only leads to be provided to the mother 1.
  • the mother can wear a wrist band which comprises the physiological sensor 22 and the biofeedback can be provided to the baby phone, for example when the mother 1 is in the same room as the child 2.
  • FIG. 4a Another example of the embodiment of Fig. 4a is a therapist-client- relationship. It is the central job of a therapist, such as the first user 1 in Fig. 4a, to synchronize with the client, such as the second user 2 in Fig. 4a. This enables the therapist 1 to empathize and guide the client 2.
  • a further example of the embodiment shown in Fig. 4a is a doctor-patient-relationship.
  • a doctor such as the first user 1 shown in Fig. 4a, might have to bring bad news to a patient, such as the second user 2 shown in Fig. 4a.
  • the doctor 1 can benefit from the fact that he is in synchronization with the patient 2.
  • Fig. 4b shows a schematic diagram of another embodiment, in which the users
  • the emotion guidance system can comprise a first emotion guidance device 10a in the environment of the first user 1 and a second emotion guidance device 10b in the environment of the second user 2.
  • the first user 1 and the second user 2 can each be using a telephone, such as a mobile telephone.
  • the first emotion guidance device 10a can be integrated into a first telephone and the second emotion guidance device 10b can be integrated into a second telephone.
  • the emotion guidance devices 10a, 10b the users 1 , 2 will be able to synchronize, even though they are in different environments or rooms.
  • Fig. 4c shows a schematic diagram of another embodiment, in which a first user 1 supervises a group of users 2, 3 as a supervisor or team manager.
  • the group can not only consist of two users (like in this example) but any higher number of users.
  • the emotion guidance device 10 can be integrated into a mobile telephone used by a supervisor or team manager, such as the first user 1 shown in Fig. 4c.
  • the emotion guidance device 10 can be implanted in software as a mobile telephone tool.
  • the supervisor or team manager 1 can for example observe which team members 2, 3 do not fit well within the team.
  • the input interface 1 1 of the emotion guidance device 10 receives the first physiological signal 16 of the first user 1 from the first physiological sensor 22, the second physiological signal 18 of the second user 2 from the second physiological sensor 24, and further receives a third physiological signal 18a of a third user 3 from a third physiological sensor 24a.
  • the calculating unit 13 calculates a first intermediary synchronization index Ii based on the first physiological signal 16 of the first user 1 and the second physiological signal 18 of the second user 2.
  • the calculating unit 13 further calculates a second intermediary synchronization index I 2 based on the first physiological signal 16 and the third physiological signal 18a.
  • the calculating unit 13 calculates the final synchronization index I by calculating an average of the first intermediary synchronization index Ii and the second intermediary synchronization index I 2 .
  • a weighted average can be calculated. It will be understood that this approach can also be used when physiological signals of multiple other users are received.
  • physiological signals can be evaluated individually.
  • Another example for the embodiment shown in Fig. 4c is team performance.
  • team performance can increase the synchronization between team members, such as users 1 , 2, 3 shown in Fig. 4c.
  • all users are all equally important.
  • the group can not only consist of three users (like in this example) but any higher number of users.
  • Biofeedback can for example be provided to team members that are out of synchronization and can help them to acquire the synchronization.
  • the emotion guidance device 10 can be integrated into a mobile telephone used the users 1 ,2 3.
  • the emotion guidance device 10 can be implanted in software as a mobile telephone tool.
  • the input interface 1 1 of the emotion guidance device 10 receives the first physiological signal 16 of the first user 1 from the first physiological sensor 22, the second physiological signal 18 of the second user 2 from the second physiological sensor 24, and further receives the third physiological signal 18a of a third user 3 from a third physiological sensor 24a.
  • the calculating unit 13 calculates a first intermediary synchronization index Ii based on the first physiological signal 16 of the first user 1 and the second physiological signal 18 of the second user 2.
  • the calculating unit 13 further calculates a second intermediary synchronization index I 2 based on the first physiological signal 16 and the third physiological signal 18a.
  • the calculating unit 13 calculates a third intermediary synchronization index I 3 based on the second physiological signal 18 and the third physiological signal 18a. In this way, the physiological signal of each user in the group is compared to each of the other users' physiological signals. An intermediary synchronization index for each possible user pair is calculated. Then, the calculating unit 13 calculates the final synchronization index I by calculating an average of the first intermediary synchronization index Ii, the second intermediary synchronization index I 2 , and the third synchronization index I3. The final synchronization index I is thus based on all of the calculated intermediary synchronization indices. In particular, a weighted average can be calculated. It will be understood that this approach can also be used when physiological signals of multiple other users are received. Each of the physiological signals can be evaluated individually.
  • FIG. 4c Another example for the embodiment shown in Fig. 4c is a teacher-student- relationship.
  • the educational qualities of a teacher such as the first user 1 shown in Fig. 4c, are likely to be highest when the teacher 1 understands the students, such as the users 2, 3 shown in Fig. 4c.
  • the teacher 1 can then empathize with the students 2, 3.
  • the student 2, 3 identifies the similarity with the teacher 1 she or he will probably listen more carefully and learn better.
  • FIG. 4c Another example for the embodiment shown in Fig. 4c is an entertainer- audience-relationship.
  • an entertainer such as the first user 1 shown in Fig. 4c
  • an audience such as users 2, 3 shown in Fig. 4c
  • it is beneficial if she or he is in synchronization with the audience. If the entertainer is not in synchrony with the audience, the messages might not come across correctly.
  • biofeedback will now be explained in more detail.
  • open feedback can be used, so that both the first user 1 and the second user 2 understand that they share a common goal. This can in particular be important, when the first user 1 and the second user 2 have equal roles.
  • the roles of the first user 1 and the second user 2 are not equal, such as a therapist-client-relationship or a teacher-student- relationship, it can be important to shield one of the users from the biofeedback.
  • private biofeedback can be used, in which the biofeedback is provided to only one single user.
  • haptic feedback can be beneficial, as it can be easily incorporated into a variable device. Haptic feedback can also be beneficial, when private feedback is required.
  • visual or audio biofeedback can also be used.
  • color or intensity of light can indicate synchronization, so that all users can see it at the same time.
  • audio feedback different types or properties of the audio, such as pitch or volume, can also be used to indicate the level of synchronization.
  • the biofeedback signal 19 is repeatedly or continuously output to the biofeedback device 26, 28. Also, the biofeedback device 26, 28 can repeatedly or
  • biofeedback can be provided continuously in real-time. However, this can be intentionally demanding. Thus, biofeedback can also be provided repeatedly, for example at preset intervals (e.g. every minute). In this way, the intentional demand of the biofeedback can be limited. Again, this depends on the usage domain. For example, in the context of team performance, additional cognitive resources of the user are also required for the main task and occasional feedback might be better. Any task requires cognitive resources of the user. Hence, if the user continuously attends to the biofeedback in real-time, this requires cognitive resources of the user all the time. Because every user/person has only a limited amount of cognitive resources, this might decrease performance on other parallel tasks.
  • the biofeedback device might give biofeedback at preset intervals (e.g. every minute), to make sure the main task that is being conducted by the user is not compromised.
  • preset intervals e.g. every minute
  • separate biofeedback on the progress of each of the first and second group can be provided to an impartial facilitator or mediator, in order to guide the adverse groups towards synchronization.
  • an impartial facilitator or mediator in order to guide the adverse groups towards synchronization.
  • biofeedback with an explicit meaning, like a specific color of light to signal the amount of
  • synchronization can be used.
  • implicit biofeedback can be used, that will the user learn to use over time, like music quality.
  • the meaning of the biofeedback can be explained to the user or it can be left implicit, so that the users have to learn themselves.
  • the users can be told what the biofeedback means, and so the users explicitly know that when the color of the light changes they should try to guide their emotions towards synchronization, thus be more empathic. This might work for some users/people, but not for everyone. For other users/people, it might be better to leave the meaning of the biofeedback implicit, so that they don't know what the biofeedback represents, but they will learn it unconsciously during use of the device or system.
  • the described emotion guidance device and system does not merely give biofeedback about the physiological state of the other user which would be insufficient to improve synchronization. It would be difficult to recognize patterns in the other user's physiological state and it would be difficult to compare it to the user's own physiological state.
  • the described emotion guidance provides an unobtrusive and practical implementation in which both users share a common emotional stimulus. For instance the same stimulus can have very different effects on the users, such as on a baby and the mother.
  • the described emotion guidance provides an automated comparison of the physiological signals and automated calculation of a synchronization index based on the physiological signals. Further, through interpersonal biofeedback the state of the level of synchronization of emotions of the users is provided to the users.
  • the described emotion guidance uses biofeedback that provides the users feedback on their synchronization, so that they know when to improve synchronization.
  • This biofeedback may be an explicit informational representation of the biofeedback, such as a color denoting the level of synchronization. It may however also be an implicit informational representation, such as to an increase in pleasurable biofeedback like music, or the decrease of an unpleasant stimulus like noise. Over time, the users can learn which strategies work to improve their synchronization and which strategies do not work.
  • the described emotion guidance can be implemented for one- to-one interactions, or can be used for many-to-many interactions or one-to-many
  • a computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.
  • a suitable medium such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.

Abstract

The present invention relates to an emotion guidance device (10) for guiding emotions of at least a first user (1) and a second user (2) towards synchronization or de- synchronization. The device comprises an input interface (11) for receiving a first physiological signal (16) of the first user (1) from a first physiological sensor (22) and a second physiological signal (18) of the second user (2) from a second physiological sensor (24). The device further comprises a calculating unit (13) for calculating a synchronization index (I) based on the first physiological signal (16) and the second physiological signal (18), the synchronization index (I) representing the level of synchronization of emotions of the users (1, 2). Further, the device comprises a control unit (14) for controlling at least one biofeedback signal (19) depending on the calculated synchronization index (I), and an output interface (15) for repeatedly or continuously outputting the at least one biofeedback signal (19) to at least one biofeedback device (26, 28).

Description

Emotion Guidance Device and Method
FIELD OF THE INVENTION
The present invention relates to an emotion guidance device and method for guiding emotions of at least a first user and a second user towards synchronization or de- synchronization. The present invention also relates to a wearable device, a biofeedback device, and an emotion guidance system, each comprising such an emotion guidance device. The present invention also relates to a computer program for implementing such a method.
BACKGROUND OF THE INVENTION
WO 2007/042947 Al discloses a handheld device comprising a heartbeat sensor for monitoring a first heartbeat signal of a user of the handheld device, a
communication unit for receiving a second heartbeat signal of a user of another handheld device, a comparison unit for comparing the first and second heartbeat signals so as to determine a level of coherence between them, and a decision unit for indicating a potential lover if the level of coherence is above a given threshold. The indication can be delivered to the user via a specific ringing tone or via a given message on the screen. In this way, the handheld device indicates a potential lover to the user of the handheld device. However, this functionality is very specialized.
SUMMARY OF THE INVENTION
It is therefore an object of the present invention to provide a device with more functionality. It is also an objective of the present invention to provide a respective wearable device, biofeedback device, system and computer program.
It has been found that there is a particular need to provide a device for guiding emotions, as one of the essential mechanisms of daily social interactions is that of communicating or sharing emotions. Thus, the present invention provides an emotion guidance device and method for guiding emotions of at least a first user and a second user towards synchronization or de-synchronization, in particular in a repeated or continuous manner. In a first aspect of the present invention an emotion guidance device for guiding emotions of at least a first user and a second user towards synchronization or de- synchronization is presented. The device comprises an input interface for receiving a first physiological signal of the first user from a first physiological sensor and a second physiological signal of the second user from a second physiological sensor. The device further comprises a calculating unit for calculating a synchronization index based on the first physiological signal and the second physiological signal, the synchronization index representing the level of synchronization of emotions of the users. The device further comprises a control unit for controlling at least one biofeedback signal depending on the calculated synchronization index, and an output interface for repeatedly or continuously outputting the at least one biofeedback signal to at least one biofeedback device.
In a further aspect of the present invention a wearable device wearable by a user is presented, in particular a wristband. The wearable device comprises the emotion guidance device and the physiological sensor for sensing the physiological signal of the user.
In a further aspect of the present invention a biofeedback device comprising the emotion guidance device is presented. The biofeedback device is adapted for providing biofeedback based on the biofeedback signal.
In a further aspect of the present invention an emotion guidance system is presented. The system comprises the emotion guidance device, the first physiological sensor for sensing the first physiological signal of the first user, the second physiological sensor for sensing the second physiological signal of the second user, and the at least one biofeedback device for providing biofeedback based on the at least one biofeedback signal.
In a further aspect of the present invention an emotion guidance method for guiding emotions of at least a first user and a second user towards synchronization or de- synchronization is presented. The method comprises receiving a first physiological signal of the first user from a first physiological sensor, and receiving a second physiological signal of the second user from a second physiological sensor. The method further comprises calculating a synchronization index based on the first physiological signal and second physiological signal, the synchronization index representing the level of synchronization of the users. The method further comprises controlling at least one biofeedback signal depending on the calculated synchronization index, and repeatedly or continuously outputting the at least one biofeedback signal to at least one biofeedback device. In a further aspect of the present invention a computer program comprising program code means for causing a computer to carry out the steps of the emotion guidance method when said computer program is carried out on the computer is presented.
The basic idea of the invention is to calculate a synchronization index based on the first physiological signal and the second physiological signal, the synchronization index representing the level of synchronization of emotions of the users, and to control the repeated or continuous biofeedback signal output depending on that calculated
synchronization index. In this way a (more or less) continuous feedback loop through the user is provided, with repeated or continuous updates. With this repeatedly updated or continuous biofeedback signal output it is made easy for the users to enhance (or worsen) their synchronization of emotions, such as mutual empathy. Repeated output is output that is not only performed once (e.g. upon detection of a certain criterion), but output that is performed a second time, a third time and so on, thus repeatedly. For example, the output can be repeated at preset time intervals (e.g. every minute). For example, the preset time interval can be between 1 second and 10 minutes, in particular between 5 seconds and 5 minutes, in particular between 30 seconds and 3 minutes, in particular between 1 minute and 2 minutes. The time intervals can be constant or non-constant. Continuously output can be at very short time intervals (e.g. such that the output appears continuous to the user) or without interruption, for example in real-time. For example, the output can be provided at the same rate as the synchronization index is calculated.
As already mentioned, in this way a (more or less) continuous feedback loop through the user is provided. When guiding emotions towards synchronization, the users can enhance their synchronization. The users can then gradually get in tune with each other. Alternatively, when guiding emotions towards de-synchronization, the users can worsen their synchronization (de-synchronization). The users can then gradually get out of tune with each other. In general, the idea is that the users can (repeatedly or continuously) track their synchronization level with the other user and therefore adapt their emotions. The users will be primed to enhance (or worsen) their synchronization of emotions and the users see the direct effects of their efforts. For example, users can control their physiological state by cognitive strategies like changing their own emotional state, trying to emphasize or focus more on the other person, or by physiological strategies like changing their breathing. All these strategies influence the synchronization of emotions with the other person.
In addition, the biofeedback based on the biofeedback signal could also be presented in a less conscious way. For instance, the biofeedback could be provided through varying or changing the quality of an audio output, such as background music. For example, if the synchronization index is low, the quality of the audio output could be relatively bad. The user would then gradually learn that enhancing his empathy with the other user (e.g. his partner), will result in better quality audio output, since the synchronization index will go up. Thus, the users are gently guided towards a higher synchronization, without their explicit awareness of the synchronization index or synchronization state as a necessary prerequisite.
Preferred embodiments of the invention are defined in the dependent claims. It shall be understood that the claimed emotion guidance method, computer program, wearable device, biofeedback device or emotion guidance system has similar and/or identical preferred embodiments as the claimed emotion guidance device and as defined in the dependent claims.
In one embodiment, the synchronization index is calculated using at least one measure selected from the group comprising an absolute difference between the first and second physiological signals, the difference between the derivatives of the first and second physiological signals, the correlation of the first and second physiological signals, the correlation of the derivatives of the first and second physiological signals, the coherence of the first and second physiological signals, the coherence of the derivatives of the first and second physiological signals, and the difference between the directions of the first and second physiological signals. Using the measure of the absolute difference between the first and the second physiological signals provides for an easy implementation of calculating the synchronization index. The measures of the difference between the derivatives of the first and second physiological signals and the measure of the correlation or coherence of the first and second physiological signals or their derivatives, provides for a more advanced calculation of the synchronization index. Using the measure of the difference between the directions of the first and second physiological signals is another approach.
In a further embodiment, the input interface is further adapted to receive a third physiological signal of a third user from a third physiological sensor, and the calculating unit adapted to calculate a first intermediary synchronization index based on the first physiological signal and the second physiological signal and a second intermediary synchronization index based on the first physiological signal and third physiological signal. This enables the first user to supervise a group of users, such as a teacher teaching a group of students or an entertainer entertaining a group of people.
In a variant of this embodiment, the calculating unit is further adapted to also calculate a third intermediary synchronization index based on the second physiological signal and the third physiological signal. In this way, the physiological signal of each user in the group is compared to each of the other users' physiological signals. An intermediary synchronization index for each possible user pair is calculated. This variant is in particular useful if all users (the first, second and third user) are equally important.
In another variant of this embodiment, wherein the calculating unit is adapted to calculate a final synchronization index by calculating an average of the first intermediary synchronization index and the second synchronization intermediary index. This enables the first user to present a synchronization index representative of the whole group of users supervised by the first user. In particular, the synchronization index can be calculated by calculating a weighted average, so as to give certain users more weight.
In a variant of this variant, in combination with the variant of calculating a third synchronization index, the calculating unit is adapted to calculate a final
synchronization index by calculating an average of the first intermediary synchronization index, the second synchronization intermediary index and the third intermediary
synchronization index. Thus, the final synchronization index is based on all of the calculated intermediary synchronization indices. This variant is in particular useful if all users (the first, second and third user) are equally important.
In a further embodiment, the emotion guidance device further comprises a preprocessing unit for pre-processing each of the first physiological signal and the second physiological signal, wherein the synchronization index is calculated based on the pre- processed first physiological signal and the pre-processed second physiological signal. This improves the quality of the signal.
Still further, in an embodiment, the physiological signal, in particular the first physiological signal, the second physiological signal and/or the third physiological signal, comprises or is at least one signal selected from the group comprising a skin conductance signal, a skin temperature signal and a respiration signal. These physiological signals are particularly suitable for representing the emotion of the user. These physiological signals are known to be coupled to emotions in general. In particular, skin conductance is very strongly coupled to emotions and has relatively fast response times. The skin conductance rises sharply within a few seconds after the experience of a stressor. Furthermore, skin
conductance is strongly coupled to sympathetic activity of the nervous system, compared to cardiovascular signals which are also coupled to parasympathetic branches of the nervous system. Thus, it is difficult to know if changes in a cardiovascular signal are due to sympathetic or parasympathetic shifts, but a skin conductance signal can be used to identify sympathetic shifts of the nervous system. Moreover, skin conductance can be relatively unobtrusively measured with a skin conductance sensor, as opposed to measurements of electrocardiograms which are relatively obtrusive. Skin temperature can also be
unobtrusively measured. Skin temperature changes more gradually and is therefore better suited for longer term measurements. Respiration can be under conscious control of the user. Thus, it can be easier for the users to synchronize, especially when the goal of
synchronization is made explicit to the users.
According to a further embodiment, wherein the at least one biofeedback signal is a signal for providing biofeedback to the user by changing color or intensity of light output and/or changing type or quality of audio output. This enables the users to easily synchronize (or de-synchronize) their emotions, for example while relaxing. Here, the biofeedback signal is presented in a less conscious way. Thus, the users are gently guided toward a higher (or lower) synchronization, without their explicit awareness of the synchronization index or synchronization state as a necessary prerequisite.
In another embodiment, wherein the at least one biofeedback signal is a signal for increasing a pleasant biofeedback output and/or decreasing an unpleasant biofeedback output, when the synchronization index increases. This enables the users to easily
synchronize their emotions, for example by playing more pleasant music, when the synchronization effect increases.
Still further, in an embodiment, the emotion guidance device further comprises a wireless communication link for receiving the first physiological signal and/or second physiological signal, and/or a wireless communication link for transmitting the biofeedback signal. This provides for a more flexible emotion guidance device, for example a portable emotion guidance device.
In another embodiment, the emotion guidance device further comprises at least one memory unit for storing the calculated synchronization indices over time. This enables that the calculated synchronization indices over time, thus the synchronization data, can be used after the emotion guidance session to evaluate the process. Alternatively or
cumulatively, the memory unit stores the first physiological signal and second physiological signal over time. This enables to calculate the synchronization index based on the stored data over time, which improves accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
These and other aspects of the invention will be apparent from and elucidated with reference to the embodiment(s) described hereinafter. In the following drawings
Fig. 1 shows a schematic diagram of an emotion guidance system and device according to an embodiment,
Fig. 2 shows a perspective view of a biofeedback device according to an embodiment,
Fig. 3 shows a perspective view of a wearable device according to an embodiment,
Fig. 4a to Fig. 4c each shows a schematic diagram of the emotion guidance system and device according to different embodiments.
DETAILED DESCRIPTION OF THE INVENTION
Fig. 1 shows a schematic diagram of an emotion guidance system 100 and an emotion guidance device 10 according to an embodiment. The emotion guidance system 100 and emotion guidance device 10 guide emotions of at least a first user 1 and a second user 2 towards synchronization or de-synchronization. The emotion guidance system 100 comprises the emotion guidance device 10, a first physiological sensor 22 for sensing a first
physiological signal 16 of the first user 1, a second physiological sensor 24 for sensing a second physiological signal 18 of the second user 2, and at least one biofeedback device 26, 28 for providing biofeedback based on at least one biofeedback signal 19. In the embodiment of Fig. 1, the emotion guidance device 10 is a separate part. It will be understood that the emotion guidance device 10 can also be incorporated into any other part or entity, such as for example the biofeedback device 26, 28 or a wearable device 30, as will be explained.
The first physiological sensor 22 and/or the second physiological sensor 24 can comprise at least one sensor selected from the group comprising a skin conductance sensor, a skin temperature sensor and a respiration sensor. Thus, the first physiological signal 16 and/or the second physiological signal 18 can comprise at least one signal selected from the group comprising a skin conductance signal, a skin temperature signal and a respiration signal. These physiological signals are known to be coupled to emotions in general. In particular, skin conductance is very strongly coupled to emotions and has relatively fast response times. Thus, the most relevant sensor and signal for determining the synchronization of emotions is an electrodermal activity (EDA), or skin conductance sensor and signal. The skin conductance rises sharply within a few seconds after the experience of a stressor. The amount of skin conductance level change between an onset point and a peak point is correlated with the intensity of arousal. A smooth decrease of the skin conductance, in particular according to an exponential Arrhenius behavioral curve (l/exp(A/t)), indicates relaxation.
Electrodermal activity, or skin conductance, is normally measured between two electrodes which are placed on the skin of the user 1. In particular, it can be measured unobtrusively on the wrist, ankle, feet, or hands. Other sensors and signals from which emotional synchronization can be determined are a skin temperature sensor and signal and a respiration sensor and signal. For example, skin temperature is measured using a skin temperature sensor placed on the skin. Thus, skin temperature can also be unobtrusively measured. Generally, skin temperature reacts slowly to arousal (rising) and relaxation (dropping). Thus, skin temperature changes more gradually and is therefore better suited for longer term measurements. Respiration can be measured using a respiration sensor, such as an accelerometer or gauge device, for example around the chest or waist of the user. Another example is respiration measurement by means of video, for example by extracting the chest/abdomen movement from a video stream. Generally, respiration reacts fast to startle (e.g. holding breath for a short time) and shows relaxation by slow regular abdominal breathing (e.g. 6-8 times/minute). Respiration can be under conscious control of the user. Thus, it can be easier for the users to synchronize, especially when the goal of
synchronization is made explicit to the users.
The emotion guidance device 10 will now be explained in more detail.
Emotion guidance device 10 comprises an input interface 11 for receiving the first physiological signal 16 of the first user 1 from the first physiological sensor 22 and the second physiological signal 18 of the second user 2 from the second physiological sensor 24. In the embodiment of Fig. 1, the emotion guidance device further comprises a pre-processing unit 12 for pre-processing each of the first physiological signal 16 and the second
physiological signal 18 received from the input interface 11. The pre-processing can for example comprise at least one operation selected from the group comprising smoothing of the signal, frequency filtering of the signal, and normalization of the signal. The
normalization of the signal can for example comprise mean subtraction or division by standard deviation.
The emotion guidance device 10 further comprises a calculating unit 13 for calculating a synchronization index I based on the pre-processed first physiological signal 16 and the pre-processed second physiological signal 18. The synchronization index I represents the level of synchronization of the emotions of the first user 1 and the second user 2. In particular, the synchronization index I is calculated using at least one measure selected from the group comprising an absolute difference between the first and second physiological signals 16, 18, the difference between the derivatives of the first and second physiological signals 16, 18, the correlation of the first and second physiological signals 16, 18, the correlation of the derivatives of the first and second physiological signals 16, 18, the coherence of the first and second physiological signals 16, 18, the coherence of the derivatives of the first and second physiological signals 16, 18, and the difference between the directions of the first and second physiological signals 16, 18. Calculating the absolute difference between the signals can in particular be used after normalization has been performed by the pre-processing unit 12. Using the difference between the derivatives of the signals (derivative matching) uses the differences of the slope of the signals, and thus the shape of the signal. Using the correlation of the signals or their derivatives can in particular be performed after a phase shift operation. Using the directions of the signals is another approach. Each of the above-mentioned calculation measures can provide a synchronization index I which is number from 0 to 1. A synchronization index I of 0 represents a minimal level of synchronization and a synchronization index I of 1 represents a maximal level of synchronization, or vice versa. The first and second physiological signals 16, 18 can for example be compared at different times or lags. Thus, the timing between the first and second physiological signals 16, 18 is shifted. The lag can for example be zero. However, the lag can also be a positive number, thus larger than zero, to take into account the time it takes for an emotion to get the other user (e.g. 5 seconds).
The emotion guidance device 10 further comprises a control unit 14 for controlling at least one biofeedback signal 19 depending on the calculated synchronization index I. The emotion guidance device 10 further comprises an output interface 15 for repeatedly or continuously outputting the at least one biofeedback signal 19 to at least one biofeedback device 26, 28. In this way a (more or less) continuous feedback loop through the user is provided. As the at least one biofeedback signal 19 is repeatedly or continuously output, the control unit 14 can repeatedly or continuously control the at least one biofeedback signal 19, the calculating unit 13 can repeatedly or continuously calculate the synchronization index I, and/or the input interface 11 can repeatedly or continuously receive the first physiological signal and the second physiological signal 18.
The emotion guidance device 10 in the embodiment of Fig. 1 further comprises a first memory unit 17 for storing the calculated synchronization indices I over time. This synchronization data can be used afterwards to evaluate the process. Thus, an overview or summary is provided which can be evaluated, and the users may know how to change their behavior.
The physiological sensors 22, 24 can repeatedly or continuously sense the physiological signals 16, 18. All the sensed signal can be further processed (repeatedly or continuously) or only the sensed signal for a specific time period can be used, for example at least ten seconds, in particular between one minute and ten minutes. The emotion guidance device 10 can comprise a second memory unit 17a for storing the first physiological signal 16 and the second physiological signal 18 over time, as can be seen in Fig. 1. This stored data can then be transmitted to the calculating unit 13, or first to the pre-processing unit 12. For example, a running window can be used over this data. Updates of this data can be made at high frequency, so that changes are reflected immediately to the user.
It shall be understood that any number of units selected from the group comprising the input interface 11, pre-processing unit 12, calculating unit 13, control unit 14 and output interface 15 can be implemented in one single processing device, such as for example a processor (e.g. microprocessor) or any other suitable processing device. In particular, the calculating unit 13 and the control unit 14 (and optionally the pre-processing unit 12) can be implemented in one single processing device. Also, the first memory unit 17 and the second memory unit 17a can be separate entities, or can be implemented in one single memory device.
The emotion guidance device 10 can further comprise a wireless communication link for receiving the first physiological signal 16 and/or the second physiological signal 18, as for example illustrated in Figs. 4a to 4c. For example, the physiological signals 16, 18 can be transmitted using a bluetooth communication link.
Alternatively or cumulatively, the emotion guidance device 10 can further comprise a wireless communication link for transmitting the biofeedback signal 19.
The biofeedback device 26, 28 provides an indication of the level of synchronization of emotions of the users 1, 2 to the users 1, 2. In the embodiment shown in Fig. 1, the biofeedback signal 19 is output to a single biofeedback device 26 (solid line in Fig. 1). The first biofeedback device 26 is here in the environment El of the first user 1. In this case, the biofeedback is provided to only the first user 1, not to the second user.
Alternatively, the first biofeedback device 26 can be in the environment of both the first user 1 and the second user 2. In this case, the biofeedback is provided to both the first user 1 and the second user 2. Optionally, as indicated by the dashed line in Fig. 1, the biofeedback signal 19 can also be output to a second biofeedback device 28 in the environment E2 of the second user. Again, in this case, the biofeedback is provided to both the first user 1 and the second user 2. Also, the biofeedback device can be in the environment of another person which is not taking part in the emotion guidance, for example an impartial facilitator or mediator or a therapist.
The biofeedback device 26, 28 can be adapted for haptic, visual and/or audio biofeedback. In particular, the at least one biofeedback signal 19 can be a signal for providing biofeedback to the user by changing color or intensity of light output. Thus, the biofeedback device 26, 28 can be adapted to change color or intensity of light output. A perspective view of an embodiment of such a biofeedback device 26, 28 is shown in Fig. 2. The biofeedback device 26, 28 of Fig. 2 is a lamp, in particular an LED lamp, for outputting multiple colors, such as for example "LivingColors" by Philips. A remote control 29 can also be used as an extra control mechanism/device for the users (e.g. to switch the light on or off). For example, the skin conductance signal 16, 18 can be linked in software to the lamp 26, 28 by means of an address of the remote control 29, e.g. using a wireless PC dongle normally used for software firmware upload to the device.
Alternatively or cumulatively, the biofeedback signal 19 can be a signal for changing type or quality of audio output. For example, the type of audio output, such as music, can be changed by changing the pitch or volume of the audio output. Also, the quality of music can for example be changed.
In particular, when guiding emotions towards synchronization, the at least one biofeedback signal 19 can be a signal for increasing a pleasant biofeedback output and/or decreasing an unpleasant biofeedback output, when the synchronization index I increases. Similarly, the at least one biofeedback signal 19 can be a signal for decreasing a pleasant biofeedback output and/or increasing an unpleasant biofeedback output, when the
synchronization index I decreases. For example, when the synchronization index I increases, the lighting type, such as color or intensity of the light, in the room in which the users are present can be made more pleasant, thus improving the lighting atmosphere, or the quality of music to which the users listen can be made better. Because the users have a natural tendency to prefer a more pleasant biofeedback, such as more pleasant lighting type or better quality music, the users will adjust their emotions towards synchronization, in order to achieve the more pleasant biofeedback. In an example, the first user can be in a very negative emotional state, and the second user can be in the opposite, thus positive, emotional state. The second user can try to cheer the first user up. The first user will then try to guide his/her emotions towards synchronization with the second user.
Alternatively, when guiding emotions towards de-synchronization, at least one biofeedback signal 19 can be a signal for increasing an unpleasant biofeedback output and/or decreasing a pleasant biofeedback output, when the synchronization index I increases. Similarly, the at least one biofeedback signal 19 can be a signal for decreasing an unpleasant biofeedback output and/or increasing a pleasant biofeedback output, when the
synchronization index I decreases. For example, when the synchronization index I increases, the lighting type, such as color or intensity of the light, in the room in which the users are present can be made less pleasant, thus worsen the lighting atmosphere, or the quality of music to which the users listen can be made worse. For example, before a competition or fight, the users will then adjust their emotions towards de-synchronization, in order to achieve the worse biofeedback and become thus more aggressive.
Fig. 3 shows a perspective view of an embodiment of a wearable device 30 wearable by a user. In the embodiment of Fig. 3, the wearable device 30 is in form of a wristband comprising a wristband material part 33 and a casing 34. It will be understood that the wearable device 30 could also be worn around any other suitable body part, such as the ankle, foot or hand. In Fig. 3, two skin conductance electrodes 31, 32 are integrated into the wrist band material 33. The skin conductance electrodes 31, 32 are used for sensing the skin conductance of the user. Thus, the wearable device 30 comprises the first physiological sensor 22 or the second physiological sensor 24. In this example, the physiological sensor 22, 24 is a skin conductance sensor. The physiological sensor 22, 24 can also be any other suitable sensor, such as a skin temperature sensor. Also, the wearable device 30 can for example comprise a respiration sensor and be in the form of a respiration belt.
Further, the wearable device 30 shown in Fig. 3 can for instance comprise the emotion guidance device 10, for example the emotion guidance device described with reference to Fig. 1. The emotion guidance device 10 can be integrated into the casing 34 of the wearable device 30. The wearable device 30 can further comprise a transmitter for wirelessly transmitting data over a wireless communication link, such as the biofeedback signal 19.
The emotion guidance device 10 shown in the embodiment of Fig. 1 is a central emotion guidance device 10 which is remotely located from the users 1, 2.
Alternatively, the emotion guidance device 10 can also be integrated into the first
physiological sensor 22 and/or the second physiological sensor 24. For example, as explained with reference to Fig. 3, the wearable device 30 can comprise the emotion guidance device 10. In a further alternative, the emotion guidance device 10 can be integrated into the biofeedback device 26, 28. For example, the emotion guidance device 10 can be integrated into the lamp shown in Fig. 2. Also, the emotion guidance device 10 can be a static device.
Fig. 4a to 4c show schematic diagrams of the emotion guidance system 100 and device 10 according to different embodiments. Fig. 4a shows a first user 1 and a second user 2 in the same environment, for example in the same room. The first physiological signal 16 of the first user 1 from the first physiological sensor 22 and the second physiological signal 18 of the second user 2 from the second physiological sensor 24 are wirelessly transmitted and received at the emotion guidance device 10 over a wireless transmission link. The first physiological sensor 22 and/or the second physiological sensor 24 can each be integrated into a wearable device 30 wearable by the user. The emotion guidance device 10 can be a separate part, as shown in Fig. 4a. The biofeedback device 26, 28 can be a separate part or can be incorporated into the emotion guidance device 10 or the wearable device 30.
One example for the embodiment of Fig. 4a is mother-child-care. The emotion guidance system 100 and device 10 can enable that a mother, such as the first user 1 shown in Fig. 4a, is better in tune with her child, such as the second user 2 shown in Fig. 4a. Thus, the mother 1 can better understand the child's 2 needs and can take better care of the child 2. For instance, the emotion guidance device 10 of Fig. 4a can be incorporated into a baby phone. The feedback only leads to be provided to the mother 1. The mother can wear a wrist band which comprises the physiological sensor 22 and the biofeedback can be provided to the baby phone, for example when the mother 1 is in the same room as the child 2.
Another example of the embodiment of Fig. 4a is a therapist-client- relationship. It is the central job of a therapist, such as the first user 1 in Fig. 4a, to synchronize with the client, such as the second user 2 in Fig. 4a. This enables the therapist 1 to empathize and guide the client 2. A further example of the embodiment shown in Fig. 4a is a doctor-patient-relationship. A doctor, such as the first user 1 shown in Fig. 4a, might have to bring bad news to a patient, such as the second user 2 shown in Fig. 4a. The doctor 1 can benefit from the fact that he is in synchronization with the patient 2.
Fig. 4b shows a schematic diagram of another embodiment, in which the users
1, 2 are not in the same environment, but in different environments, such as in different rooms (indicated by the vertical dashed line in Fig. 4b). The emotion guidance system can comprise a first emotion guidance device 10a in the environment of the first user 1 and a second emotion guidance device 10b in the environment of the second user 2. For example, the first user 1 and the second user 2 can each be using a telephone, such as a mobile telephone. The first emotion guidance device 10a can be integrated into a first telephone and the second emotion guidance device 10b can be integrated into a second telephone. When communicating over a distance, it is normally very difficult to tune into another person, as there is limited information available. For instance, facial expressions are missing when using a telephone. When using the emotion guidance devices 10a, 10b, the users 1 , 2 will be able to synchronize, even though they are in different environments or rooms.
Fig. 4c shows a schematic diagram of another embodiment, in which a first user 1 supervises a group of users 2, 3 as a supervisor or team manager. It will be understood that the group can not only consist of two users (like in this example) but any higher number of users. For example, the emotion guidance device 10 can be integrated into a mobile telephone used by a supervisor or team manager, such as the first user 1 shown in Fig. 4c. For example, the emotion guidance device 10 can be implanted in software as a mobile telephone tool. The supervisor or team manager 1 can for example observe which team members 2, 3 do not fit well within the team. In the embodiment shown in Fig. 4c, the input interface 1 1 of the emotion guidance device 10 receives the first physiological signal 16 of the first user 1 from the first physiological sensor 22, the second physiological signal 18 of the second user 2 from the second physiological sensor 24, and further receives a third physiological signal 18a of a third user 3 from a third physiological sensor 24a. The calculating unit 13 calculates a first intermediary synchronization index Ii based on the first physiological signal 16 of the first user 1 and the second physiological signal 18 of the second user 2. The calculating unit 13 further calculates a second intermediary synchronization index I2 based on the first physiological signal 16 and the third physiological signal 18a. Then, the calculating unit 13 calculates the final synchronization index I by calculating an average of the first intermediary synchronization index Ii and the second intermediary synchronization index I2. In particular, a weighted average can be calculated. It will be understood that this approach can also be used when physiological signals of multiple other users are received. Each of the
physiological signals can be evaluated individually.
Another example for the embodiment shown in Fig. 4c is team performance. When having to work closely together, team performance can increase the synchronization between team members, such as users 1 , 2, 3 shown in Fig. 4c. In particular, all users (the first, second and third user 1 ,2,3) are all equally important. It will be understood that the group can not only consist of three users (like in this example) but any higher number of users. Biofeedback can for example be provided to team members that are out of synchronization and can help them to acquire the synchronization. For example, the emotion guidance device 10 can be integrated into a mobile telephone used the users 1 ,2 3. For example, the emotion guidance device 10 can be implanted in software as a mobile telephone tool. Thus, this is an example where physiological signals are received from more than two users and biofeedback signals are output back to more than two users. The input interface 1 1 of the emotion guidance device 10 receives the first physiological signal 16 of the first user 1 from the first physiological sensor 22, the second physiological signal 18 of the second user 2 from the second physiological sensor 24, and further receives the third physiological signal 18a of a third user 3 from a third physiological sensor 24a. The calculating unit 13 calculates a first intermediary synchronization index Ii based on the first physiological signal 16 of the first user 1 and the second physiological signal 18 of the second user 2. The calculating unit 13 further calculates a second intermediary synchronization index I2 based on the first physiological signal 16 and the third physiological signal 18a. Also, the calculating unit 13 calculates a third intermediary synchronization index I3 based on the second physiological signal 18 and the third physiological signal 18a. In this way, the physiological signal of each user in the group is compared to each of the other users' physiological signals. An intermediary synchronization index for each possible user pair is calculated. Then, the calculating unit 13 calculates the final synchronization index I by calculating an average of the first intermediary synchronization index Ii, the second intermediary synchronization index I2, and the third synchronization index I3. The final synchronization index I is thus based on all of the calculated intermediary synchronization indices. In particular, a weighted average can be calculated. It will be understood that this approach can also be used when physiological signals of multiple other users are received. Each of the physiological signals can be evaluated individually.
Another example for the embodiment shown in Fig. 4c is a teacher-student- relationship. The educational qualities of a teacher, such as the first user 1 shown in Fig. 4c, are likely to be highest when the teacher 1 understands the students, such as the users 2, 3 shown in Fig. 4c. The teacher 1 can then empathize with the students 2, 3. When the student 2, 3 identifies the similarity with the teacher 1 she or he will probably listen more carefully and learn better.
Another example for the embodiment shown in Fig. 4c is an entertainer- audience-relationship. For an entertainer, such as the first user 1 shown in Fig. 4c, in front of an audience, such as users 2, 3 shown in Fig. 4c, it is beneficial, if she or he is in synchronization with the audience. If the entertainer is not in synchrony with the audience, the messages might not come across correctly.
The biofeedback will now be explained in more detail. For example, open feedback can be used, so that both the first user 1 and the second user 2 understand that they share a common goal. This can in particular be important, when the first user 1 and the second user 2 have equal roles. In circumstances, where the roles of the first user 1 and the second user 2 are not equal, such as a therapist-client-relationship or a teacher-student- relationship, it can be important to shield one of the users from the biofeedback. Thus, private biofeedback can be used, in which the biofeedback is provided to only one single user.
Different modalities can be used to provide the biofeedback, such as haptic, visual and/or audio biofeedback. The modality of the biofeedback depends mainly on the context. If a mobile setting is used, haptic feedback can be beneficial, as it can be easily incorporated into a variable device. Haptic feedback can also be beneficial, when private feedback is required. When open feedback is used, visual or audio biofeedback can also be used. When using visual biofeedback, color or intensity of light can indicate synchronization, so that all users can see it at the same time. When using audio feedback, different types or properties of the audio, such as pitch or volume, can also be used to indicate the level of synchronization.
The biofeedback signal 19 is repeatedly or continuously output to the biofeedback device 26, 28. Also, the biofeedback device 26, 28 can repeatedly or
continuously provide biofeedback to the user based on the biofeedback signal 19. For example, the biofeedback can be provided continuously in real-time. However, this can be intentionally demanding. Thus, biofeedback can also be provided repeatedly, for example at preset intervals (e.g. every minute). In this way, the intentional demand of the biofeedback can be limited. Again, this depends on the usage domain. For example, in the context of team performance, additional cognitive resources of the user are also required for the main task and occasional feedback might be better. Any task requires cognitive resources of the user. Hence, if the user continuously attends to the biofeedback in real-time, this requires cognitive resources of the user all the time. Because every user/person has only a limited amount of cognitive resources, this might decrease performance on other parallel tasks. Therefore, in certain usage domains the biofeedback device might give biofeedback at preset intervals (e.g. every minute), to make sure the main task that is being conducted by the user is not compromised. In an example, if the described device or method is used in mediation between a first and a second group having a conflict, separate biofeedback on the progress of each of the first and second group can be provided to an impartial facilitator or mediator, in order to guide the adverse groups towards synchronization. For example, in the context of a therapist- client-relationship, it is the therapist's main activity to stay in synchronization with the client, so continuous biofeedback is more appropriate.
Also, implicit or explicit biofeedback can be provided. Either, biofeedback with an explicit meaning, like a specific color of light to signal the amount of
synchronization, can be used. Alternatively, implicit biofeedback can be used, that will the user learn to use over time, like music quality. The meaning of the biofeedback can be explained to the user or it can be left implicit, so that the users have to learn themselves. In one example, the users can be told what the biofeedback means, and so the users explicitly know that when the color of the light changes they should try to guide their emotions towards synchronization, thus be more empathic. This might work for some users/people, but not for everyone. For other users/people, it might be better to leave the meaning of the biofeedback implicit, so that they don't know what the biofeedback represents, but they will learn it unconsciously during use of the device or system. For instance, by decreasing the quality of audio output, such as music quality, when the synchronization index decreases, the users/people will want to increase the quality of the audio output (e.g. playing in the background) and therefore the users have to guide their emotions towards synchronization, thus get more in synchronization. In general, it has been found that, for all the different types of biofeedback, after a while the users will unconsciously learn how to do this.
It is important to note that the described emotion guidance device and system does not merely give biofeedback about the physiological state of the other user which would be insufficient to improve synchronization. It would be difficult to recognize patterns in the other user's physiological state and it would be difficult to compare it to the user's own physiological state. The described emotion guidance provides an unobtrusive and practical implementation in which both users share a common emotional stimulus. For instance the same stimulus can have very different effects on the users, such as on a baby and the mother. The described emotion guidance provides an automated comparison of the physiological signals and automated calculation of a synchronization index based on the physiological signals. Further, through interpersonal biofeedback the state of the level of synchronization of emotions of the users is provided to the users. Instead of trying to influence synchronization by providing the same emotional stimulus to the users, the described emotion guidance uses biofeedback that provides the users feedback on their synchronization, so that they know when to improve synchronization. This biofeedback may be an explicit informational representation of the biofeedback, such as a color denoting the level of synchronization. It may however also be an implicit informational representation, such as to an increase in pleasurable biofeedback like music, or the decrease of an unpleasant stimulus like noise. Over time, the users can learn which strategies work to improve their synchronization and which strategies do not work. The described emotion guidance can be implemented for one- to-one interactions, or can be used for many-to-many interactions or one-to-many
interactions.
While the invention has been illustrated and described in detail in the drawings and foregoing description, such illustration and description are to be considered illustrative or exemplary and not restrictive; the invention is not limited to the disclosed embodiments. Other variations to the disclosed embodiments can be understood and effected by those skilled in the art in practicing the claimed invention, from a study of the drawings, the disclosure, and the appended claims.
In the claims, the word "comprising" does not exclude other elements or steps, and the indefinite article "a" or "an" does not exclude a plurality. A single element or other unit may fulfill the functions of several items recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage.
A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state medium supplied together with or as part of other hardware, but may also be distributed in other forms, such as via the Internet or other wired or wireless telecommunication systems.
Any reference signs in the claims should not be construed as limiting the scope.

Claims

CLAIMS:
1. Emotion guidance device (10) for guiding emotions of at least a first user (1) and a second user (2) towards synchronization or de-synchronization, the device comprising:
an input interface (11) for receiving a first physiological signal (16) of the first user (1) from a first physiological sensor (22) and a second physiological signal (18) of the second user (2) from a second physiological sensor (24),
a calculating unit (13) for calculating a synchronization index (I) based on the first physiological signal (16) and the second physiological signal (18), the synchronization index (I) representing the level of synchronization of emotions of the users (1, 2),
a control unit (14) for controlling at least one biofeedback signal (19) depending on the calculated synchronization index (I), and
an output interface (15) for repeatedly or continuously outputting the at least one biofeedback signal (19) to at least one biofeedback device (26, 28).
2. Emotion guidance device (10) of claim 1, wherein the synchronization index (I) is calculated using at least one measure selected from the group comprising an absolute difference between the first and second physiological signals (16, 18), the difference between the derivatives of the first and second physiological signals (16, 18), the correlation of the first and second physiological signals (16, 18), the correlation of the derivatives of the first and second physiological signals (16, 18), the coherence of the first and second physiological signals (16, 18), the coherence of the derivatives of the first and second physiological signals (16, 18), and the difference between the directions of the first and second physiological signals (16, 18).
3. Emotion guidance device (10) of claim 1, the input interface (11) further adapted to receive a third physiological signal (18a) of a third user (3) from a third physiological sensor (24a), and the calculating unit (13) adapted to calculate a first intermediary synchronization index (Ii) based on the first physiological signal (16) and the second physiological signal (18) and a second intermediary synchronization index (I2) based on the first physiological signal (16) and third physiological signal (18a), in particular also a third intermediary synchronization index (I3) based on the second physiological signal (18) and the third physiological signal (18a).
4. Emotion guidance device (10) of claim 5, wherein the calculating unit (13) is adapted to calculate a final synchronization index (I) by calculating an average of the first intermediary synchronization index (Ii) and the second synchronization intermediary index (I2), in particular also of the third intermediary synchronization index (I3).
5. Emotion guidance device (10) of claim 1, further comprising a pre-processing unit (12) for pre-processing each of the first physiological signal (16) and the second physiological signal (18), wherein the synchronization index is calculated based on the pre- processed first physiological signal and the pre-processed second physiological signal.
6. Emotion guidance device (10) of claim 1, wherein the physiological signal (16, 18, 18a) comprises at least one signal selected from the group comprising a skin conductance signal, a skin temperature signal and a respiration signal.
7. Emotion guidance device (10) of claim 1, wherein the at least one biofeedback signal (19) is a signal for providing biofeedback to the user by changing color or intensity of light output and/or changing type or quality of audio output.
8. Emotion guidance device (10) of claim 1, wherein the at least one biofeedback signal (19) is a signal for increasing a pleasant biofeedback output and/or decreasing an unpleasant biofeedback output, when the synchronization index (I) increases.
9. Emotion guidance device (10) of claim 1, further comprising a wireless communication link for receiving the first physiological signal (16) and/or second physiological signal (18), and/or a wireless communication link for transmitting the biofeedback signal (19).
10. Emotion guidance device (10) of claim 1, further comprising at least one memory unit (17, 17a) for storing the calculated synchronization indices (I) over time and/or the first physiological signal (16) and second physiological signal (18) over time.
11. A wearable device (30) wearable by a user (1 ,2), the wearable device comprising the emotion guidance device (10) of claim 1 and the physiological sensor (22, 24) for sensing the physiological signal (16, 18) of the user (1, 2).
12. A biofeedback device (26, 28) comprising the emotion guidance device (10) of claim 1, the biofeedback device (30) adapted for providing biofeedback based on the biofeedback signal (19).
13. Emotion guidance system (100) comprising :
- the emotion guidance device (10) of claim 1,
the first physiological sensor (22) for sensing the first physiological signal (16) of the first user (1),
the second physiological sensor (24) for sensing the second physiological signal (18) of the second user (2), and
- the at least one biofeedback device (26, 28) for providing biofeedback based on the at least one biofeedback signal (19).
14. Emotion guidance method for guiding emotions of at least a first user (1) and a second user (2) towards synchronization or de- synchronization, the method comprising: - receiving a first physiological signal (16) of the first user (1) from a first physiological sensor (22),
receiving a second physiological signal (18) of the second user (2) from a second physiological sensor (24),
calculating a synchronization index (I) based on the first physiological signal (16) and second physiological signal (18), the synchronization index (I) representing the level of synchronization of emotions of the users (1, 2),
controlling at least one biofeedback signal (19) depending on the calculated synchronization index (I), and
repeatedly or continuously outputting the at least one biofeedback signal (19) to at least one biofeedback device (26, 28).
15. Computer program comprising program code means for causing a computer to carry out the steps of the emotion guidance method as claimed in claim 14 when said computer program is carried out on the computer.
PCT/IB2012/051817 2011-04-21 2012-04-13 Emotion guidance device and method WO2012143834A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP11163324.4 2011-04-21
EP11163324 2011-04-21

Publications (1)

Publication Number Publication Date
WO2012143834A1 true WO2012143834A1 (en) 2012-10-26

Family

ID=46025833

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2012/051817 WO2012143834A1 (en) 2011-04-21 2012-04-13 Emotion guidance device and method

Country Status (1)

Country Link
WO (1) WO2012143834A1 (en)

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20170103669A1 (en) * 2015-10-09 2017-04-13 Fuji Xerox Co., Ltd. Computer readable recording medium and system for providing automatic recommendations based on physiological data of individuals
CN107405120A (en) * 2015-04-23 2017-11-28 索尼公司 Information processor, control method and program
US20210075859A1 (en) * 2019-09-09 2021-03-11 Lg Electronics Inc. Server
US11227248B2 (en) 2018-08-21 2022-01-18 International Business Machines Corporation Facilitation of cognitive conflict resolution between parties
EP4181151A1 (en) * 2021-11-11 2023-05-17 Koninklijke Philips N.V. A system and method for assisting the development of an adult-child attachment relationship

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2007042947A1 (en) 2005-10-12 2007-04-19 Koninklijke Philips Electronics N.V. Handheld device for indicating a potential lover to the user of the device.
US20070270668A1 (en) * 1999-03-02 2007-11-22 Quantum Intech, Inc. Physiological coherence in animals

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20070270668A1 (en) * 1999-03-02 2007-11-22 Quantum Intech, Inc. Physiological coherence in animals
WO2007042947A1 (en) 2005-10-12 2007-04-19 Koninklijke Philips Electronics N.V. Handheld device for indicating a potential lover to the user of the device.

Cited By (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107405120A (en) * 2015-04-23 2017-11-28 索尼公司 Information processor, control method and program
EP3287075A4 (en) * 2015-04-23 2018-12-05 Sony Corporation Information processing device, control method, and program
US20170103669A1 (en) * 2015-10-09 2017-04-13 Fuji Xerox Co., Ltd. Computer readable recording medium and system for providing automatic recommendations based on physiological data of individuals
US11227248B2 (en) 2018-08-21 2022-01-18 International Business Machines Corporation Facilitation of cognitive conflict resolution between parties
US20210075859A1 (en) * 2019-09-09 2021-03-11 Lg Electronics Inc. Server
US11509722B2 (en) * 2019-09-09 2022-11-22 Lg Electronics Inc. Server
EP4181151A1 (en) * 2021-11-11 2023-05-17 Koninklijke Philips N.V. A system and method for assisting the development of an adult-child attachment relationship
WO2023083623A1 (en) 2021-11-11 2023-05-19 Koninklijke Philips N.V. A system and method for assisting the development of an adult-child attachment relationship

Similar Documents

Publication Publication Date Title
US11839473B2 (en) Systems and methods for estimating and predicting emotional states and affects and providing real time feedback
US20160339300A1 (en) Controlling user devices based on biometric readings
CN101198277B (en) Systems for physiological and psycho-physiological monitoring
EP3337401B1 (en) Method and system for adjusting audio signals based on motion deviation
US10292646B2 (en) Mobile health care device and operating method thereof
US11185281B2 (en) System and method for delivering sensory stimulation to a user based on a sleep architecture model
US10939866B2 (en) System and method for determining sleep onset latency
US20210141453A1 (en) Wearable user mental and contextual sensing device and system
WO2015190042A1 (en) Activity evaluation device, evaluation processing device, and program
US20210401338A1 (en) Systems and methods for estimating and predicting emotional states and affects and providing real time feedback
WO2012143834A1 (en) Emotion guidance device and method
CN112384131A (en) System and method for enhancing sensory stimuli delivered to a user using a neural network
WO2015095924A1 (en) A biofeedback, stress management and cognitive enhancement system
Hänsel et al. Wearable computing for health and fitness: exploring the relationship between data and human behaviour
Georgiou Rhythmic haptic cueing for gait rehabilitation of hemiparetic stroke and brain injury survivors
Isenstein et al. Rapid assessment of hand reaching using virtual reality and application in cerebellar stroke
US11497883B2 (en) System and method for enhancing REM sleep with sensory stimulation
US20170209335A1 (en) Physiological State Feedback System and Control Method Thereof
Jafarzadeh Esfahani et al. Citizen neuroscience: wearable technology and open software to study the human brain in its natural habitat
US20230107691A1 (en) Closed Loop System Using In-ear Infrasonic Hemodynography and Method Therefor
Oh Exploring Design Opportunities for Technology-Supported Yoga Practices at Home
US20230380793A1 (en) System and method for deep audio spectral processing for respiration rate and depth estimation using smart earbuds
US20210170138A1 (en) Method and system for enhancement of slow wave activity and personalized measurement thereof
Kressbach Bodies in Action
Oliveira Body Perception Manipulation with Real-Time Movement Sonification of Upper Body Exercises

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 12718399

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 12718399

Country of ref document: EP

Kind code of ref document: A1