JP7385892B2 - Emotion estimation system and emotion estimation device - Google Patents

Description

The present invention relates to a technique for estimating a subject's emotions.

Patent Document 1 discloses a technique for estimating the emotions of a subject based on biological information acquired from the subject.

Japanese Patent Application Publication No. 2013-046691

An object of the present invention is to provide a new user interface based on a subject's emotions estimated through the acquisition of biometric information.

One aspect of achieving the above objective is an emotion estimation system,
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores colors associated with each of the plurality of emotions;
a display device that displays a coordinate plane formed by a first coordinate axis corresponding to the first biological signal and a second coordinate axis corresponding to the second biological signal;
It is equipped with
The processor displays a marker at a position on the coordinate plane determined by the first data set, and causes the display device to display a color associated with one having a maximum value among the plurality of emotional values. .

One aspect of achieving the above object is an emotion estimation device, comprising:
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores colors associated with each of the plurality of emotions;
an output interface;
It is equipped with
The processor includes:
Displaying a coordinate plane formed by a first coordinate axis corresponding to the first biological signal and a second coordinate axis corresponding to the second biological signal on a display device, and a position on the coordinate plane determined by the first data set. a first display signal for displaying a sign;
a second display signal that causes the display device to display a color associated with the one having the maximum value among the plurality of emotional values;
is output to the output interface.

The two coordinate axes forming the coordinate plane correspond to the subject's heartbeat and brain waves. Heartbeat and brain waves are closely related to two indicators known in psychology: Pleasantness-Unpleasantness and Arousal-Sleep. Therefore, by displaying markers corresponding to the subject's heartbeat and brain waves on a coordinate plane formed with these two indicators as the coordinate axes, and by associating the adjectives used in Russell's circular model, we can The psychological state of the subject can be intuitively grasped based on the analogy. Furthermore, since colors associated with the estimated emotions of the subject are displayed at the same time, visual understanding of the emotions can be further facilitated.

It is difficult to consciously change biological information such as heartbeat and brain waves. However, according to the above-described configuration, it becomes easy to understand the emotion that is not expressed in the subject's appearance through displaying the color associated with the marker on the coordinate plane and the estimated emotion. This can encourage others to understand the subject's emotions, or the subject to understand his or her own emotions.

Therefore, it is possible to provide a new user interface based on the subject's emotions estimated through the acquisition of biometric information.

One aspect of achieving the above objective is an emotion estimation system,
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores colors associated with each of the plurality of emotions;
a display device;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of emotions based on the identified first emotion;
A color associated with the second emotion is displayed on the display device.

One aspect of achieving the above object is an emotion estimation device, comprising:
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores colors associated with each of the plurality of emotions;
an output interface;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of emotions based on the identified first emotion;
A display signal that causes a display device to display a color associated with the second emotion is output to the output interface.

According to the above configuration, the subject's current emotion estimated based on biological information such as heartbeat and brain waves, which is difficult to change consciously, can be guided to the target emotion through color display. In other words, it is possible to provide a novel user interface that enables emotional control that does not depend on the facial expressions or actions of the subject.

One aspect of achieving the above objective is an emotion estimation system,
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores at least one of a sound, a scent, an image, and a language associated with each of the plurality of emotions;
an output device;
It is equipped with
The processor includes:
The output device outputs at least one of a sound, a scent, an image, and a language associated with the one having the maximum value among the plurality of emotional values.

One aspect of achieving the above object is an emotion estimation device, comprising:
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores at least one of a sound, a scent, an image, and a language associated with each of the plurality of emotions;
an output interface;
It is equipped with
The processor includes:
The output interface outputs a control signal that causes the output device to output at least one of a sound, a scent, an image, and a language associated with the one having the maximum value among the plurality of emotional values.

It is difficult to consciously change biological information such as heartbeat and brain waves. However, according to the above configuration, it becomes easier to understand the emotions that are not expressed in the subject's outward appearance through the presentation of at least one of sounds, scents, images, and language that are associated with the estimated emotions. . This can encourage others to understand the subject's emotions, or the subject to understand his or her own emotions.

Therefore, it is possible to provide a new user interface based on the subject's emotions estimated through the acquisition of biometric information.

One aspect of achieving the above objective is an emotion estimation system,
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores at least one of a sound, a scent, an image, and a language associated with each of the plurality of emotions;
an output device;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of emotions based on the identified first emotion;
The output device is caused to output at least one of a sound, a scent, an image, and a language associated with the second emotion.

One aspect of achieving the above object is an emotion estimation device, comprising:
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores at least one of a sound, a scent, an image, and a language associated with each of the plurality of emotions;
an output interface;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of emotions based on the identified first emotion;
The output interface outputs a control signal that causes the output device to output at least one of a sound, a scent, an image, and a language associated with the second emotion.

According to the above configuration, the subject's current emotions estimated based on biological information such as heartbeat and brain waves, which are difficult to change consciously, are communicated to the target through the presentation of at least one of sound, scent, image, and language. Can be guided by emotions. In other words, it is possible to provide a novel user interface that enables emotional control that does not depend on the facial expressions or actions of the subject.

One aspect of achieving the above object is an emotion estimation device, comprising:
an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
an output interface;
It is equipped with
The processor generates a first parameter signal corresponding to a temporal change of each of the plurality of emotional values in a specific period, a second parameter signal corresponding to a first time integral value of the temporal change in the specific period, and At least one of the third parameter signals corresponding to the second time integral value of the first time integral value in a specific period is outputted to the output interface.

According to the above configuration, parameters that can correspond to acceleration, speed, and distance traveled in physics are used to estimate the subject's emotions based on biological information such as heartbeat and brain waves that are difficult to change consciously. can be defined. This enables quantitative analysis of changes in emotions over time, and can facilitate understanding of changes in the subject's emotions.

1 illustrates the configuration of an emotion estimation system according to the first embodiment. An example of processing executed in the emotion estimation system of FIG. 1 is shown. 2 illustrates details of the process in FIG. 2. 2 illustrates details of the process in FIG. 2. Another example of the process executed in the emotion estimation system of FIG. 1 is shown. 5 illustrates details of the process in FIG. 5. The configuration of the emotion estimation system according to the second embodiment is illustrated. An example of processing executed in the emotion estimation system of FIG. 7 is shown. Another example of the process executed in the emotion estimation system of FIG. 7 is shown.

Examples of embodiments will now be described in detail with reference to the accompanying drawings.

FIG. 1 illustrates the configuration of an emotion estimation system 10 according to the first embodiment. The emotion estimation system 10 includes an emotion estimation device 11 and a display device 12.

The emotion estimation device 11 estimates the emotion of the target person based on the pulse and brain waves acquired by the pulse sensor 51 and the brain wave sensor 52 attached to the target person, and causes the display device 12 to perform an operation based on the estimation result. It is a device.

The pulse sensor 51 is configured to detect the subject's pulse and output a first biological signal VS1 corresponding to the detected pulse. For example, the pulse sensor 51 is attached to the subject's fingertips, wrist, earlobe, etc., and detects the pulse based on photoplethysmography. Since the configuration of such a pulse sensor 51 itself is well known, detailed explanation will be omitted. Pulse may be detected based on electrocardiogram, blood pressure, heart sounds, etc.

Specifically, the first biological signal VS1 can be made to correspond to fluctuations in heart rate used to evaluate the state of autonomic nerves. Fluctuations in heart rate can be expressed by an index such as pNNx. For example, pNN50 indicates the percentage of heartbeats in which the RR interval, which is the interval between peaks of adjacent heartbeat waveforms, exceeds 50 milliseconds in 30 consecutive heartbeats. The larger the value of pNNx, the more dominant the sympathetic nerves are, indicating that the physical activity is more active.

The brain wave sensor 52 is configured to detect the subject's brain waves and output a second biological signal VS2 corresponding to the detected brain waves. The brain wave sensor 52 is a non-invasive sensor mounted on the subject's head, and detects the subject's brain waves by, for example, acquiring the potential difference between the earlobe and the forehead.

MindWave Mobile by NeuroSky is known as an example of such a brain wave sensor 52. The device outputs an index value called Attention that indicates the degree of concentration of the subject based on the acquired β waves. The device also outputs an index value called Meditation that indicates the degree of rest of the subject based on the acquired α waves. The second biological signal VS2 corresponds to the difference between the Attention value and the Meditation value. The larger the Attention value, the more awake the person is. The larger the Meditation value, the more sleepy the person is. An appropriate brain wave sensor may be used as long as the second biological signal VS2 having a single value is finally obtained.

The emotion estimation device 11 includes an input interface 111 and a processor 112. The input interface 111 receives the first biological signal VS1 output from the pulse sensor 51 and the second biological signal VS2 output from the brain wave sensor 52. The input interface 111 may include a signal conversion circuit that converts the first biosignal VS1 and the second biosignal VS2 so that they can be processed by the processor 112, which will be described later, as necessary.

Communication between the input interface 111 and the pulse sensor 51 and the brain wave sensor 52 may be performed via a wired connection or a wireless connection.

FIG. 2 shows an example of the flow of processing executed by the processor 112. The processor 112 acquires the first data set D1 based on the first biosignal VS1 and the second biosignal VS2 received by the input interface 111 (STEP 11). The first data set D1 includes the value of the first biological signal VS1 (first value) and the value of the second biological signal VS2 (second value) at a certain point in time. That is, the first data set D1 represents the heartbeat fluctuation and brain wave state of the subject at the relevant time point.

The first data set D1 may be represented as a point on the coordinate plane CP shown in FIG. 3(A). The coordinate plane CP is a two-dimensional coordinate plane formed by a horizontal axis corresponding to the first biological signal VS1 and a vertical axis corresponding to the second biological signal VS2. The horizontal axis is an example of the first coordinate axis. The vertical axis is an example of the second coordinate axis.

The coordinate plane CP can correspond to the coordinate plane used in Russell's toroidal model. Russell's circular model is used for psychometric measurements with high reliability in the field of psychology. The coordinate plane is formed by a horizontal axis representing pleasure/unpleasantness (Valence-Unpleasant) and a vertical axis representing arousal/drowsiness (Arousal-Deactivation).

In Russell's circular model, adjectives expressing various emotions are arranged in a circular pattern on a two-dimensional plane. In this embodiment, eight representative emotions are selected from Russell's circular model, and four emotion coordinate axes shown in FIG. 3B are defined. Each of the four emotion coordinate axes is associated with a pair of emotions arranged on Russell's circular model. Specifically, the four emotional coordinate axes are the emotional coordinate axis corresponding to "surprise - sleepy", the emotional coordinate axis corresponding to "excitement - depression", the emotional coordinate axis corresponding to "joy - sadness", and the emotional coordinate axis corresponding to "relaxation - anger". Contains emotional coordinate axes corresponding to .

In this example, the angle formed between the four emotion coordinate axes takes a constant value (45°). Based on the correspondence between the coordinate plane CP and Russell's circular model described above, the four emotion coordinate axes can be overlapped with the coordinate plane CP so that they share the origin.

Subsequently, the processor 112 converts the first data set D1 into a second data set D2 (STEP 12 in FIG. 2). The second data set D2 includes eight emotion values. Each of the eight emotion values represents a value in any one of the four emotion coordinate axes.

Conversion from the first data set D1 to the second data set D2 is performed as follows. First, as shown in (A) of FIG. 4, the emotional coordinate axis located closest to the point P specified by the first data set D1 at a certain point in time and the emotional coordinate axis located next closest are be identified. The two emotional coordinate axes are the emotional coordinate axis with the longest length from the origin to the intersection point and the next longest emotional coordinate axis among the three emotional coordinate axes that intersect the two perpendicular lines drawn from point P to the horizontal and vertical axes of the coordinate plane CP. chosen as the emotional coordinate axis. In the illustrated example, an emotional coordinate axis including "joy" and an emotional coordinate axis including "excitement" are selected.

Note that if the point P is located on either the horizontal axis or the vertical axis in the coordinate plane CP, only one emotional coordinate axis will be selected.

Subsequently, as shown in FIG. 4B, vectors from the origin to point P are decomposed for the two selected emotional coordinate axes. As a result, emotions associated with one or two emotion coordinate axes have significant values. In the illustrated example, "joy" and "excitement" have significant values. The emotion value associated with each of the remaining emotion coordinate axes is set to zero.

Subsequently, the processor 112 estimates the emotion of the subject (STEP 13 in FIG. 2). Specifically, one of the eight emotions selected from Russell's circular model is specified as the subject's estimated emotion at a certain time t1. More specifically, the emotion associated with the emotion coordinate axis having the largest emotion value is identified through the above-described processing. In the example shown in FIG. 4B, the emotional coordinate axis associated with "joy" has the largest emotional value. Therefore, the processor 112 estimates "joy" as the subject's emotion at a certain time t1.

As shown in FIG. 1, the emotion estimation device 11 includes a storage 113. The storage 113 is a device that can store predetermined data in cooperation with the processor 112. As the storage 113, a semiconductor memory or a hard disk drive device can be exemplified.

As shown in FIG. 3B, in this embodiment, one color is associated with each of the eight emotions described above. The storage 113 stores color data corresponding to eight colors associated with eight emotions.

Subsequently, the processor 112 causes the display device 12 to perform a display operation based on the detection results by the pulse sensor 51 and the brain wave sensor 52 and the estimation results of the subject's emotions (STEP 14 in FIG. 2). The display device 12 is configured to display the coordinate plane CP shown in FIG. 3(A). The display device 12 is configured to display a plurality of colors based on the emotion estimation results.

Specifically, processor 112 generates first display signal DS1. The first display signal DS1 is a signal that causes the display device 12 to display the coordinate plane CP and to display a marker at a position on the coordinate plane CP determined by the first data set D1. In the example shown in FIG. 3A, the marker M1 is displayed at a position determined by the first data set D1 based on the first biosignal VS1 and the second biosignal VS2 at time t1.

As shown in FIG. 1, the emotion estimation device 11 includes an output interface 114. The output interface 114 is communicatively connected to the display device 12. The processor 112 causes the output interface 114 to output the first display signal DS1. The output interface 114 may include a signal conversion circuit that applies conversion to the first display signal DS1 according to the specifications of the display device 12.

Communication between output interface 114 and display device 12 may occur via a wired or wireless connection.

The processor 112 reads color data indicating the color associated with the emotion estimated in STEP 13 from the storage 113, and generates the second display signal DS2. The second display signal DS2 is a signal that causes the display device 12 to display a color associated with the estimated emotion. The processor 112 causes the output interface 114 to output the second display signal DS2. The output interface 114 may include a signal conversion circuit that applies conversion to the second display signal DS2 according to the specifications of the display device 12.

In the example shown in FIG. 3A, the subject's emotion estimated based on the first data set D1 corresponding to the marker M1 is "joy". Therefore, "yellow green" associated with "joy" is displayed on the display device 12. For example, the background color of the coordinate plane CP is changed to yellow-green.

In the example shown in FIG. 3A, a marker M2 is displayed at a position corresponding to the first data set D1 acquired at time t2. The emotion of the subject at time t2 estimated based on this first data set D1 is "anger." Therefore, “red”, which is a color associated with “anger”, is displayed on the display device 12. For example, the background color of the coordinate plane CP is changed to red.

Color display based on the second display signal DS2 can be performed in various ways. The color of the sign may be changed instead of the background color of the coordinate plane CP, or a display area provided at a location different from the coordinate plane CP may be colored.

Subsequently, as shown in FIG. 2, the processing performed by the processor 112 returns to STEP11. That is, each time the first data set D1 corresponding to the detection results of the pulse sensor 51 and the brain wave sensor 52 is acquired, the emotion of the subject is estimated. On the display device 12, a marker is displayed at a position on the coordinate plane CP corresponding to the first data set D1, and a color associated with the estimated emotion is displayed. If the detection result changes, the marker moves on the coordinate plane CP. If the estimated emotion changes, the displayed color changes.

The two coordinate axes forming the coordinate plane CP correspond to the subject's heartbeat and brain waves. Heartbeat and brain waves are closely related to two indicators known in psychology: Pleasantness-Unpleasantness and Arousal-Sleep. Therefore, by displaying markers corresponding to the subject's heartbeat and brain waves on the coordinate plane CP formed with these two indices as the coordinate axes, and further associating the adjectives used in Russell's circular model, the model It is possible to intuitively grasp the psychological state of the subject based on analogies from . Furthermore, since colors associated with the estimated emotions of the subject are displayed at the same time, visual understanding of the emotions can be further facilitated.

It is difficult to consciously change biological information such as heartbeat and brain waves. However, according to the above-described configuration, it becomes easy to grasp the emotions that are not expressed in the subject's appearance through the display of the color associated with the marker on the coordinate plane CP and the estimated emotion. This can encourage others to understand the subject's emotions, or the subject to understand his or her own emotions.

Therefore, it is possible to provide a new user interface based on the subject's emotions estimated through the acquisition of biometric information.

As shown in FIG. 1, processor 112 may generate at least one of a first parameter signal PS1, a second parameter signal PS2, and a third parameter signal PS3. The processor 112 may cause the output interface 114 to output at least one of the generated first parameter signal PS1, second parameter signal PS2, and third parameter signal PS3.

ここでＥ（ｔ_n）は、時点ｔ_nにおけるある感情座標軸の感情値（感情量）を表している。Ｅ（ｔ_n-1）は、時点ｔ_n-1における当該感情座標軸の感情値（感情量）を表している。 The first parameter signal PS1 corresponds to a first parameter PR1 representing a change over time in each emotional value during a specific period. The first parameter PR1(t _n ) at time t _n is expressed by the following equation.

Here, E(t _n ) represents the emotion value (emotion amount) of a certain emotion coordinate axis at time t _n . E(t _n-1 ) represents the emotion value (amount of emotion) of the emotion coordinate axis at time t _n-1 .

The second parameter signal PS2 corresponds to a second parameter PR2 representing a time integral value of the first parameter PR1 during the specific period. The second parameter PR2(t _n ) at time t _n is expressed by the following equation.

The third parameter signal PS3 corresponds to a third parameter PR3 representing a time integral value of the second parameter PR2 during the specific period. The third parameter PR3(t _n ) at time t _n is expressed by the following equation.

By defining parameters that can correspond to acceleration, speed, and travel distance in physics for the emotional values (emotional quantities) obtained for each emotional coordinate axis, quantitative analysis of changes in emotions over time becomes possible. sell. This may facilitate understanding of the subject's emotional changes.

As shown in FIG. 1, processor 112 may generate a synthesis parameter signal SPS. Processor 112 may cause output interface 114 to output the generated synthesis parameter signal SPS.

第二パラメータＰＲ２と第三パラメータＰＲ３についても同様に合成パラメータが定義されうる。プロセッサ１１２は、第一パラメータＰＲ１、第二パラメータＰＲ２、および第三パラメータＰＲ３の少なくとも一つについて合成パラメータを演算し、得られた合成パラメータに対応する合成パラメータ信号ＳＰＳを出力しうる。 The composite parameter signal SPS corresponds to a composite parameter SPR that is a composite value of eight first parameters PR1 obtained for eight emotion coordinate axes. The synthesis parameter SPR(t _n ) at time t _n is expressed by the following equation.

Synthesis parameters can be similarly defined for the second parameter PR2 and the third parameter PR3. The processor 112 can calculate a synthesis parameter for at least one of the first parameter PR1, the second parameter PR2, and the third parameter PR3, and output a synthesis parameter signal SPS corresponding to the obtained synthesis parameter.

By defining a synthesis parameter corresponding to a synthesis value of emotion values (amounts of emotions) obtained for each emotion coordinate axis, quantitative analysis of the totality of emotions may be possible. This may facilitate understanding of the subject's overall emotions.

Furthermore, it is possible to obtain an output corresponding to a more complex estimated emotion. For example, in the case of the example shown in FIG. 3B, the subject's emotion estimated based on the first data set D1 is a composite emotion of "joy" and "excitement." The above synthetic parameter signal SPS can have a value corresponding to this synthetic emotion. On the display device 12, a color representing this composite emotion can be displayed. For example, an intermediate color between yellow-green corresponding to "joy" and yellow corresponding to "excitement" may be displayed.

Note that, instead of outputting the first display signal DS1 and the second display signal DS2, the emotion estimation device 11 outputs at least the first parameter signal PS1, the second parameter signal PS2, the third parameter signal PS3, and the composite parameter signal SPS. It can be configured as a device that outputs one.

FIG. 5 shows another example of processing performed by the processor 112. Processes that are substantially the same as those described with reference to FIG. 2 are given the same reference numerals, and repeated explanations will be omitted.

In this example, the processor 112 estimates the subject's current emotion using a method similar to STEP 13 shown in FIG. 2 (STEP 23). Present emotion is an example of a primary emotion. That is, one of the eight emotions selected from Russell's circular model is specified as the estimated emotion of the subject at the present moment. In the example shown in FIG. 6, "anger" is estimated as the subject's current emotion.

Subsequently, the processor 112 sets a target emotion (STEP 24). The target emotion is an example of a second emotion. The target emotion is also selected as one of the eight emotions selected from Russell's circular model to be used during conversion to the second data set D2. In the example shown in FIG. 6, "relaxation" is set as the target emotion.

The target emotion may be set in advance as a specific emotion regardless of the subject's current emotion. For example, positive emotions such as "joy" and "relaxation" may be selected in advance. Alternatively, an emotion located on the opposite side of the origin from the current emotion in Russell's circular model may be automatically set as the target emotion. For example, if the current emotion is "depressed," "excitement" may be set as the target emotion.

Subsequently, the processor 112 causes the display device 12 to display the color associated with the target emotion (STEP 25). Specifically, color data indicating a color associated with the target emotion set in STEP 23 is read from the storage 113, and a second display signal DS2 for displaying the color on the display device 12 is generated. The processor 112 causes the output interface 114 to output the second display signal DS2.

In the example shown in FIG. 6, "green" associated with the target emotion "relax" is displayed on the display device 12. A subject who is currently feeling "angry" can be prompted to shift to a "relaxed" state by seeing "green" displayed on the display device 12. If the subject recognizes the correspondence between colors and emotions in advance, the transition of emotions can be facilitated more smoothly.

Note that in this example, displaying the index on the coordinate plane CP based on the first display signal DS1 is not essential.

According to the above configuration, the subject's current emotion estimated based on biological information such as heartbeat and brain waves, which is difficult to change consciously, can be guided to the target emotion through color display. In other words, it is possible to provide a novel user interface that enables emotional control that does not depend on the facial expressions or actions of the subject.

As shown in FIG. 5, the processor 112 can cause the display device 12 to display the color associated with the subject's current emotion prior to displaying the color associated with the target emotion (STEP 26).

Specifically, the processor 112 reads color data indicating the color associated with the emotion estimated in STEP 23 from the storage 113, and generates the second display signal DS2 that causes the display device 12 to display the color. The processor 112 causes the output interface 114 to output the second display signal DS2.

In the example shown in FIG. 6, "red" associated with the current emotion "anger" is first displayed on the display device 12. Subsequently, "green" associated with the target emotion "relax" is displayed on the display device 12.

According to such a configuration, the subject's current emotion, which is the starting point for emotional control, can be visually grasped. Since it is difficult to consciously change biological information such as heartbeat and brain waves, it is possible to encourage others or the subject themselves to understand emotions, which are the starting point for emotional control, regardless of the subject's facial expressions or actions.

In addition to or in place of STEP 26 shown in FIG. 5, the processor 112, prior to displaying the color associated with the target emotion, displays the color associated with the subject's current emotion and the color associated with the target emotion. can be displayed on the display device 12 (STEP 27).

Specifically, the processor 112 reads color data indicating at least one color associated with at least one emotion located between the current emotion and the target emotion in Russell's circular model from the storage 113, and A second display signal DS2 that causes the display device 12 to display one color is generated. The processor 112 causes the output interface 114 to output the second display signal DS2. The at least one color is an example of an intermediate color.

In the example shown in FIG. 6, "orange" is associated with "surprise," "excitement," and "joy," which are located between the current emotion "anger" and the target emotion "relaxation." , "yellow", and "yellow-green" are first displayed on the display device 12. Subsequently, "green" associated with the target emotion "relax" is displayed on the display device 12. When a plurality of intermediate colors are displayed, it is preferable that the display be performed based on the order of the emotions arranged on the path from the current emotion to the target emotion in Russell's circular model. In the illustrated example, the display may be performed in the order of "orange", "yellow", and "yellow-green".

According to such a configuration, it is possible to smoothly transition from the current emotion to the target emotion. In particular, the greater the difference between the current emotion and the target emotion, the more abrupt the presentation of the color associated with the target emotion can be suppressed, and the smoother the emotion transition can be promoted.

The functions of processor 112 described above may be implemented by a general purpose microprocessor operating in conjunction with general purpose memory. Examples of general-purpose microprocessors include CPUs, MPUs, and GPUs. Examples of general-purpose memory include ROM and RAM. In this case, the ROM may store a computer program that executes the above-described processing. A ROM is an example of a storage medium that stores a computer program. The processor 112 specifies at least a part of the program stored on the ROM, loads it on the RAM, and executes the above-described processing in cooperation with the RAM. Storage 113 may be used as the general purpose memory described above. Processor 112 may be implemented by a dedicated integrated circuit such as a microcontroller, ASIC, or FPGA that can execute a computer program that implements the processes described above. Processor 112 may be implemented by a combination of a general purpose microprocessor and a special purpose integrated circuit.

The emotion estimation system 1 can be realized in various ways. For example, the emotion estimation device 11 and the display device 12 may be built into devices that operate independently, or the results of processing by the emotion estimation device 11 located remotely may be displayed on the display device 12 located near the user. may be done.

FIG. 7 illustrates the configuration of an emotion estimation system 20 according to the second embodiment. The emotion estimation system 20 includes an emotion estimation device 21 and an output device 22. Components that are substantially the same as those described with reference to the emotion estimation system 10 according to the first embodiment are given the same reference numerals, and repeated explanations will be omitted.

The emotion estimation device 21 includes a processor 212 and a storage 213. The storage 213 is a device that can store predetermined data in cooperation with the processor 212. As the storage 213, a semiconductor memory or a hard disk drive device can be exemplified. The storage 213 stores output data corresponding to at least one of sounds, scents, images, and languages associated with each of the eight emotions shown in FIG. 3B.

FIG. 8 shows an example of the flow of processing executed by the processor 212. Processes that are substantially the same as those described with reference to FIG. 2 are given the same reference numerals, and repeated explanations will be omitted.

The processor 212 causes the output device 22 to perform an output operation based on the detection results by the pulse sensor 51 and the brain wave sensor 52 and the estimation results of the subject's emotions (STEP 34).

Specifically, the processor 212 reads output data representing at least one of the sound, scent, image, and language associated with the emotion estimated in STEP 13 from the storage 213, and generates the control signal CS. The control signal CS is a signal that causes the output device 22 to output sounds, scents, images, and language associated with the estimated emotion.

As shown in FIG. 7, the emotion estimation device 21 includes an output interface 214. The output interface 214 is communicatively connected to the output device 22. Communication between output interface 214 and output device 22 may occur via a wired or wireless connection. Processor 212 causes output interface 214 to output control signal CS. The output interface 214 may include a signal conversion circuit that applies conversion to the control signal CS according to the specifications of the output device 22.

For example, if the emotion estimated in STEP 13 is "joy" shown in FIG. Output.

It is difficult to consciously change biological information such as heartbeat and brain waves. However, according to the above configuration, it becomes easier to understand the emotions that are not expressed in the subject's outward appearance through the presentation of at least one of sounds, scents, images, and language that are associated with the estimated emotions. . This can encourage others to understand the subject's emotions, or the subject to understand his or her own emotions.

Therefore, it is possible to provide a new user interface based on the subject's emotions estimated through the acquisition of biometric information.

FIG. 9 shows another example of the processing executed by the processor 212. Processes that are substantially the same as those described with reference to FIG. 8 are given the same reference numerals, and repeated explanations will be omitted.

In this example, the processor 212 estimates the subject's current emotion using a method similar to STEP 13 shown in FIG. 8 (STEP 43). Present emotion is an example of a primary emotion. That is, one of the eight emotions selected from Russell's circular model is specified as the estimated emotion of the subject at the present moment. In the example shown in FIG. 6, "anger" is estimated as the subject's current emotion.

Subsequently, the processor 212 sets a target emotion (STEP 44). The target emotion is an example of a second emotion. The target emotion is also selected as one of the eight emotions selected from Russell's circular model to be used during conversion to the second data set D2. In the example shown in FIG. 6, "relaxation" is set as the target emotion.

The target emotion may be set in advance as a specific emotion regardless of the subject's current emotion. For example, positive emotions such as "joy" and "relaxation" may be selected in advance. Alternatively, an emotion located on the opposite side of the origin from the current emotion in Russell's circular model may be automatically set as the target emotion. For example, if the current emotion is "depressed," "excitement" may be set as the target emotion.

Subsequently, the processor 212 causes the output device 22 to output at least one of the sound, scent, image, and language associated with the target emotion (STEP 45). Specifically, output data indicating at least one of the sound, scent, image, and language associated with the target emotion set in STEP 23 is read from the storage 213, and the output data indicating at least one of the sound, scent, image, and language is read out from the storage 213. A control signal CS is generated to cause the output device 22 to output. Processor 212 causes output interface 214 to output control signal CS.

In the example shown in FIG. 6, at least one of a sound, a scent, an image, and a language associated with the target emotion "relax" is output from the output device 22. A subject who is currently feeling "angry" can be prompted to transition to a "relaxed" state by receiving at least one of sound, scent, image, and language from the output device 22. If the subject recognizes in advance the association between at least one of sounds, scents, images, and language and emotions, the transition of emotions can be facilitated more smoothly.

According to the above configuration, the subject's current emotion estimated based on biological information such as heartbeat and brain waves, which are difficult to change consciously, is changed to the target emotion through the presentation of at least one of sound, scent, image, and language. can be led to In other words, it is possible to provide a novel user interface that enables emotional control that does not depend on the facial expressions or actions of the subject.

As shown in FIG. 9, prior to outputting at least one of the sounds, scents, images, and language associated with the target emotion, the processor 212 outputs the sounds, scents, images, and language associated with the subject's current emotion. At least one of the languages can be outputted to the output device 22 (STEP 46).

Specifically, the processor 212 reads output data indicating at least one of the sound, scent, image, and language associated with the emotion estimated in STEP 43 from the storage 213, and A control signal CS that causes at least one to be output to the output device 22 is generated. Processor 212 causes output interface 214 to output control signal CS.

In the example shown in FIG. 6, at least one of a sound, a scent, an image, and a language associated with the current emotion "anger" is first output from the output device 22. Subsequently, at least one of a sound, a scent, an image, and a language associated with the target emotion "relaxation" is output from the output device 22.

According to such a configuration, the subject's current emotion, which is the starting point for emotional control, can be intuitively grasped. Since it is difficult to consciously change biological information such as heartbeat and brain waves, it is possible to encourage others or the subject themselves to understand emotions, which are the starting point for emotional control, regardless of the subject's facial expressions or actions.

In addition to or in place of STEP 46 shown in FIG. 9, the processor 112, prior to outputting at least one of the sounds, scents, images, and language associated with the target emotion, outputs a sound associated with the subject's current emotion. , fragrance, image, and language can be outputted by the output device 22 (STEP 47).

Specifically, the processor 212 outputs output data indicating at least one of a sound, a scent, an image, and a language associated with at least one emotion located between the current emotion and the target emotion in Russell's circular model. A control signal CS is generated that reads out from the storage 213 and causes the output device 22 to display at least one of the sound, scent, image, and language. Processor 212 causes output interface 214 to output a control signal.

In the example shown in FIG. 6, the target emotion is associated with at least one of "surprise," "excitement," and "joy," which are located between the current emotion "anger" and the target emotion "relaxation." At least one of sounds, scents, images, and language is first output from the output device 22. Subsequently, at least one of a sound, a scent, an image, and a language associated with the target emotion "relaxation" is output from the output device 22. When at least one of sounds, scents, images, and language corresponding to multiple types of intermediate emotions is displayed, based on the order of the intermediate emotions arranged on the path from the current emotion to the target emotion in Russell's circular model, Preferably, at least one of sound, scent, image, and language is output. In the illustrated example, outputs of at least one of corresponding sounds, scents, images, and language may be made in the order of "surprise," "excitement," and "delight."

According to such a configuration, it is possible to smoothly transition from the current emotion to the target emotion. In particular, the larger the difference between the current emotion and the target emotion, the more the sudden feeling associated with the presentation of at least one of the sound, scent, image, and language associated with the target emotion can be suppressed, and a smooth emotional transition can be promoted.

The functions of processor 212 described above may be implemented by a general purpose microprocessor operating in conjunction with general purpose memory. Examples of general-purpose microprocessors include CPUs, MPUs, and GPUs. Examples of general-purpose memory include ROM and RAM. In this case, the ROM may store a computer program that executes the above-described processing. A ROM is an example of a storage medium that stores a computer program. The processor 212 specifies at least a portion of the program stored on the ROM, loads it on the RAM, and executes the above-described processing in cooperation with the RAM. Storage 213 may be used as the general purpose memory described above. Processor 212 may be implemented by a dedicated integrated circuit such as a microcontroller, ASIC, or FPGA that can execute a computer program that implements the processes described above. Processor 212 may be implemented by a combination of a general purpose microprocessor and a special purpose integrated circuit.

The emotion estimation system 2 can be realized in various ways. For example, the emotion estimation device 21 and the output device 22 may be built into devices that operate independently, or the results of processing by the emotion estimation device 21 located remotely may be output from the output device 22 located near the user. may be done.

Each of the above embodiments is merely an example to facilitate understanding of the present invention. The configurations according to each of the embodiments described above may be modified and improved as appropriate without departing from the spirit of the present invention.

In each of the embodiments described above, eight emotions arranged on Russell's circular model are selected as the plurality of emotions associated with the plurality of emotion values constituting the second data set D2. Further, the angular intervals between the four emotion coordinate axes corresponding to the eight emotions are constant. However, the number and types of emotions selected from Russell's circular model can be arbitrarily determined. For example, three emotion coordinate axes may be defined corresponding to "joy," "anger," and "sadness." Therefore, depending on the number and type of the plurality of emotions selected, the angular intervals between the plurality of emotion coordinate axes may not be constant. Moreover, emotions other than the eight above may be selected.

10, 20: Emotion estimation system, 11, 21: Emotion estimation device, 111: Input interface, 112, 212: Processor, 113, 213: Storage, 114, 214: Output interface, 12: Display device, 22: Output device, VS1: first biological signal, VS2: second biological signal, D1: first data set, D2: second data set, CP: coordinate plane, DS1: first display signal, DS2: second display signal, PS1: second One parameter signal, PS2: second parameter signal, PS3: third parameter signal, SPS: composite parameter signal, CS: control signal

Claims (6)

an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores colors associated with each of the plurality of emotions;
a display device;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of types of emotions based on the identified first emotion;
displaying a color associated with the second emotion on the display device;
Emotion estimation system. The processor causes the display device to display the color associated with the first emotion prior to displaying the color associated with the second emotion.
The emotion estimation system according to claim 1. The processor causes the display device to display an intermediate color between the color associated with the first emotion and the color associated with the second emotion, prior to displaying the color associated with the second emotion.
The emotion estimation system according to claim 1 or 2. an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores at least one of sounds, scents, images, and languages associated with each of the plurality of types of emotions;
an output device;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of types of emotions based on the identified first emotion;
causing the output device to output at least one of a sound, a scent, an image, and a language associated with the second emotion;
Emotion estimation system. an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores colors associated with each of the plurality of emotions;
an output interface;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of types of emotions based on the identified first emotion;
causing the output interface to output a display signal that causes a display device to display a color associated with the second emotion;
Emotion estimation device. an input interface that receives a first biosignal corresponding to the subject's pulse and a second biosignal corresponding to the subject's brain waves;
A first data set including a first value corresponding to the first biosignal and a second value corresponding to the second biosignal is converted into quantities of multiple types of emotions arranged on Russell's circular model. a processor for converting into a second data set including a plurality of corresponding emotion values;
a storage that stores at least one of sounds, scents, images, and languages associated with each of the plurality of types of emotions;
an output interface;
It is equipped with
The processor includes:
identifying a first emotion included in the plurality of emotions corresponding to one having the maximum value among the plurality of emotion values;
setting a second emotion different from the first emotion included in the plurality of types of emotions based on the identified first emotion;
causing the output interface to output a control signal that causes the output device to output at least one of a sound, a scent, an image, and a language associated with the second emotion;
Emotion estimation device.

Images