JP2023178077A - Emotion estimation device, emotion estimation method, and emotion estimation system - Google Patents

Abstract

To easily improve emotion estimation accuracy based on a biological signal.SOLUTION: An estimation device (a server) according to an embodiment estimates an emotion of a user based on variation with the lapse of time of a plurality of indexes based on a biological signal of the user.SELECTED DRAWING: Figure 2

Description

The present invention relates to an emotion estimation device, an emotion estimation method, and an emotion estimation system.

2. Description of the Related Art There is a known technique for estimating a subject's emotion by applying information obtained from a subject's heart waveform (electrocardiographic waveform) to a Russell circular model (for example, see Patent Document 1).

JP2019-63324A

However, the conventional technology has a problem in that it may not be possible to easily improve the accuracy of emotion estimation based on biological signals.

Biometric information is greatly affected by individual and environmental differences (physical condition, emotions, etc. at the time). Therefore, when estimating emotions by applying information obtained from biological signals to an emotion estimation model such as the Russell ring model, it is necessary to correct (calibrate) the emotion estimation model according to the subject in order to improve estimation accuracy. )Is required.

On the other hand, calibrating an emotion estimation model takes time and places a burden on the subject. Since calibration is performed using measured values in that state, there are problems such as placing a physical burden on the subject) and being influenced by the external environment.

The present invention has been made in view of the above, and an object of the present invention is to suppress the influence of individual differences and environmental differences, and to appropriately estimate emotions based on biological signals.

In order to solve the above-mentioned problems and achieve the purpose, an estimation device according to the present invention is an emotion estimation device for estimating emotions, and includes a controller, and the controller has a plurality of indicators based on a user's biological signals. The user's emotions are estimated based on changes over time.

According to the present invention, it is possible to suppress the influence of individual differences and environmental differences, and to appropriately estimate emotions based on biological signals.

FIG. 1 is a diagram illustrating a configuration example of an estimation system according to an embodiment. FIG. 2 is a diagram illustrating a configuration example of a server according to the embodiment. FIG. 3 is a diagram showing an example of an emotion estimation model (psychological plane). FIG. 4 is a diagram illustrating an example of an emotion estimation model using a physical condition index as a parameter. FIG. 5 is a diagram showing an example of a sensor table and a psychological plane table. FIG. 6 is a diagram showing an example of an emotion change table. FIG. 7 is a diagram illustrating a method for extracting emotion types. FIG. 8 is a diagram showing an example of the evaluation screen and the usage status of information regarding emotions. FIG. 9 is a diagram showing an example of the evaluation screen and the usage status of information regarding emotions. FIG. 10 is a diagram illustrating a method for calibrating an emotion estimation model. FIG. 11 is a diagram illustrating a learning method of a machine learning model. FIG. 12 is a flowchart showing the model generation process. FIG. 13 is a flowchart showing an emotion estimation process for estimating an emotional state.

Hereinafter, embodiments of an emotion estimation device, an emotion estimation method, and an emotion estimation system will be described in detail with reference to the accompanying drawings. Note that the present invention is not limited to the embodiments described below.

First, an estimation system according to an embodiment will be described using FIG. 1. FIG. 1 is a diagram illustrating a configuration example of an estimation system according to an embodiment.

As shown in FIG. 1, the estimation system 1 includes a server 10, a terminal device 20, a sensor 31a, a sensor 32a, a sensor 31b, and a sensor 32b. The estimation system 1 estimates the emotions of the test subject U02a and the test subject U02b. The server 10 is an example of an emotion estimation device.

Subject U02a and subject U02b are e-sports players, for example. Estimation system 1 estimates the emotions of test subject U02a and test subject U02b who are playing a video game. In the description of this embodiment, in order to make the description concrete and easy to understand, the above-described application scene in e-sports will be assumed as an application example, and state transitions will also be explained.

The emotion estimation results are used, for example, for mental training of test subject U02a and test subject U02b in e-sports. For example, in a scene where the subject U02 feels disadvantageous to the competition (anxiety, anger, etc.) while playing a video game, it is determined that intensive training corresponding to the emotional state is required. In the training, the emotional information of the test subject U02a and the test subject U02b estimated by the estimation system 1 is used.

In addition, in various e-sports, the subjects U02a and U02b may play cooperatively or competitively, but in such e-sports, displaying the emotional state of each player, etc. A game that has increased the element of enjoyment by allowing advanced gameplay, such as changing game tactics depending on the emotional state, and allowing game spectators to watch the game while also understanding the emotional state of each player. This has the effect of making it possible to watch the game.

In addition, as another application example, the subject may be a patient at a medical institution. In this case, the emotions estimated by the estimation system 1 are used for examinations, treatments, and the like.

For example, if a test subject who is a patient feels anxious, staff at a medical institution can provide countermeasures such as counseling.

Furthermore, the subject may be a student at an educational institution. In this case, the emotions estimated by the estimation system 1 are used to improve the lesson content.

For example, if a subject who is a student finds the lesson boring, the teacher can improve the content of the lesson to make it more interesting for the student.

Furthermore, the subject may be a vehicle driver. In this case, the emotions estimated by the estimation system 1 are used to promote safe driving.

For example, if a test subject who is a driver does not feel moderate tension while driving, the in-vehicle device can output a message urging the test subject to concentrate on driving.

Furthermore, the subject may be a viewer of content such as video and music. In this case, the emotions estimated by the estimation system 1 are used to create further content.

For example, a video content distributor can create a highlight video by collecting scenes that viewers (test subjects) found enjoyable. In this way, the estimated emotions can be used for various purposes.

Return to system description. The server 10 and the terminal device 20 are connected via a network N. For example, network N is the Internet or an intranet.

For example, the terminal device 20 includes a personal computer, a smartphone, a tablet computer, and the like. The terminal device 20 is used by the analyst U01.

Sensor 31a, sensor 32a, sensor 31b, and sensor 32b are attached to test subject U02a and test subject U02b, detect biological signals of test subject U02a and test subject U02b as sensor signals, and transmit the detected sensor signals to terminal device 20.

Specifically, the sensor 31a and the sensor 31b are, for example, headgear type brain wave sensors. Further, the sensor 32a and the sensor 32b are, for example, wristband type pulse sensors.

Moreover, the sensor 31a, the sensor 32a, the sensor 31b, and the sensor 32b have a communication interface, and are communicatively connected to the terminal device 20 according to communication standards such as Wi-Fi (registered trademark) and Bluetooth (registered trademark), and the terminal device The sensor signal is transmitted to 20.

The processing flow of the estimation system 1 will be explained using FIG. 1.

The server 10 has registered models (calculation formulas and conversion data tables) that calculate biological state index values for emotion estimation based on the subject's biological signals, and also calculates the subject's emotions from a plurality of indicators based on the subject's biological signals. It is assumed that the emotion estimation model to be estimated has been registered. Note that the emotion estimation model is created based on medical evidence (papers, etc.). Note that the emotion estimation model will be described later.

The sensor 31a, the sensor 32a, the sensor 31b, and the sensor 32b measure the biological conditions (signals) of the subject U02a and the subject U02b, and output the measured sensor signals to the terminal device 20 (step S1). Then, the terminal device 20 transmits the input sensor signal to the server 10 (step S2).

The server 10 generates a biological condition index value, which is a value of an index indicating a biological condition, based on the input sensor signal biological signal. For example, the server 10 generates two biological condition index values related to brain waves or heartbeats, which are sensor signals. For example, the index based on the sensor signal of the heartbeat sensor is "average heartbeat interval" and "heartbeat LF component", and the server 10 stores the index value (for example, numeric value) is calculated based on the sensor signal of the heart rate sensor.

Note that the sensor 31a, the sensor 32a, the sensor 31b, and the sensor 32b are provided with a calculation function (computer, etc. are built-in), and the sensor 31a, the sensor 32a, the sensor 31b, and the sensor 32b are configured to calculate the biological condition index value. It is also possible to do so. It is also possible to configure the terminal device 20 to calculate the biological condition index value based on the input biological signals from the sensor 31a, the sensor 32a, the sensor 31b, and the sensor 32b.

Then, the server 10 estimates the emotional states of the test subject U02a and the test subject U02b based on the calculated biological state index value using a built-in emotion estimation model or the like (step S3). Note that in this embodiment, the server 10 estimates emotional state information such as emotional change state based on changes in biological state index values. The detailed content and estimation method of the emotional state information such as the emotional change state will be described later.

Then, the server 10 provides the estimation result of the emotional state information to the terminal device 20 (step S4). The terminal device 20 performs operations such as displaying the received emotional state information based on the operation of the analyst U01, providing it to an external information user based on the operator's operation, and providing it to an external game system. State information will be used in a variety of ways.

Note that the server 10, the terminal device 20, and the user side device (for example, a game device) have a controller that performs the above-mentioned functions and operations.

FIG. 2 is a diagram illustrating a configuration example of a server according to the embodiment. The server 10 is an example of a computer that performs estimation of emotional state information. Further, the server 10 is an example of an estimation device.

As shown in FIG. 2, the server 10 includes a communication section 11, a storage section 12, and a control section 13.

The communication unit 11 is an interface for communicating data with other devices via the network N. The communication unit 11 is, for example, a NIC (Network Interface Card).

The storage unit 12 is realized by, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory, or a storage device such as a hard disk or an optical disk. The storage unit 12 includes a sensor table 121 and a psychological plane table 122 (it is expressed as a psychological plane table for easy understanding because it is a map table with two types of indicators, but if there are three or more types, it is a three-dimensional (map)). A psychological judgment table 123 is provided in the storage area. The storage unit 12 is an example of a storage device.

The control unit 13 is a controller, and various programs (not shown) stored in the storage unit 12 are executed by, for example, a CPU (Central Processing Unit) or an MPU (Micro Processing Unit), using the RAM as a work area. This is achieved by Further, the control unit 13 can be realized by, for example, an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

The control unit 13 includes an extraction unit 131, a psychological analysis unit 132, a generation unit 133, a specification unit 134, a provision unit 135, an acquisition unit 136, and a model selection unit 137, and has information processing functions described below. accomplish or carry out an action.

Note that the server 10 may acquire the above-mentioned programs and various information via another computer or a portable recording medium connected via a wired or wireless network.

Next, the data table (database) for various processes stored in the storage unit 12 will be explained, but for ease of understanding, the psychological estimation model (psychological estimation principle) will be briefly explained first.

FIG. 3 is a diagram showing an example of an emotion estimation model (psychological plane). According to various medical evidence related to psychology, there is a technical philosophy that states that psychology can be estimated based on two types of indicators that indicate physical conditions. The psychological plane centered on two types of mental and physical state indicators shown in Figure 3 is an example of such a model, with the vertical axis representing the "activation of thinking and judgment" and the horizontal axis representing the "activation of the body." There is. Then, the corresponding psychological state is assigned to each quadrant region separated from the vertical axis and the horizontal axis based on the origin of the psychological plane (for example, the midpoint between "activity level of thinking judgment" and "activity level of body"). ing. For example, in the example of FIG. 3, the psychological states of "fun, joy, anger, sadness" are assigned to the first quadrant. The distance from the origin indicates the strength of the corresponding psychological state.

In other words, the psychological state and its strength are estimated based on which quadrant on the psychological plane the coordinates of the measured mental and physical state indicators (two types) of the subject exist in, and the distance from the origin. It turns out.

Further, it is known from academic papers and the like that physical states such as "activity level of thinking and judgment" and "activity level of body" can be estimated from the subject's biological signals (outputs of various sensors attached to the subject). From these, we estimated two types of indicators indicating the subject's physical condition based on the subject's biological signals, and then used the estimated two types of indicators to create an emotion estimation model (using these two types of indicators) as shown in Figure 3. model), it is possible to estimate the subject's psychology.

The relationship between biological signals and indicators indicating physical conditions, and the relationship between indicators indicating physical conditions and psychological conditions are, for example, biological signals (measurement of various sensor signals), physical conditions (measurement of various sensor signals), and psychological conditions. The state will be identified and estimated through measurement experiments, etc. (measurement of various sensor signals, questionnaires, observation and analysis of subjects' facial expressions, etc.).

Furthermore, in this embodiment, the mental and physical state is estimated based on the coordinate change state (coordinate trajectory) rather than the coordinate position of the physical state index (two types) on the psychological plane. Next, the relationship between the coordinate change state and the psychological state will be explained.

FIG. 4 is a diagram illustrating an example of an emotion estimation model using a physical condition index as a parameter. Similar to the emotion estimation model shown in Figure 3, this is a model expressed on a psychological plane centered on two types of physical state indicators, with the vertical axis representing the "activity of thinking and judgment" and the horizontal axis representing the "activity of the body." ”. Note that models of other types of physical states with vertical and horizontal axes can be created using the same idea.

The model shown in FIG. 4 shows changes in emotions in each direction of change in the active state of the body and in each direction of change in the active state of thought and judgment. Note that this model is based on changes in the coordinates of indicators of physical condition, and basically the coordinate positions of indicators of physical condition have no meaning (this model cannot be used in combination with a model based on the coordinate positions of indicators of physical condition). etc. are possible). For example, if the active state of the body is "body sedation/promotes digestion" and the active state of thought/judgment is "active thinking/judgment, high arousal", the change state of emotion is "confusion/depression direction/physical system: low". Fatigue, brain system: high level of thought confusion.” The model shown in FIG. 4, for example, calculates how the coordinate positions according to two types of physical states move with respect to the psychological state area placed in each quadrant in the emotion estimation model shown in FIG. It is generated based on which psychological state area (which psychological state is moving in the direction of becoming stronger or which psychological state is moving in the direction of becoming weaker).

By conducting a stratified analysis of the corresponding changes in emotions in each direction of movement between the active state of the body and the active state of thinking and judgment in each direction of the vertical and horizontal axes in FIG. It is possible to generate a psychological judgment table 123 shown in FIG.

FIG. 5 is a diagram showing an example of the sensor table 121 (table (A) in FIG. 5) and the psychological plane table 122 (table (B) in FIG. 5). As shown in the table in FIG. 3A, the items (data storage cells (memory frames)) of the sensor table 121 include "sensor ID", "sensor type", "biological signal type", "corresponding index ID", It includes "corresponding index type" and "index conversion information".

The item “sensor ID” in the sensor table 121 stores sensor ID data that is identification information for identifying data records in the sensor table 121. The sensor ID data is also the primary key of the data record in the sensor table 121. That is, in the sensor table 121, a data record is configured for each sensor ID data, and data of each item linked to the sensor ID data is stored in the data record.

The item "corresponding index type" of the sensor table 121 stores index type data representing the type of biological condition index.

The item "sensor type" in the sensor table 121 stores information for specifying the sensor type, here the sensor name (data such as model number may also be used).

The item "corresponding index ID" of the sensor table 121 stores identification information for identifying an emotion estimation index generated (calculated) based on the biological signal detected by the sensor. The item "corresponding index type" of the sensor table 121 stores the type (name, etc.) of the index.

The item “biological signal type” in the sensor table 121 stores the type of measurement value based on the biological signal detected by the sensor. This biosignal type data is data that is correlated with the corresponding index type data (it is academically recognized that the corresponding index type data can be estimated (calculated) using the corresponding biosignal type data).

The item "index conversion information" in the sensor table 121 stores conversion information (arithmetic expressions, conversion data tables, etc.) for calculating index values based on biological signals detected by the sensor. In other words, by converting the biological signal detected by the sensor corresponding to the sensor ID data according to the index conversion information data, the index data (value) of the emotion estimation index identified by the corresponding index ID is estimated (calculated). will be done.

For example, the data record of sensor IDSN01 in the sensor table 121 shown in FIG. The information states that the index of "activation of thinking recognition judgment" can be calculated using the index conversion information of "FX01".

The psychological plane table 122 is a two-dimensional matrix table with index types (specifically, using index ID data) as parameters on the vertical and horizontal axes, and is a two-dimensional matrix table that uses index types (specifically, index ID data is used) as parameters on the vertical and horizontal axes. Psychological plane type data that can be used with two types of index type data is stored. For example, if the index types used as indicators are index type VSm and index type VSn, the psychological plane used for psychological estimation is psychological plane mn, information for processing using psychological plane mn is read out, and psychological plane mn is used for psychological estimation. It will be used for estimation processing.

Note that common data is used for the index ID of the sensor table 121 and the index ID data of the psychological plane table 122. That is, based on the sensor table 121 and the psychological plane table 122, the psychological plane mn used for psychological estimation can be determined using two types of sensors attached to the subject. For example, if the sensor types worn by the subject are "brain wave sensor BA (index IDVS01)" and "heartbeat sensor HA (index IDVS02)," then "brainwave sensor BA (index IDVS01)" and "heartbeat sensor HA (index IDVS02)" are used. )" is determined based on the sensor table 121 (for example, VS01: an index ID value corresponding to "activation of thinking recognition judgment", VS02: index ID value corresponding to "physical activity"). Then, based on the psychological plane table 122, psychological planes 01-02, whose indicators are "activation of thinking recognition and judgment" (VS01) and "physical activity" (VS02), are determined as psychological planes to be used for psychological estimation. .

Figure 6 shows the index values of two types of indicators (“activity level of thinking judgment” and “level of sympathetic nerve activity”) on the vertical axis (“activity level of thinking judgment”) and horizontal axis (“level of sympathetic nerve activity”). )'') is a diagram showing an example of an emotion change table showing emotional states based on coordinate changes of index values in a two-dimensional emotion plane in which emotions are arranged on a plane. That is, it is a diagram showing an example of a data table for forming an emotion estimation model using the physical condition index shown in FIG. 4 as a parameter.

As shown in FIG. 6, the items of the psychological judgment table 123 (memory cells (memory frames) in the data table) include "emotional change ID", "change pattern", "display symbol", "meaning of change direction", "Emotion type" is included.

The item “emotional change ID” of the psychological judgment table 123 stores emotional change ID data that is identification information for identifying the data record of the psychological judgment table 123. Emotion change ID data Emotion change is also the primary key in the data record of the psychological judgment table 123. That is, in the psychological judgment table 123, a data record is configured for each emotion change ID data, and the data emotion change ID data of each item linked to the emotion change ID data is stored in the data record.

The item "change pattern" of the psychological judgment table 123 stores change direction information indicating a change pattern (shape type of coordinate locus) of the coordinate positions of two types of index data on the two-dimensional emotion plane. Here, the direction of change is stratified into 8 directions step by step (the angular region is divided into 45 degree directions, and the direction of change is stratified according to which region it belongs to), and among the 8 stratified directions, Data in the direction to which the target data corresponds is stored. In addition to the direction of change, the change pattern of the coordinate position includes the shape of the coordinate locus (combination of curves, straight lines, etc.; in this case, the change pattern of the measurement value to be judged is judged based on the degree of similarity) and the change speed. etc., and combinations thereof can be applied.

The item "display symbol" of the psychological judgment table 123 stores a symbol (image) indicating the direction of change in the coordinate position of two types of index data on the two-dimensional emotional plane. This symbol (image) is used when displaying emotional changes on a display or the like. Note that the symbol (image) indicating the direction of change is also a symbol (image) stratified into eight directions, similar to the above-mentioned change direction information, and is stored in the corresponding data record. Moreover, the data content of this "display symbol" may be appropriately set according to the data content of the "change pattern". For example, if the data content of the "change pattern" is the shape of a coordinate locus, the data of the "display symbol" may be an image showing the shape of the coordinate locus.

The item "meaning of direction of change" in the psychological judgment table 123 stores information (text data) indicating the state of change in the mental and physical state with respect to the direction of change in the coordinate positions of the two types of index data on the two-dimensional emotional plane.

The item "emotion type" of the psychological judgment table 123 is the emotion type corresponding to the data record, that is, the change pattern of the coordinate positions of the two types of index data on the two-dimensional emotion plane stored in the "change pattern" in the data record. The emotion type corresponding to (the type of emotion whose intensity increases) is memorized.

For example, information on a data record where the value of the item "Change ID" is "D02" is used for display as the direction of change in the coordinate positions of the two types of index data on the two-dimensional emotion plane is "coordinate change toward the upper right direction." The symbol (image) is "an arrow pointing to the upper right", the physical state in response to the change in the coordinate position is "body and brain activated", and the corresponding emotion type (type of emotion whose intensity increases) is "fun, joy". , anger, sadness.” In other words, if the direction of change in the coordinate positions of the two types of index data on the two-dimensional emotional plane is "coordinate change toward the upper right," the physical state is presumed to be "body and brain activated," and it is assumed that the body/brain is activated. Emotions (emotions that are becoming stronger) are estimated to be "fun, joy, anger, and sadness." Using this information, the display displays, for example, an ``arrow pointing to the upper right'' as information on changes in physical condition, and changes in emotions that are occurring (emotions that are becoming stronger). As the information display, text information such as "fun, joy, anger, sadness" will be displayed.

Note that the psychological judgment table 123 is created by a designer/developer based on academic information in psychology as shown in FIG. It is also possible to generate it by performing analysis processing or the like. In this embodiment, the server 10 (control unit 13) generates the psychological judgment table 123 through language analysis processing or the like.

In the example of FIG. 4, in a coordinate plane (e.g., the emotion estimation model described later) with the horizontal axis representing physical activity/inactivity (sedation) and the vertical axis representing brain activity (alertness)/inactivity (low arousal), Estimates are made for each increase/decrease state in the direction in which each parameter on the horizontal and vertical axes (body activation/inactivity (sedation) degree, and brain activation (arousal)/inactivity (low arousal) degree) is increased or decreased. It expresses the technical (psychological) idea of assigning emotional states. For example, the example in FIG. 4 shows that when the index based on biological signals changes in a direction where the tendency toward physical sedation and low arousal becomes stronger, the emotion shifts toward "rest and relaxation."

The psychological judgment table 123 shown in FIG. 6 converts the technical idea shown in FIG. 4 into a data table, and enables processing by a controller such as the control unit 13.

The processing contents of each part of the control unit 13 will be explained. In the following description, the main body of processing by the extraction unit 131, generation unit 133, identification unit 134, and provision unit 135 can be translated as the control unit 13. Further, the control unit 13 is an example of a controller.

The extraction unit 131 extracts information regarding the relationship between physical condition index data and emotion type from psychological medical evidence (papers, etc.) obtained from an external academic literature database or the like via the communication unit 11. FIG. 7 is a diagram illustrating a method for extracting emotion types.

As shown in FIG. 7, the extraction unit 131 performs natural language analysis on texts written in papers, books, etc. to extract information regarding the relationship between biological signals, indicators of physical conditions, and emotion types.

The extraction unit 131 performs natural language analysis using, for example, existing machine learning techniques or AI (artificial intelligence), and extracts information regarding the relationship between biological signals, physical condition indicators, and emotion types.

Specifically, in the example of FIG. 7, the extraction unit 131 extracts information from a living body based on text information such as "As the β waves (of the brain waves) become larger, the activity of thinking and judgment increases." ``Beta-wave predominance'' regarding signals is associated with ``increased activity of thinking and judgment'' regarding indicators for emotion estimation. In addition, the extraction unit 131 extracts ``The activity of thinking judgment increases as the activity of thinking judgment becomes stronger, the emotions such as happiness, anger, and sadness increase.'' ``enhancement (activation of thinking and judgment becomes stronger)'' is associated with emotions such as ``fun'', ``anger'', and ``sadness''.

In addition, the extraction unit 131 extracts information regarding "α wave predominance" related to biological signals based on text information such as "As alpha waves (of brain waves) become larger, the activity of thinking and judgment decreases." in the medical evidence information. Associating "decreased thinking and judgment activity" with regard to indicators for emotion estimation. In addition, the extraction unit 131 extracts "As the activity of thinking and judging becomes weaker, the emotions such as anxiety and fear increase." ``Anxiety'' and ``Fear'' related to emotions are associated with ``The activity of thinking and judgment becomes weaker.''

The generation unit 133 generates an emotion estimation model based on the biological signals extracted by the extraction unit 131, the index of the physical condition, and the information regarding the relationship between emotion types. The emotion estimation model is a model that generates a physical state index value based on biological signals and then estimates an emotional state from two types of physical state index values. Specifically, for example, the two types shown in FIG. This model is mainly based on a psychological judgment table 123 that determines (estimates) the emotional state from the state of change in the index value of the physical state. The generation unit 133 also generates data tables used when using the emotion estimation model, that is, the sensor table 121, the psychological plane table 122, and the psychological judgment table 123 shown in FIGS. 5 and 6.

To explain with a specific example (the value of the data record of sensor IDSN01 in FIG. 3 is exemplified), the generation unit 133 adds a new value to the "sensor type" of the sensor table 121 due to an operator's operation or an automatic search process for Internet information. The input sensor type data (brain wave sensor BA) is stored, and a "sensor ID" (SN01) is automatically assigned accordingly. Note that, when obtaining newly input sensor type data, the generation unit 133 also separately stores various information regarding the sensor (for example, measurement items) in the storage unit 12. Then, the generation unit 133 generates the information in the sensor table 121 based on information indicating the relationship between the measurement item data regarding the sensor, the biological signal extracted by the extraction unit 131, and the index for emotion estimation (judgment based on commonality). Data to be stored in "Biological Signal Type" and "Corresponding Index Type" are generated and stored in the sensor table 121 (β waves/α waves of brain waves and activity of thought recognition judgment). At this time, the generation unit 133 adds and stores identification data for the index in the "corresponding index ID" for a new index for emotion estimation, and stores the identification data for the index for the existing index for emotion estimation. Store the corresponding identification data (VS01). Also, based on information indicating the relationship between the biological signal extracted by the extraction unit 131 and the index for emotion estimation (the relational expression between the two and conversion table information), data to be stored in the "index conversion information" of the sensor table 121 is stored. It is generated and stored in the sensor table 121 (FX01).

Furthermore, when generating an index for new emotion estimation (specific data is exemplified as VS03), the generation unit 133 adds a new "index" ( Set VS03) on both the vertical and horizontal axes. Based on the information regarding the relationship between the physical condition index and the emotion type extracted by the extraction unit 131, the generation unit 133 generates the emotion for the index value in the new index (VS03) and the index in each existing index (VS01, VS02). Psychological plane data (Psychological Plane 01-03 , psychological planes 02-03) are generated and stored in the corresponding storage cells (frames) of the psychological plane table 122.

Note that the psychological plane is expressed conceptually, and the concept is similar to the plane space shown in FIG. 3. Specifically, it is composed of a data group (data table, etc.) that can be processed on the psychological plane, and is as shown in the psychological judgment table 123 shown in FIG.

In this embodiment, the psychological plane table 122 is used in an example in which emotions are estimated using two types of physical condition indicators, but a psychological three-dimensional table is used in an example in which emotions are estimated using three types of physical condition indicators. For example, psychological tables with dimensions corresponding to the number of types of physical condition indicators used are used.

Next, a specific example of a method for generating an emotion estimation model by the generation unit 133 will be described using the example of the emotion estimation model (psychological plane) shown in FIG.

Here, the vertical axis of the index axis on the psychological plane uses the index of "activation of thinking recognition judgment" (biological signal: β wave/α wave of brain waves (index ID: "VS01")), and the horizontal axis of the index axis An index of "physical activity" (biological signal: low frequency component fluctuation of heart waveform (index ID: "VS01")) is used for the axis.

Note that the extraction unit 131 has previously extracted emotional content information based on medical evidence (papers, etc.) regarding the strength of the index "activation of thinking recognition and judgment" and the index "physical activation" as follows. shall be.
``Activity of thinking, recognition, and judgment'' is strong (activation state): Depression, enjoyment, joy, anger, sadness ``Activity of thinking, recognition, and judgment'' is weak (inactive state): Relaxation, calmness, anxiety, fear, unpleasantness ``Body Strong "activity" (activity state): enjoyment, joy, anger, sadness, anxiety, fear, unpleasantness Low "physical activity" (inactive state): depression, relaxation, calmness

The generating unit 133 allocates each emotion (content) to each quadrant of the corresponding psychological plane based on the above-mentioned indicators extracted by the extracting unit 131 and the relationship between the emotional content and the strength of the indicator, and completes the psychological plane. .

For example, the first quadrant of the psychological plane is an area of situations where the index value of "Activity of thinking, recognition and judgment" is strong, and the index value of "Physical activity" is strong. A common emotional content is assigned to the emotional content corresponding to a strong index value of "activity" and the emotion corresponding to a strong index value of "physical activity."

Specifically, the generation unit 133 generates the emotional contents on the positive side (the side with strong “activation of thought recognition and judgment”) corresponding to the index ID “VS01” and the emotional content on the positive side (the side with “stronger activity of thinking recognition judgment”) corresponding to the index ID “VS02” The emotional content of the "active" side) and the emotional content common to "fun, joy, anger, sadness" are placed in the region 210 of the first quadrant.

In addition, the generation unit 133 generates the emotional content “depression” that is common to the positive emotional content corresponding to the index ID “VS01” and the negative emotional content corresponding to the index ID “VS02” in the second quadrant. It is placed in area 220.

In addition, the generation unit 133 generates a third emotion type “relaxation, calmness” that is common to the negative emotion content corresponding to the index ID “VS01” and the negative emotion content corresponding to the index ID “VS02”. It is placed in the quadrant area 230.

In addition, the generation unit 133 generates emotional content “anxiety, fear, unpleasantness” that is common to the positive emotional content corresponding to the index ID “VS01” and the negative emotional content corresponding to the index ID “VS02”. It is placed in the area 240 of the fourth quadrant.

The acquisition unit 136 receives data such as various biological signals from the terminal device 20 via the communication unit, and stores necessary data in the internal memory or storage unit 12 for subsequent processing.

The model selection unit 137 selects a model to be used for emotion estimation based on the types of various biological signals included in the data acquired by the acquisition unit 136. Specifically, the model selection unit 137 collates the "sensor type" data of the sensor table 121 with the type of biosignal and extracts the index type data of the "corresponding index type" corresponding to the biosignal. Then, the model selection unit 137 collates the psychological plane table 122 with the extracted index type data (two types used for emotion estimation), and creates an emotion estimation model using the corresponding model (data at the intersection of the two types of indicators). cell's psychological plane model). For example, when the sensor types are brain wave sensor BA (sensor IDSN01) and heart rate sensor HA (sensor IDSN02), the corresponding biological state indicators are "activity of thinking recognition judgment" (index IDVS01) and "physical activity" (index IDVS02). , the model selection unit 137 selects the model of psychological plane 01-02 as the emotion estimation model to be used.

The identification unit 134 adds the biological signal acquired by the acquisition unit 136 to the emotion estimation model selected by the model selection unit 137, that is, the emotion estimation model corresponding to the index of the physical condition to be used (in other words, the sensor type to be used). In other words, the type of emotion is identified (estimated emotion) by applying an index based on actually measured biological signals.

Specifically, for the two types of biosignals acquired by the acquisition unit 136, the identification unit 134 extracts index conversion information corresponding to the biosignals from the sensor table 121 (for example, the sensor type is brain wave sensor BA (sensor IDSN01)). In the case of heart rate sensor HA (sensor IDSN02), index conversion information FX01 and index conversion information FX02) are read out, and the biological signal is converted into a biological condition index value using the index conversion information.

The specifying unit 134 then converts the emotion estimation model selected by the model selecting unit 137 onto the psychological plane (when using the brain wave sensor BA and heart rate sensor HA, on the psychological plane 01-02). The coordinates of two types of biological condition index values are sequentially plotted, and the emotion type is estimated based on the trajectory. Specifically, the specifying unit 134 performs matching processing (graphic pattern recognition of the locus) on each data of the "change pattern" in the psychological judgment table 123 in the state of change of the two types of index values based on the biological signals, and performs corresponding processing. The "emotion type" data is used as the emotion estimation result.

The identification unit 134 acquires a plurality of indicators based on biosignals of the user (for example, the subject U02a and the subject U02b), and estimates the user's emotion based on changes in the plurality of indicators over time.

In addition, the emotion estimation model converts biosignals into emotion estimation indicators based on indicator conversion information, and converts the state of change in coordinates expressed by multiple types of emotion estimation indicators into psychological information that associates the change state with emotional information. This is a model that estimates the user's emotions based on map information (for example, the psychological plane table 122).

Changes in biological signals (indicators based on biological signals) are the difference components between biological signals (indicators), so errors based on individual differences and differences in the surrounding environment are canceled out, and emotions based on changes in biological signals (indicators) are canceled out. The estimation of can suppress the influence of individual differences and differences in the surrounding environment.

Next, the emotion estimation method when using the psychological plane table 122 shown in FIG. 3 will be specifically explained.

Here, the first index ("activation of thinking recognition and judgment") at time t is x _t (X coordinate value), and the second index ("physical activity") at time t is y _t (Y coordinate value). do. In other words, the coordinates (x _t , y _t ) (XY plane coordinate values) on the psychological plane of the target at time t (the psychological plane corresponding to the psychological plane table 122).

In this case, the coordinates at time t+1 after time t are (x _t+1 , y _t+1 ). The specifying unit 134 displays the amount of change from time t to time t+1 as a vector (x _t+1 ^- x _t , y _t+1 − y _t ).

Subsequently, the specifying unit 134 specifies which change direction (change pattern) of the "change patterns" in the psychological plane table 122 in FIG. 3 the direction indicated by the vector is close to. Specifically, the specifying unit 134 determines the direction of the coordinate position of the vector (x _t+1 ⁻ x _t , y _t+1 − y _t ) from the origin (with the X-axis positive direction (right direction) being 0°, Calculate the conversion formula for the increasing angle) from rectangular coordinates to polar coordinates. Since the index value coordinate change direction is classified into 8 directions, the comparison angles for vector direction discrimination are 0°, 45°, and 90° for the right, upper right, upper, upper left, left, lower left, lower, and lower right directions, respectively. °, 135°, 180°, 225°, 270°, and 315°.

The specifying unit 134 specifies the angle closest to the direction (angle) of the above-mentioned vector from among the above-mentioned angles, and sets the corresponding direction as the direction of change of the index value coordinates. For example, if the vector is (1.2, 1.5), the angle in polar coordinates is approximately 51°, which is the closest to 45° of the above angles. Therefore, the identifying unit 134 identifies the direction of change in the index value coordinates as the upper right.

Furthermore, the specifying unit 134 collates the "change pattern" data of the psychological plane table 122 in the specified direction (for example, upper right), and selects the "emotion type" data of the subject when the direction of change of the index value coordinate is upper right. Estimate as an emotion (e.g. uplifting psychic up direction (increase in pleasure)).

The providing unit 135 provides the terminal device 20 with the emotion information estimated by the specifying unit 134.

Then, the analyst U01 uses the information regarding emotions provided by the providing unit 135 by, for example, viewing it via the terminal device 20. FIG. 8 is a diagram showing an example of the evaluation screen and the usage status of information regarding emotions.

The providing unit 135 provides a message in natural language along with an arrow indicating a change in emotion (using the corresponding "display symbol" data in the psychological judgment table 123), that is, information indicating a direction of coordinate change in the corresponding psychological plane. The evaluation screen 21 of the terminal device 20 displays information provided from the providing unit 135 for each subject. Note that this information can be realized by additionally storing message data in the psychological judgment table 123 or by installing a database in which emotional information and message information are associated in the terminal device 20.

In the following explanation, "Mr. A" refers to subject U02a. Furthermore, "Mr. B" refers to subject U02b.

In the example of FIG. 8, the evaluation screen 21 displays a message "Mr. A is psyching up. (This is the correct direction.)" along with an arrow in the upper right direction. Analyst U01 refers to the content displayed on the evaluation screen 21 and gives appropriate advice to subject U02a according to A's mental state, for example, "Mr. A, how are you doing?" (Advice information etc. can be sent to Mr. A's terminal such as a game machine).

Further, in the example of FIG. 8, the evaluation screen 21 displays a message "Mr. B is depressed (parasympathetic nerves are dominant)" along with an arrow in the upper left direction. Analyst U01 refers to the content displayed on the evaluation screen 21 and gives appropriate advice to subject U02b according to Mr. B's mental state, such as "Mr. B, please be conscious of activating your sympathetic nervous system." can be sent to Mr. B (sending advice information, etc. to Mr. A's terminal such as a game machine).

Next, an example of how the estimation system 1 is used in e-sports will be explained. Note that the subject U01a and the subject U02b are e-sports players. Here, it is assumed that in the e-sports, it is desirable for the player to consciously cause emotional changes in the upper right direction and lower left direction on the relevant psychological plane, and to avoid emotional changes in the upper left direction and lower right direction as much as possible. Note that what kind of emotional change is desirable varies depending on the title, scene, etc. of the game to be played. Furthermore, what kind of emotional change is desirable also differs depending on the type of use other than games. Therefore, a database etc. in which emotional information and message information are associated according to the usage pattern of the estimation system 1 is prepared, the database etc. is selected and used according to the usage style, and messages etc. according to the usage style are presented. It is preferable that the

In the example of FIG. 9, the evaluation screen 21 displays the message "Mr. A is in the relaxing direction. (This is the correct direction.)" along with the direction of the arrow at the bottom left. Analyst U01 can refer to the content displayed on the evaluation screen 21 and give appropriate advice to subject U02a according to Mr. A's mental state, such as, "Mr. A is learning well."

Further, in the example of FIG. 9, the evaluation screen 21 displays a message "Mr. B is anxious (sympathetic nerves are dominant)" along with an arrow pointing toward the lower right. Analyst U01 refers to the content displayed on the evaluation screen 21 and tells subject U02b, "Mr. B's brain arousal is going well. Let's do our best to control the autonomic nervous system," depending on Mr. B's mental state. can give appropriate advice.

In this way, it becomes possible to give appropriate advice to the player based on the estimation result of the emotional state.

In addition, the emotion estimation index includes a first index representing the degree of brain arousal (for example, an index corresponding to the vertical axis in Figure 3) and a second index representing the degree of activation of the user's autonomic nerves (for example, the index corresponding to the vertical axis in Figure 3). (index corresponding to the horizontal axis of 3).

(Calibration of emotion estimation model)
The emotion estimation method described above captures changes in emotions (for example, changes in the direction of depression (increasing tendency)), that is, captures relative emotions, but It can also be applied to the calibration of things that capture things (for example, the level of depression). In this case, the server 10 estimates the emotion based on the values of the plurality of indicators themselves when calibration using a plurality of indicators based on the collected biological signals of the user becomes possible.

FIG. 10 is a diagram illustrating a method for calibrating an emotion estimation model. The set of continuous arrows in FIG. 10 represents the change status of the coordinate position of the biological condition index specified by the specifying unit 134 at each time using arrows (indicating the starting point and ending point of the vector).

Furthermore, the vertical and horizontal axes represented by broken lines on the left side of FIG. 10 are the respective index axes in the psychological plane of the emotion estimation model arranged at tentative positions.

The specifying unit 134 calculates the start point (or end point) of each arrow, that is, the average of each coordinate value of the two types of biological condition indicators (after collecting a predetermined number of coordinate values that can be expected to be accurate). Then, index correction values for the two types of biological conditions are calculated so that the average coordinate value becomes the origin on the psychological plane. (the vertical and horizontal axes are moved to the vertical and horizontal axes represented by solid lines on the right side of FIG. 10). Then, the specifying unit 134 corrects the two types of biological condition index values using the correction values, applies the corrected correction index values to the emotion estimation model that estimates absolute emotions, and calculates the emotion. presume. That is, statistical processing is performed on many biological condition index values, and calibration is performed using the statistically processed values.

As a result, errors in biological signals due to individual differences and the surrounding environment are canceled out and suppressed, and the accuracy of emotion estimation in the emotion estimation model that estimates absolute emotions can be improved.

In addition, in the above example, calibration was performed using the average value of each coordinate value in two types of biological condition indicators, but the center of gravity is in the area determined by the maximum and minimum values of each coordinate value in two types of biological condition indicators ( It is also possible to apply a correction in which the origin is the center of gravity of a rectangle on the psychological plane in which each coordinate value fits (correction that adjusts the size of the rectangle is also effective).

In addition, since this calibration takes time to collect the data necessary for calibration, emotions are initially estimated based on changes in biological condition index values, and after a sufficient amount of data has been collected, absolute Estimating emotions using an emotion estimation model that estimates emotions (combined use is also possible) is also effective in terms of responsiveness.

(Machine learning model that learns patterns of emotional changes)
The estimation of the emotional state by the identification unit 134 can also be performed using artificial intelligence. In that case, the specifying unit 134 performs learning control on the built-in artificial intelligence based on the operation of the operator or the like to instruct learning and the learning data provided by the operator or the like. The specifying unit 134 estimates the emotional state by applying the input (acquired) biosignal to the trained machine learning model, and provides the estimation result to the terminal device 20 by the providing unit 135. In this case, the emotion estimation model is a machine learning model that has learned the correspondence between emotions and patterns of changes in multiple indicators over time.

FIG. 11 is a diagram illustrating a learning method of a machine learning model. As shown in FIG. 11, the identification unit 134 uses a machine learning model LM (for example, a deep neural network) to determine the change data of the measured biological signals (difference data of the biological signals measured at predetermined time intervals) and the change data of the measured biological signals. The emotional state (emotional change: determined by analysis of questionnaires, facial expressions, etc.) data is combined with the learning data (LNIN ). Through such learning, a machine learning model is generated in which the emotional state is estimated and output (FEOUT) based on the input of biological signal data (LDIN).

In this way, in this example, the providing unit 135 estimates the user's emotion using a machine learning model that has learned the correspondence between patterns of changes in a plurality of indicators over time and emotions.

Next, the flow of processing performed by the server 10 (control unit 13) will be described using FIGS. 12 and 13. FIG. 12 is a flowchart showing a process of generating an emotion estimation model based on external medical evidence (papers, etc.), that is, a model generation process of generating a sensor table 121, a psychological plane table 122, and a psychological judgment table 123. , the control unit 13 executes the model generation processing start operation by an operator or the like.

In step S101, the control unit 13 (extraction unit 131) of the server 10 collects medical evidence (papers, etc.) from a medical information database, Internet information, etc., and moves to step S102.

In step S102, the control unit 13 (extraction unit 131) performs language analysis of the collected medical evidence (papers, etc.), and extracts sensor-related information regarding various sensors (sensor identification data, sensor type, output biosignal type, etc.) , stratification, etc. to extract data on the relationship between biological signals and biological condition indicators, and data on the relationship between biological condition indicators and emotions (especially the relationship between changes in biological condition index values and emotional changes) and convert them into table data. After processing, the process moves to step S103.

In step S103, the control unit 13 (generation unit 133) generates the sensor table 121 based on the sensor-related information and data regarding the relationship between biological signals and biological condition indicators, and proceeds to step S104.

In step S104, the control unit 13 (generation unit 133) generates an emotion estimation model (psychological plane) based on the two types of biological condition indicators based on data regarding the relationship between the two types of biological condition indicators and emotions, and further generates an emotion estimation model (psychological plane) based on the two types of biological condition indicators. Emotion estimation models are similarly generated for each combination of state indicators, and the emotional plane table 122 is generated by arranging the emotion estimation models in the matrix table of biological state indicators, and the process ends. Note that the emotion estimation model (psychological plane) becomes a psychological judgment table 123 by converting it into table data.

FIG. 13 is a flowchart showing an emotion estimation process for estimating an emotional state, in which an emotion estimation request signal (by a request operation by an operator, etc.) is received from the terminal device 20, or a activation signal for a system for using estimated emotions such as a game. The control unit 13 starts execution upon reception of the estimated emotion, and the execution is repeated until the estimated emotion utilization system is stopped (e.g., the end of the game).

In step S201, the control unit 13 (acquisition unit 136) of the server 10 receives data such as various biological signals from the terminal device 20 via the communication unit 11, and stores the data in the internal memory or storage for subsequent processing. The necessary data is stored in the section 12, and the process moves to step S202.

In step S202, the control unit 13 (model selection unit 137) selects a model to be used for emotion estimation based on the types of various biosignals (two types of biosignals in this example) included in the data acquired from the terminal device 20 in step S201. is selected, and the process moves to step S203.

In step S203, the control unit 13 (acquisition unit 136) receives data of various biological signals from the terminal device 20 via the communication unit 11, and saves data to the internal memory or storage unit 12 for subsequent processing. The data is stored and the process moves to step S204.

In step S204, the control unit 13 (identification unit 134) uses the emotion estimation model (sensor table 121) selected in step S202 to identify the biological signals (two types of biological signals in this example) acquired from the terminal device 20. It is converted into a state index value, and the process moves to step S205.

In step S205, the control unit 13 (identification unit 134) determines whether the biological condition index values converted and generated in step S204 have reached a predetermined number (the number required for trajectory determination, for example, 10), If the predetermined number is reached, the process moves to step S206 (at this time, the count number of the generated biological condition index value is reset), and if the predetermined number has not been reached, the process returns to step S203.

In step S205, the control unit 13 (specification unit 134) determines the coordinates based on a predetermined number of biological condition index values (in the case of two types of biological condition index values, one of the psychological planes is a vertical axis value and the other axis is a horizontal axis value). Then, the trajectory (change state) of the result is calculated, and the process moves to step S206.

In step S206, the control unit 13 (identifying unit 134) performs a matching process (graphic pattern recognition of the trajectory) on the psychological judgment table 123 using the trajectory of the biological condition index calculated in step S205, and selects and extracts based on the matching result. With the emotion information as the estimation result, the process moves to step S207. The processing in steps S204, S205, and S206 is the application processing of the biosignal to the emotion estimation model.

In step S207, the control unit 13 (providing unit 135) transmits (provides) the emotional information estimated in step S206 to the terminal device 20 through the communication unit 11, and ends the process.

In this embodiment, the server 10 performs many processes such as emotion estimation processing and emotion estimation model generation processing, but the terminal device 20 may be provided with the above-mentioned configuration to perform these processings, and the game It is also possible to perform these processes by providing a configuration in the game machine used by the player. Further, it is also possible to appropriately distribute each process among the server 10, the terminal device 20, and the game machine that are communicatively connected, for example, the terminal device 20 performs the process of converting a biological signal into a biological condition index value.

The emotion estimation system includes a game device on which a user plays a game, a server 10 that estimates emotions, and a terminal device 20 that manages the game. For example, the game device is a device operated by test subject U02a and test subject U02b in FIG. 1, and may be a PC or a so-called consumer device. Further, the subject U02a and the subject U02b are examples of users.

The game device has a game machine controller (for example, a CPU). The game machine controller measures the biosignal of the user using the biosensor, transmits the measured biosignal of the user to the terminal device 20, receives the estimated emotion-related information transmitted from the terminal device 20, and receives the estimated emotion-related information. Provide emotion-related information to users.

The terminal device 20 has a terminal controller (for example, a CPU). The terminal controller receives the user's biosignal from the game device, transmits the received user's biosignal to the server 10, receives the estimated emotion-related information transmitted from the server 10, and uses the received estimated emotion-related information in the game. Send to device.

The server 10 includes an emotion estimation controller (for example, a CPU). The emotion estimation controller receives the user's biosignal from the terminal device 20, estimates the user's emotion based on changes over time in a plurality of indicators based on the received user's biosignal, and estimates the estimated emotion related to the estimated emotion. The information is transmitted to the terminal device 20, a plurality of indicators are obtained based on the user's biological signals measured by the biosensor, and the user's emotions are estimated based on changes in the plurality of indicators over time.

Further, the terminal device 20 outputs information indicating the user's emotion estimated by the server 10.

Furthermore, in this embodiment, the biosignal is converted into an emotion estimation index and applied to the emotion estimation model. It is also possible to apply the model generation method to the model generation method for the estimation index).

As described above, according to the present embodiment, the subject's emotional information is generated based on changes and fluctuations in the subject's biological signals, so that calibration of the emotion estimation model, etc. can be omitted. In other words, since changes in biosignals are the difference component between biosignals, errors based on individual differences and differences in the surrounding environment are canceled out. The influence of differences can be suppressed. As a result, according to the embodiment, the accuracy of estimating emotions based on biological signals can be easily improved.

Furthermore, in this embodiment, a biological signal is converted into an emotion estimation index, and the emotion estimation index is applied to an emotion estimation model for the emotion estimation index to estimate an emotion. There are many types of sensors that measure biological signals, and it is difficult to prepare a complex emotion estimation model for each sensor. However, the main correction point for converting biological signals into emotion estimation indicators is to match the output characteristics of the sensors, which can be achieved relatively simply and easily. Therefore, as in this embodiment, by including conversion processing to standardize the biological signals of each sensor, the types of emotion estimation models can be reduced, which is advantageous in terms of system scale.

Further advantages and modifications can be easily deduced by those skilled in the art. Therefore, the broader aspects of the invention are not limited to the specific details and representative embodiments shown and described above. Accordingly, various changes may be made without departing from the spirit or scope of the general inventive concept as defined by the appended claims and their equivalents.

N Network U01 Analyst U02 Subject 1 Estimation system 10 Server 11 Communication unit 12 Storage unit 13 Control unit 20 Terminal device 21 Evaluation screen 31a, 31b, 32a, 32b Sensor 121 Sensor table 122 Psychological plane table 123 Psychological judgment table 131 Extraction unit 132 Psychological analysis section 133 Generation section 134 Specification section 135 Provision section 136 Acquisition section 137 Model selection section 210, 220, 230, 240 Area

Claims (9)

An emotion estimation device for estimating emotions, comprising a controller,
The controller includes:
An emotion estimation device that estimates a user's emotion based on changes over time in multiple indicators based on the user's biological signals. has a memory device,
The controller includes:
The emotion estimation device according to claim 1, wherein the emotion estimation device estimates the user's emotion based on an emotion estimation model stored in the memory device. The emotion estimation model is
converting the biological signal into an emotion estimation index based on index conversion information;
The emotion estimation device according to claim 2, which is a model that estimates the user's emotion based on psychological map information in which a change state of coordinates represented by a plurality of types of emotion estimation indicators is associated with emotional information. The emotion estimation index is
The emotion estimation device according to claim 3, comprising: a first index representing a degree of brain arousal; and a second index representing a degree of autonomic nerve activation of the user. The emotion estimation model is
The emotion estimation device according to claim 2, which is a machine learning model that has learned a correspondence relationship between a pattern of changes in the plurality of indicators over time and emotions. The controller includes:
The emotion estimation device according to claim 2, wherein the emotion estimation model is generated based on evidence information acquired from the outside. The controller includes:
The emotion estimation device according to any one of claims 1 to 6, wherein when calibration using a plurality of indicators based on collected biological signals of the user becomes possible, emotions are estimated based on the values of the plurality of indicators themselves. . An emotion estimation method for estimating emotions, the method comprising:
An emotion estimation method that estimates a user's emotion based on changes over time in multiple indicators based on the user's biological signals. An emotion estimation system comprising a game device on which a user plays a game, an emotion estimation device for estimating an emotion, and a terminal device for managing the game,
The game device has a game machine controller, and the game machine controller includes:
Measures the user's biosignal using a biosensor,
transmitting the measured biosignal of the user to the terminal device;
receiving estimated emotion related information transmitted from the terminal device;
Providing the received estimated emotion related information to the user,
The terminal device includes a terminal controller, and the terminal controller includes:
receiving a biosignal of the user from the game device;
transmitting the received user's biosignal to the emotion estimation device;
receiving estimated emotion related information transmitted from the emotion estimation device;
transmitting the received estimated emotion related information to the game device;
The emotion estimation device includes an emotion estimation controller,
The emotion estimation controller is
receiving a user's biosignal from the terminal device;
Estimating the user's emotion based on changes over time in multiple indicators based on the user's biological signals received,
transmitting estimated emotion related information regarding the estimated emotion to the terminal device;
obtaining a plurality of indicators based on biosignals of the user measured by the biosensor;
Estimating the user's emotion based on changes in the plurality of indicators over time;
The terminal device is
An emotion estimation system that outputs information indicating the user's emotion estimated by the emotion estimation device.

Images