WO2022144978A1

WO2022144978A1 - Information processing device, control method, and storage medium

Info

Publication number: WO2022144978A1
Application number: PCT/JP2020/049145
Authority: WO
Inventors: あずさ古川; 剛範辻川; 恵渋谷
Original assignee: 日本電気株式会社
Priority date: 2020-12-28
Filing date: 2020-12-28
Publication date: 2022-07-07
Also published as: JPWO2022144978A1

Abstract

An information processing device 1X mainly comprises an acquisition means 16XA and a mental tendency estimation means 16XB. The acquisition means 16XA acquires a plurality of sets of a stressed state of a subject, and a mental state when the subject is in the stressed state. The mental tendency estimation means 16XB estimates, on the basis of the plurality of sets acquired by the acquisition means 16XA, a tendency in the mental state of the subject.

Description

Information processing equipment, control method and storage medium

The present disclosure relates to technical fields of information processing devices, control methods, and storage media that perform processing related to estimation of an internal state.

A device or system for estimating the internal state of the subject is known. For example, Patent Document 1 discloses a mental and physical condition awareness support device that determines the degree of stress based on the measurement result of heart rate equivalent data and determines the mental state of the subject based on the questionnaire result.

Japanese Unexamined Patent Publication No. 11-169362

There are individual differences in what kind of mental state will occur in what situation, and in order to estimate the mental state without subjective evaluation by the subject himself, the tendency of the mental state of each individual is accurately estimated. There is a need to.

An object of the present disclosure is to provide an information processing device, a control method, and a storage medium capable of suitably estimating a tendency of a mental state in view of the above-mentioned problems.

One aspect of the information processing device is
An acquisition means for acquiring a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state, and
A mental tendency estimation means for estimating the tendency of the mental state of the subject based on the plurality of sets, and
It is an information processing device provided with.

One aspect of the control method is
The computer
Obtaining a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state,
Estimate the tendency of the subject's mental state based on the plurality of sets.
It is a control method. The "computer" includes any electronic device (which may be a processor included in the electronic device), and may be composed of a plurality of electronic devices.

One aspect of the storage medium is
Obtaining a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state,
It is a storage medium in which a program for causing a computer to execute a process of estimating a tendency of the mental state of the subject based on the plurality of sets is stored.

According to this disclosure, it is possible to accurately estimate the tendency of the subject's mental state.

A schematic configuration of the mental state estimation system according to the first embodiment is shown. The hardware configuration of the information processing device is shown. This is an example of a functional block of an information processing device. (A) It is a figure which visualized the learning data set acquired from the same person used in the 1st generation example. (B) It is a table showing the statistical information of the training data set in the 1st generation example. (C) An example of mental tendency information in the first generation example is shown. An example of the data structure of the mental tendency information generated by the second generation example is shown. (A) The schematic diagram regarding the generation of the training data set in the 3rd generation example is shown. (B) An example of the data structure of the mental tendency information generated in the third generation example is shown. This is an example of a flowchart showing a processing procedure for generating mental trend information. This is an example of a flowchart showing a processing procedure for estimating a mental state. A schematic configuration of the mental state estimation system according to the second embodiment is shown. It is a block diagram of the information processing apparatus in 3rd Embodiment. This is an example of a flowchart executed by the information processing apparatus in the third embodiment.

Hereinafter, embodiments of an information processing device, a control method, and a storage medium will be described with reference to the drawings.

<First Embodiment>
(1) System Configuration Figure 1 shows a schematic configuration of the mental state estimation system 100 according to the first embodiment. The mental state estimation system 100 estimates the tendency of the subject's mental state according to the stress state, estimates the mental state based on the estimated tendency, and presents the estimation result. Here, the "target person" may be an athlete or an employee whose mental state is managed by the organization, or may be an individual user. The "mental state" refers to the mental state (psychological state, mental state) of the subject, and specifically, the mood, emotion, or behavioral tendency of the subject (for example, the tendency of attitude / attitude toward a task, etc.). (Including the tendency of reaction to things, the tendency of attitude / attitude toward others, etc.).

The mental state estimation system 100 mainly includes an information processing device 1, an input device 2, an output device 3, a storage device 4, and a sensor 5.

The information processing device 1 performs data communication with the input device 2, the output device 3, and the sensor 5 via a communication network or by direct communication by radio or wire. Then, the information processing device 1 is mental of the subject based on the input signal "S1" supplied from the input device 2, the sensor signal "S3" supplied from the sensor 5, and the information stored in the storage device 4. Estimate the tendency of the state. Further, the information processing apparatus 1 generates an output signal "S2" regarding the tendency of the mental state of the subject and / and the estimation result of the mental state based on the tendency, and supplies the generated output signal S2 to the output device 3.

The input device 2 is an interface that accepts manual input (external input) of information about each target person. The user who inputs information using the input device 2 may be the target person himself / herself, or may be a person who manages or supervises the activities of the target person. The input device 2 may be various user input interfaces such as a touch panel, a button, a keyboard, a mouse, and a voice input device. The input device 2 supplies the generated input signal S1 to the information processing device 1. The output device 3 displays predetermined information or outputs sound based on the output signal S2 supplied from the information processing device 1. The output device 3 is, for example, a display, a projector, a speaker, or the like.

The sensor 5 measures the biometric data (biological signal) of the subject, and supplies the measured biometric data or the like to the information processing apparatus 1 as the sensor signal S3. In this case, the sensor signal S3 is arbitrary biological data (for example, heartbeat, brain wave, sweating amount, hormone secretion amount, cerebral blood flow, blood pressure, body temperature, myoelectricity, electrocardiogram, respiratory rate) used for stress estimation of the subject. Etc.). Further, the sensor 5 may be a device that analyzes blood collected from a subject and outputs the analysis result as a sensor signal S3. Further, the sensor 5 may be a wearable terminal worn by the target person, or may be a camera for photographing the target person, a microphone for generating an audio signal of the target person's utterance, or the like.

The storage device 4 is a memory that stores various information necessary for estimating the mental state and the like. The storage device 4 may be an external storage device such as a hard disk connected to or built in the information processing device 1, or may be a storage medium such as a flash memory. Further, the storage device 4 may be a server device that performs data communication with the information processing device 1. Further, the storage device 4 may be composed of a plurality of devices.

The storage device 4 functionally has a learning data set storage unit 41 and a mental tendency information storage unit 42.

The learning data set storage unit 41 stores the learning data set necessary for estimating the tendency of the mental state (also referred to as “mental tendency”) according to the stress state of each individual. Specifically, the learning data set storage unit 41 stores a learning data set which is a plurality of sets of the stress state of the subject and the mental state of the subject at the time when the stress state is measured. .. In other words, the learning data set storage unit 41 stores a set of a stress state and a mental state measured periodically or irregularly as a learning data set. The learning data set is stored in the learning data set storage unit 41 in association with, for example, the identification information of the subject.

The mental tendency information storage unit 42 stores mental tendency information, which is information indicating a mental tendency according to the stress state of each individual. The mental tendency information is generated by the information processing apparatus 1 for each subject based on the learning data set stored in the learning data set storage unit 41. The mental tendency information is stored in the mental tendency information storage unit 42 in association with the identification information of the corresponding target person. The mental tendency information may be table information showing the correspondence between the stress state and the mental state, or may be information about an inference model (also referred to as a “mental estimation model”) that estimates the mental state from the stress state. good. In the latter case, the mental estimation model is, for example, a regression model (statistical model) or a machine learning model, and in this case, the mental tendency information storage unit 42 stores information on parameters necessary for constructing the mental estimation model. do. For example, when the mental estimation model is a model based on a neural network such as a convolutional neural network, the mental tendency information storage unit 42 has a layer structure, a neuron structure of each layer, a number of filters and a filter size in each layer, and each element of each filter. Stores information on various parameters such as weights of.

The configuration of the mental state estimation system 100 shown in FIG. 1 is an example, and various changes may be made to the configuration. For example, the input device 2 and the output device 3 may be integrally configured. In this case, the input device 2 and the output device 3 may be configured as a tablet-type terminal integrated with or separate from the information processing device 1. Further, the input device 2 and the sensor 5 may be integrally configured. Further, the information processing device 1 may be composed of a plurality of devices. In this case, the plurality of devices constituting the information processing device 1 exchange information necessary for executing the pre-assigned process among the plurality of devices. In this case, the information processing apparatus 1 functions as an information processing system.

(2) Hardware Configuration of Information Processing Device FIG. 2 shows the hardware configuration of the information processing device 1. The information processing device 1 includes a processor 11, a memory 12, and an interface 13 as hardware. The processor 11, the memory 12, and the interface 13 are connected via the data bus 19.

The processor 11 functions as a controller (arithmetic unit) that controls the entire information processing device 1 by executing a program stored in the memory 6. The processor 11 is, for example, a processor such as a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and a TPU (Tensor Processing Unit). The processor 11 may be composed of a plurality of processors. The processor 11 is an example of a computer.

The memory 12 is composed of various volatile memories such as RAM (Random Access Memory), ROM (Read Only Memory), and flash memory, and non-volatile memory. Further, the memory 12 stores a program for executing the process executed by the information processing apparatus 1. A part of the information stored in the memory 12 may be stored by one or a plurality of external storage devices that can communicate with the information processing device 1, and is stored by a storage medium that can be attached to and detached from the information processing device 1. You may.

The interface 13 is an interface for electrically connecting the information processing device 1 and another device. These interfaces may be wireless interfaces such as network adapters for wirelessly transmitting and receiving data to and from other devices, and may be hardware interfaces for connecting to other devices by cables or the like.

The hardware configuration of the information processing device 1 is not limited to the configuration shown in FIG. For example, the information processing device 1 may include at least one of an input device 2 and an output device 3. Further, the information processing device 1 may be connected to or built in a sound output device such as a speaker.

(3) Functional block FIG. 3 is an example of a functional block of the information processing apparatus 1. The processor 11 of the information processing device 1 functionally has a mental state determination unit 14, a stress state determination unit 15, a mental tendency estimation unit 16, a mental state estimation unit 17, and an output control unit 18. In FIG. 3, the blocks in which data is exchanged are connected by a solid line, but the combination of blocks in which data is exchanged is not limited to FIG. The same applies to the figures of other functional blocks described later.

The mental state determination unit 14 determines the mental state of the questionnaire respondent based on the input signal S1 representing the questionnaire response result. In this case, the mental state determination unit 14 estimates the mental state of the questionnaire respondent from the questionnaire response result indicated by the input signal S1 based on the subjective evaluation method by an arbitrary questionnaire for measuring the mental state. As a method for measuring a mental state based on a questionnaire, for example, there are a KOKORO scale, PANAS (Positive and Negative Affect Schedule), and the like. When executing the questionnaire, the mental state determination unit 14 outputs the questionnaire response screen by transmitting the output signal S2, which is a display signal for displaying the questionnaire response screen, to the output device 3 via the interface 13. Display in 3. Further, the mental state determination unit 14 receives the input signal S1 representing the response result on the questionnaire response screen from the input device 2 via the interface 13.

The stress state determination unit 15 determines the stress state of the subject whose sensor signal S3 has been measured, based on the biological data represented by the sensor signal S3 supplied from the sensor 5. In this case, for example, the information necessary for determining the stress state from the sensor signal S3 is stored in advance in the storage device 4 or the memory 12, and the stress state determination unit 15 refers to this information to the sensor signal S3. Judge the stress state from. In this case, the above information may be, for example, information about an inference model previously learned by machine learning such as deep learning. There are various methods for determining (estimating) the stress state from biological data and the like, and the stress state determination unit 15 may determine the stress state by any of these methods. Here, the stress state determined by the stress state determination unit 15 may be the stress estimated value itself, or may be the stress level determined by comparing the stress estimated value with one or a plurality of threshold values. In the latter case, for example, the stress state is classified into a high stress state in which the stress estimated value is higher than a certain threshold value and a low stress state in which the stress estimated value is equal to or lower than the threshold value.

The determinations of the mental state determination unit 14 and the stress state determination unit 15 are executed for the same target person at substantially the same timing, and the information representing these determination results is used as a learning data set of the target person. It is associated with the identification information and the like and stored in the learning data set storage unit 41.

The mental tendency estimation unit 16 generates mental tendency information by estimating the mental tendency of the target person based on the learning data set stored in the learning data set storage unit 41. Then, the mental tendency estimation unit 16 stores the generated mental tendency information in the mental tendency information storage unit 42 in association with the identification information of the target person. The specific processing of the mental trend estimation unit 16 will be described later.

The mental state estimation unit 17 estimates the mental state of the subject whose mental tendency information exists in the mental tendency information storage unit 42. In this case, the mental state estimation unit 17 estimates the mental state of the target person based on the stress state determined by the stress state determination unit 15 based on the sensor signal S3 and the mental tendency information of the target person. In this case, the mental state estimation unit 17 preferably estimates the mental state of the subject without causing the subject to perform subjective evaluation such as answering a questionnaire. The mental state estimation unit 17 supplies the mental state estimation result to the output control unit 18.

The output control unit 18 outputs information regarding the estimation result of the mental state by the mental state estimation unit 17. For example, the output control unit 18 displays the estimation result by the mental state estimation unit 17 on the display unit, or outputs the voice by the sound output unit. In this case, the output control unit 18 may also display or output the mental tendency of the target person represented by the mental tendency information used by the mental state estimation unit 17. Further, the output control unit 18 may refer to the advice information in which the mental state and the necessary advice are associated with each other, and output the advice corresponding to the estimated mental state. In this case, the advice information is stored in advance in the storage device 4 or the memory 12.

For each component of the mental state determination unit 14, the stress state determination unit 15, the mental tendency estimation unit 16, the mental state estimation unit 17, and the output control unit 18 described in FIG. 3, for example, the processor 11 executes a program. It can be realized by. Further, each component may be realized by recording a necessary program in an arbitrary non-volatile storage medium and installing it as needed. It should be noted that at least a part of each of these components is not limited to being realized by software by a program, but may be realized by any combination of hardware, firmware, and software. Further, at least a part of each of these components may be realized by using a user-programmable integrated circuit such as an FPGA (Field-Programmable Gate Array) or a microcontroller. In this case, this integrated circuit may be used to realize a program composed of each of the above components. Further, at least a part of each component may be configured by an ASIC (Application Specific Standard Produce), an ASIC (Application Specific Integrated Circuit), or a quantum processor (quantum computer control chip). As described above, each component may be realized by various hardware. The above is the same in other embodiments described later. Further, each of these components may be realized by the collaboration of a plurality of computers by using, for example, cloud computing technology. The above is the same in other embodiments described later.

(4) Generation of Mental Trend Information Next, first generation examples to fourth generation examples, which are specific examples of generation of mental tendency information by the mental tendency estimation unit 16, will be described.

(4-1) First Generation Example FIG. 4A is a diagram that visualizes the learning data set (here, the determination results of 10 sets of mental state and stress state for the same person) used in the first generation example. be. Here, as an example, the mental state is coordinated according to the KOKORO scale, having a vertical axis in which an exciting mood is positive and an irritated mood is negative, and a horizontal axis in which a reassuring mood is positive and an anxious mood is negative. It is represented by two-dimensional coordinate values in the system. Further, the stress state is divided into two levels: a high stress state in which the stress estimated value is higher than a predetermined threshold value, and a low stress state in which the stress estimated value is equal to or lower than the threshold value. As shown in FIG. 4A, there is no difference in the tendency of the value on the vertical axis of the mental state between the low stress state and the high stress state, but there is a difference in the tendency of the value on the horizontal axis of the mental state. There is.

FIG. 4B is a table showing the statistical information of the training data set in the first generation example. The table shown in FIG. 4B shows the average value in the high stress state (may be a representative value other than the average value, the same applies hereinafter) and the average value in the low stress state for each axis of the mental state. And the difference between these average values are shown respectively. As shown in FIG. 4 (B), the difference (7) between the mean value in the high stress state and the mean value in the low stress state on the horizontal axis of the mental state is the mean value in the high stress state on the vertical axis. It is larger than the difference (1.5) between the value and the average value in the low stress state.

In this case, the mental tendency estimation unit 16 estimates the tendency of the mental state of the subject with respect to the axis with the largest difference (here, the horizontal axis). FIG. 4C shows an example of mental tendency information generated by the mental tendency estimation unit 16.

Here, the mental tendency estimation unit 16 has a mental state in a high stress state and a low stress state with respect to the target person (target person ID “X0”) who is the acquisition source of the learning data set shown in FIG. 4 (A). A record of mental trend information showing the trend related to the horizontal axis of is generated. Here, since the average value in the low stress state on the horizontal axis of the mental state is a negative value, the mental state in the low stress state is regarded as "anxious mood", and the mental state in the high stress state on the horizontal axis of the mental state is regarded as "anxious mood". Since the average value is a positive value, the mental state under high stress is regarded as "relieved mood".

In this way, the mental tendency estimation unit 16 identifies the tendency in each stress state with respect to the axis of the mental state having the largest difference (variation) among the levels of the stress state, and generates mental tendency information representing the specified tendency. .. As a result, the mental tendency estimation unit 16 can generate mental tendency information that accurately represents the mental state corresponding to each stress state.

In the first generation example, the level of the stress state was two stages (high stress state, low stress state), but even if it was three or more stages (for example, three stages of low stress, medium stress, and high stress). good. In this case, the mental trend estimation unit 16 calculates the variance of the average value for each level of the stress state for each axis of the mental state, and generates mental trend information indicating the tendency of the mental state with respect to the axis having the largest variance. .. Also with this, the mental tendency estimation unit 16 can generate mental tendency information that accurately represents the tendency of the mental state according to the level of the stress state. Similarly, the axes in the mental state are not limited to two axes, and three or more axes may exist.

(4-2) Second Generation Example FIG. 5 shows an example of the data structure of the mental tendency information generated by the second generation example. In the second generation example, as a premise, as a learning data set, a set of a stress state indicating two levels of high stress state and low stress state and a classified mental state is stored in the learning data set storage unit 41. It is assumed that it has been done. Further, "mental state A" to "mental state G" shown in FIG. 5 represent the classification of the mental state.

In this case, the mental trend estimation unit 16 aggregates the classifications of the mental states associated with each level of the stress state (that is, generates a frequency distribution), and calculates the frequency of each classification of the mental state for each level of the stress state. .. Then, the mental tendency estimation unit 16 generates mental tendency information representing the frequency of each classification of the mental state for each level of the stress state for each subject. In the example of FIG. 5, based on the aggregation of the learning data set, the mental state A accounts for 60%, the mental state C 20%, and the mental state G 20% in the high stress state, respectively, for the subject ID “X1”. There is. On the other hand, in the low stress state, the mental state B accounts for 70% and the mental state E accounts for 30%, respectively. These percentages can be considered as confidence (confidence) in the corresponding mental state classification.

As described above, in the second generation example, the mental tendency estimation unit 16 can suitably generate the mental tendency information that records the set of the mental state applied to each stress state and the degree of certainty thereof. Further, when estimating a mental state using mental tendency information, the mental state estimation unit 17 estimates a set of a mental state that may be applicable and its certainty, and the output control unit 18 estimates the set. The results can be preferably presented to the user.

Note that, instead of FIG. 5, the mental tendency estimation unit 16 may generate one mental tendency information indicating the classification of one mental state for each level of the stress state, as in FIG. 4C. In this case, the mental tendency estimation unit 16 generates mental tendency information in which the classification of the most applicable mental state for each level of stress state is associated with the certainty. Further, even when the level of the stress state is higher than the two stages (high stress state, low stress state), the second generation example is preferably applied.

(4-3) Third Generation Example FIG. 6A shows a schematic diagram relating to the generation of the training data set in the third generation example. FIG. 6B shows an example of the data structure of the mental tendency information generated in the third generation example. In the third generation example, on the premise that there are "good stress" that has a positive effect on the subject and "bad stress" that has an adverse effect on the subject, the mental tendency estimation unit 16 responds to the quality of the stress. Generates mental tendency information that represents the mental tendency.

As shown in FIG. 6A, when the training data set is generated, the mental state determination unit 14 determines the mental state in addition to the mental state determination result (here, the most applicable mental state classification). Generates a judgment flag that indicates the quality of. Then, the mental state determination result and the determination flag generated by the mental state determination unit 14 and the stress state determination result generated by the stress state determination unit 15 are associated and stored in the learning data set storage unit 41. In this case, the mental state determination unit 14 generates a determination flag from the determination of the determined mental state by referring to the determination table showing the quality of the mental state for each classification of the mental state, for example. The above-mentioned determination table is stored in advance in, for example, the storage device 4 or the memory 12.

Then, the mental trend estimation unit 16 generates mental trend information using the learning data set generated as shown in FIG. 6 (A). In this case, the mental tendency estimation unit 16 further classifies each stress state based on the determination flag, and estimates the most applicable mental state for each classified stress state. In FIG. 6B, the stress state is divided into two stages (high stress state and low stress state), and the mental tendency estimation unit 16 further indicates a determination flag for each level of these stress states. The frequency of mental status is totaled according to good / bad. As a result, the mental tendency estimation unit 16 determines the most applicable mental state for each stress level and good / bad combination. For example, when the subject with the subject ID "X1" is in a high stress state and the determination flag is "good", the "mental state G" is classified as the most applicable mental state, and the subject is in a low stress state. When the determination flag is "evil", the "mental state A" is the most applicable mental state classification.

As described above, in the third generation example, the mental tendency estimation unit 16 can suitably generate mental tendency information indicating the tendency of the mental state corresponding to each of good stress and bad stress. Then, by using such mental tendency information, the mental state estimation unit 17 estimates the mental state corresponding to the case of good stress and the case of bad stress, respectively, and the output control unit 18 estimates the estimation result. It can be preferably presented to the user.

As in the second generation example, the mental tendency estimation unit 16 generates mental tendency information in which a plurality of mental state classifications are associated with each certainty for each classification based on the quality of each stress state. You may. Further, even when the level of the stress state is three or more stages, the third generation example is preferably applied.

(4-4) Fourth Generation Example In the fourth generation example, the mental trend estimation unit 16 obtains parameters obtained by learning a mental estimation model that outputs a mental state corresponding to a stress state input. , Generated as mental trend information.

In this case, the mental tendency estimation unit 16 uses the stress state of the training data set as input data and the mental state of the training data set as correct data, and is a mental estimation model based on any machine learning or statistical model such as deep learning or a support vector machine. To learn. In this case, the stress state input to the mental estimation model may be a stress level such as a high stress state or a low stress state, or may be a stress estimation value that is a real value. Then, the mental tendency estimation unit 16 learns the mental estimation model for each target person based on the learning data set, and stores the parameters after learning the mental estimation model applied to each target person as mental tendency information.

Further, when estimating the mental state using the mental tendency information generated based on the fourth generation example, the mental state estimation unit 17 constitutes a mental estimation model based on the mental tendency information of the subject, and the stress state determination unit. The stress state determined by 15 is input to the mental estimation model. Then, the mental state estimation unit 17 supplies the mental state output from the mental estimation model to the output control unit 18 as the estimation result of the mental state of the target person, and the output control unit 18 outputs the estimation result. In this case, when the mental estimation model is a model based on deep learning, information on the corresponding certainty is obtained together with the candidate of the mental estimation model. Therefore, the output control unit 18 is similarly to the second generation example. It is also possible to present multiple mental state candidates with certainty.

As described above, even in the fourth generation example, the mental tendency estimation unit 16 can suitably generate mental tendency information for estimating the mental state according to the stress state.

(5) Processing Flow Next, the processing flow executed by the information processing apparatus 1 in the first embodiment will be described.

(5-1) Generation of Mental Trend Information FIG. 7 is an example of a flowchart showing a processing procedure of the information processing apparatus 1 regarding generation of mental tendency information. The information processing apparatus 1 executes the processing of the flowchart of FIG. 7 for each target person.

First, the information processing apparatus 1 determines the mental state and the stress state (step S11). In this case, the mental state determination unit 14 determines the mental state of the subject based on the questionnaire response represented by the input signal S1, and the stress state determination unit 15 determines the stress state of the subject based on the biological data represented by the sensor signal S3. Judgment is made.

Then, the information processing apparatus 1 stores the set of the mental state and the stress state determined in step S11 in the learning data set storage unit 41 as a learning data set (step S12). In this case, the information processing apparatus 1 associates the login information of the target person or the identification information of the target person acquired by arbitrary person recognition processing (for example, biometric authentication using a camera image or the like) with the above-mentioned learning data set. , Stored in the learning data set storage unit 41.

Then, the information processing apparatus 1 determines whether or not it is the generation timing of the mental tendency information (step S13). The information processing apparatus 1 determines, for example, that it is the generation timing of the mental tendency information when a predetermined number or more sets of mental states and stress states are accumulated in the learning data set storage unit 41 for the target person. When the stress states are classified into a plurality of categories (levels), the information processing apparatus 1 stores a predetermined number or more of mental state and stress state sets in the learning data set storage unit 41 for all the classifications. If so, it may be determined that it is the generation timing of the mental tendency information. In another example, when the information processing apparatus 1 acquires the input signal S1 related to the mental tendency information generation instruction, it determines that it is the mental tendency information generation timing.

Then, when the information processing apparatus 1 determines that it is not the timing for generating the mental tendency information (step S13; No), the processing returns to step S11, and the learning data set is continuously generated and accumulated.

On the other hand, in the case of the generation timing of the mental tendency information (step S13; Yes), the mental tendency estimation unit 16 of the information processing apparatus 1 is a target based on the learning data set of the target person stored in the learning data set storage unit 41. The mental tendency of the person is estimated (step S14). As a result, the mental tendency estimation unit 16 generates mental tendency information regarding the mental tendency according to the stress state, and stores the generated mental tendency information in the mental tendency information storage unit 42 in association with the identification information of the subject.

(5-2) Estimating the mental state FIG. 8 is an example of a flowchart showing a processing procedure of the information processing apparatus 1 regarding the estimation of the mental state. The information processing apparatus 1 executes the processing of the flowchart of FIG. 8 when there is a request for estimation of the mental state.

First, the information processing apparatus 1 determines the stress state of the subject (step S21). In this case, the stress state determination unit 15 of the information processing apparatus 1 determines the stress state of the subject based on the biological data represented by the sensor signal S3.

Next, the mental state estimation unit 17 of the information processing apparatus 1 acquires the mental tendency information associated with the target person from the mental tendency information storage unit 42 (step S22). In this case, the mental state estimation unit 17 acquires the target person's identification information by, for example, login information of the target person or arbitrary person recognition processing (for example, biometric authentication using a camera image or the like), and associates it with the identification information. The mental tendency information that has been present is extracted from the mental tendency information storage unit 42.

Then, the mental state estimation unit 17 estimates the mental state of the subject based on the stress state determined in step S21 and the mental tendency information acquired in step S22 (step S23). In this case, when the mental tendency information is table information, the mental state estimation unit 17 estimates the stress state of the target and the mental state associated with the table information as the mental state of the target person. If the mental tendency information is a parameter of the mental estimation model, the mental estimation model is constructed based on the parameter, and the stress state of the target is input to the mental estimation model, so that the mental output from the mental estimation model is performed. The state is estimated as the mental state of the subject.

Then, the output control unit 18 outputs the estimation result of the mental state by the mental state estimation unit 17 (step S24). In this case, the output control unit 18 supplies the output signal S2 to the output device 3 so that the output device 3 displays or outputs a voice representing the estimation result of the mental state. As a result, the information processing apparatus 1 can suitably present information regarding the mental state of the target person to the target person or the manager thereof.

(6) Modification Example Next, a modification suitable for the first embodiment will be described. The following modifications may be applied in combination.

(First modification)
Instead of determining the mental state based on the questionnaire response represented by the input signal S1, the mental state determination unit 14 may estimate the mental state of the subject based on the image of the subject.

In this case, for example, the mental state determination unit 14 acquires an image from the camera that captures the target person via the interface 13, and analyzes the facial expression of the target person from the acquired image, which is an example of the mental state. Recognize the subject's emotions. In this case, for example, the mental state determination unit 14 performs the above recognition using a model that infers the facial expression of a person in the image when the image is input. In this case, the storage device 4 or the memory 12 stores the parameters of the model that have been learned in advance based on deep learning or the like.

Similarly, the mental state determination unit 14 may estimate the mental state of the target person based on the voice data of the target person. In this case, the mental state determination unit 14 estimates the mental state of the subject based on, for example, the tone of the subject's utterance, the spoken word, or the like based on the voice data. In this case, the storage device 4 or the memory 12 stores information necessary for analyzing the voice data in advance.

Similarly, in the determination of the stress state, the stress state determination unit 15 determines the stress state of the subject by using an arbitrary stress state determination method based on the image of the subject or the voice data of the subject. You may judge.

(Second modification)
The stress state may be divided into chronic stress and acute stress.

In this case, for example, the stress state determination unit 15 determines the acute stress based on the latest sensor signal S3 acquired by the sensor 5, and the sensor 5 acquires the chronic stress a plurality of times in the latest predetermined period. Judgment is made based on the signal S3. In addition, the stress state determination unit 15 stores the sensor signal S3 acquired in the past predetermined period in the storage device 4 or the memory 12 in association with the identification information of the subject in order to determine the chronic stress.

Then, the information processing apparatus 1 is in a mental state in consideration of both acute stress and chronic stress, based on the first generation example to the fourth generation example explained in the section of "(4) Generation of mental tendency information ". Estimate the trend. In this case, when the information processing apparatus 1 executes, for example, the first generation example, for each of all combinations of acute stress levels and chronic stress levels (4 combinations when the level is 2 levels). Generate mental tendency information associated with the most applicable mental state. Further, in the case of the fourth generation example, the information processing apparatus 1 learns a mental estimation model that outputs a mental state when acute stress and chronic stress are input, and mentally sets parameters after learning the mental estimation model. Generated as trend information.

Also in this modification, the information processing apparatus 1 can suitably generate mental tendency information indicating a mental tendency for acute stress and chronic stress.

(Third modification example)
A device (other device) other than the information processing device 1 may have a function corresponding to the mental state estimation unit 17 and the output control unit 18. In this case, the other device refers to the mental tendency information storage unit 42 by performing data communication with the storage device 4, and similarly to the stress state determination unit 15, the biological data of the subject acquired by the sensor or the like is used. Based on this, the stress state is determined. Then, the other device estimates the mental state of the subject based on the mental tendency information stored in the mental tendency information storage unit 42 and the determined stress state.

Similarly, a device other than the information processing device 1 may have a function corresponding to the mental state determination unit 14 and the stress state determination unit 15. In this case, the mental tendency estimation unit 16 of the information processing apparatus 1 refers to a learning data set storage unit 41 that stores a learning data set generated by an apparatus having a function corresponding to the mental state determination unit 14 and the stress state determination unit 15. By doing so, the mental tendency information is generated.

<Second Embodiment>
FIG. 9 shows a schematic configuration of the mental state estimation system 100A in the second embodiment. The mental state estimation system 100A according to the second embodiment is a system of a server client model, and the information processing device 1A functioning as a server device performs the processing of the information processing device 1 in the first embodiment. Hereinafter, the same components as those in the first embodiment will be appropriately designated with the same reference numerals, and the description thereof will be omitted.

As shown in FIG. 9, the mental state estimation system 100A mainly includes an information processing device 1A that functions as a server, a storage device 4 that stores data similar to that of the first embodiment, and a terminal device 8 that functions as a client. And have. The information processing device 1A and the terminal device 8 perform data communication via the network 7.

The terminal device 8 is a terminal having an input function, a display function, and a communication function, and functions as an input device 2 and an output device 3 shown in FIG. The terminal device 8 may be, for example, a personal computer, a tablet terminal, a PDA (Personal Digital Assistant), or the like. The terminal device 8 transmits a biological signal output by a sensor (not shown), an input signal based on user input, or the like to the information processing device 1A.

The information processing device 1A has the same configuration as the information processing device 1 shown in FIGS. 1 to 3, for example. Then, the information processing device 1A receives information acquired from the input device 2 and the sensor 5 by the information processing device 1 shown in FIG. 1 from the terminal device 8 via the network 7, and has a mental tendency based on the received information. Generate information or estimate the mental state using the mental tendency information. Further, the information processing device 1A transmits an output signal indicating information regarding the estimation result of the mental state of the subject whose biometric data is detected by the sensor 5 based on the request from the terminal device 8 via the network 7. Send to. As a result, the information processing apparatus 1A preferably presents information regarding the estimation result of the mental state to the user of the terminal apparatus 8.

<Third Embodiment>
FIG. 10 is a block diagram of the information processing apparatus 1X according to the third embodiment. The information processing apparatus 1X mainly has an acquisition means 16XA and a mental tendency estimation means 16XB. The information processing device 1X may be composed of a plurality of devices.

The acquisition means 16XA acquires a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state. The mental tendency estimation means 16XB estimates the tendency of the mental state of the subject based on the plurality of sets acquired by the acquisition means 16XA. In this case, as the tendency of the mental state of the subject, the tendency of the mental state according to the stress state of the subject is estimated. It should be noted that this tendency may be expressed as a table in which the stress state and the mental state are associated with each other, or may be expressed as a model for estimating the mental state (and parameters for generating the model). The acquisition means 16XA and the mental trend estimation means 16XB can be the mental trend estimation unit 16 in the first embodiment (including a modification, the same applies hereinafter) or the second embodiment.

FIG. 11 is an example of a flowchart executed by the information processing apparatus 1X in the third embodiment. First, the acquisition means 16XA acquires a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state (step S31). The mental tendency estimation means 16XB estimates the tendency of the mental state of the subject based on the plurality of sets acquired by the acquisition means 16XA (step S32).

The information processing device 1X according to the third embodiment can suitably estimate the tendency of the mental state of the subject.

In each of the above-described embodiments, the program is stored using various types of non-transitory computer readable medium and can be supplied to a processor or the like which is a computer. Non-temporary computer-readable media include various types of tangible storage media. Examples of non-temporary computer readable media include magnetic storage media (eg flexible disks, magnetic tapes, hard disk drives), optomagnetic storage media (eg optomagnetic disks), CD-ROMs (ReadOnlyMemory), CD-Rs, It includes a CD-R / W and a semiconductor memory (for example, a mask ROM, a PROM (Programmable ROM), an EPROM (Erasable PROM), a flash ROM, and a RAM (RandomAccessMemory)). The program may also be supplied to the computer by various types of temporary computer readable media. Examples of temporary computer readable media include electrical, optical, and electromagnetic waves. The temporary computer-readable medium can supply the program to the computer via a wired communication path such as an electric wire and an optical fiber, or a wireless communication path.

Other than that, a part or all of each of the above embodiments may be described as in the following appendix, but is not limited to the following.

[Appendix 1]
An acquisition means for acquiring a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state, and
A mental tendency estimation means for estimating the tendency of the mental state of the subject based on the plurality of sets, and
Information processing device equipped with.
[Appendix 2]
The stress states are classified into multiple levels.
The information processing apparatus according to Appendix 1, wherein the mental tendency estimation means generates mental tendency information representing the tendency for each level of the stress state.
[Appendix 3]
The information processing apparatus according to Appendix 2, wherein the mental tendency estimation means generates the mental tendency information indicating the applicable mental state and the certainty of the mental state for each level of the stress state.
[Appendix 4]
The mental state is an index value represented by a plurality of axes, and is an index value.
The mental tendency estimation means calculates a representative value for each level of the stress state for each axis of the plurality of axes, and generates the mental tendency information representing the tendency regarding the axis having the largest variation in the representative value. The information processing apparatus according to Appendix 2.
[Appendix 5]
The mental tendency estimation means learns an inference model that outputs a mental state corresponding to the input of the stress state based on the plurality of sets, and obtains information about the inference model obtained by the learning. The information processing apparatus according to Appendix 1, which is generated as mental trend information indicating a tendency.
[Appendix 6]
The mental tendency estimation means estimates the tendency in the case of a stress state that has a positive effect on the subject and the tendency in the case of a stress state that adversely affects the target person, according to Supplements 1 to 5. The information processing device according to any one of the items.
[Appendix 7]
The information processing apparatus according to Appendix 6, wherein a determination flag for determining whether the positive influence or the bad influence is associated with each of the plurality of sets.
[Appendix 8]
The information processing apparatus according to any one of Supplementary note 1 to 7, wherein the acquisition means acquires chronic stress and acute stress of the subject as the stress state.
[Appendix 9]
A mental state estimation means for estimating the mental state of the subject based on the tendency and the stress state of the subject acquired after the estimation of the tendency, and
An output control means that outputs information about the estimation result by the mental state estimation means, and
The information processing apparatus according to any one of Supplementary Provisions 1 to 8, further comprising.
[Appendix 10]
The computer
Obtaining a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state,
Estimate the tendency of the subject's mental state based on the plurality of sets.
Control method.
[Appendix 11]
Obtaining a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state,
A storage medium in which a program for causing a computer to execute a process of estimating a tendency of the mental state of the subject based on the plurality of sets is stored.

Although the invention of the present application has been described above with reference to the embodiment, the invention of the present application is not limited to the above embodiment. Various changes that can be understood by those skilled in the art can be made within the scope of the present invention in terms of the configuration and details of the present invention. That is, it goes without saying that the invention of the present application includes all disclosure including claims, various modifications and modifications that can be made by those skilled in the art in accordance with the technical idea. In addition, each disclosure of the above-mentioned patent documents cited shall be incorporated into this document by citation.

1, 1A, 1X Information processing device 2 Input device 3 Output device 4 Storage device 5 Sensor 8 Terminal device 100, 100A Mental state estimation system

Claims

An acquisition means for acquiring a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state, and
A mental tendency estimation means for estimating the tendency of the mental state of the subject based on the plurality of sets, and
Information processing device equipped with.
The stress states are classified into multiple levels.
The information processing apparatus according to claim 1, wherein the mental tendency estimation means generates mental tendency information representing the tendency for each level of the stress state.
The information processing apparatus according to claim 2, wherein the mental tendency estimation means generates the mental tendency information indicating the applicable mental state and the certainty of the mental state for each level of the stress state.
The mental state is an index value represented by a plurality of axes, and is an index value.
The mental tendency estimation means calculates a representative value for each level of the stress state for each axis of the plurality of axes, and generates the mental tendency information representing the tendency regarding the axis having the largest variation in the representative value. The information processing apparatus according to claim 2.
The mental tendency estimation means learns an inference model that outputs a mental state corresponding to the input of the stress state based on the plurality of sets, and obtains information about the inference model obtained by the learning. The information processing apparatus according to claim 1, which is generated as mental trend information representing a trend.
The mental tendency estimation means estimates the tendency in the case of a stress state having a positive effect on the subject and the tendency in the case of a stress state having an adverse effect on the subject, claims 1 to 5. The information processing apparatus according to any one of the above.
The information processing apparatus according to claim 6, wherein a determination flag for determining whether the positive effect or the adverse effect is associated with each of the plurality of sets.
The information processing apparatus according to any one of claims 1 to 7, wherein the acquisition means acquires chronic stress and acute stress of the subject as the stress state.
A mental state estimation means for estimating the mental state of the subject based on the tendency and the stress state of the subject acquired after the estimation of the tendency, and
An output control means that outputs information about the estimation result by the mental state estimation means, and
The information processing apparatus according to any one of claims 1 to 8, further comprising.
The computer
Obtaining a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state,
Estimate the tendency of the subject's mental state based on the plurality of sets.
Control method.
Obtaining a plurality of pairs of the stress state of the subject and the mental state when the subject is in the stress state,
A storage medium in which a program for causing a computer to execute a process of estimating a tendency of the mental state of the subject based on the plurality of sets is stored.