JP7241318B2 - Mental state estimation system, mental state estimation method, program, estimation model generation method - Google Patents

Description

The present disclosure relates to a mental state estimation system, a mental state estimation method, a program, and an estimation model generation method. More specifically, the present disclosure relates to a psychological state estimation system, a psychological state estimation method, a program that can be used to estimate factors of a subject's psychological state, and an estimation model generation method used in this psychological state estimation system. .

Patent Literature 1 discloses a biological information display device.

The biological information display device of Patent Document 1 includes biological information detection means, detection result display means, and area display means.

The biological information detection means detects at least two types of biological information based on the subject's pulse wave signal and electrocardiographic signal. The detection result display means displays the two types of biometric information detected by the biometric information detection means, together with past detection results, in correspondence with points on the two-dimensional coordinates set on the display screen. The area display means displays, on the display screen, a plurality of areas showing the correspondence relationship between the points on the two-dimensional coordinates and the condition of the subject, superimposed on the display of the detection result by the detection result display means.

JP-A-2006-34803

The biological information display device described in Patent Literature 1 can display changes in the psychological state of a subject (subject) in real time, but factors related to changes in the psychological state are unknown.

The present disclosure provides a psychological state estimation system, a psychological state estimation method, a program, and an estimation model generation method used in this psychological state estimation system that can be used to identify factors of the psychological state of a subject. for the purpose.

A mental state estimation system according to an aspect of the present disclosure includes an acquisition unit and an estimation unit. The acquisition unit acquires information on activity levels of a plurality of sensory areas in the subject's brain. The estimating unit, when the subject is under a plurality of types of stimuli, based on the information of the activity amounts of the plurality of sensory areas acquired by the acquiring unit, a plurality of Among the types of sensations, the sensations highly relevant to the psychological state of the subject represented by the degree of comfort and the degree of arousal of the subject are estimated.

A mental state estimation method according to an aspect of the present disclosure includes an acquisition step and an estimation step. The obtaining step includes obtaining information on activity levels of a plurality of sensory areas in the subject's brain. In the estimation step, when the subject is under a plurality of types of stimuli, the plurality of types of sensations respectively corresponding to the plurality of sensory areas are evaluated based on the acquired information of the activity amounts of the plurality of sensory areas. Among these, it includes estimating a sense highly related to the subject's psychological state, which is represented by the subject's degree of comfort and degree of arousal as indices.

A program according to an aspect of the present disclosure causes one or more processors to execute the mental state estimation method.

An estimation model generation method according to an aspect of the present disclosure is an estimation model generation method used by the estimation unit of the mental state estimation system. The estimation model generation method includes an acquisition step and a model generation step. The acquiring step includes acquiring information on activity levels of multiple sensory areas in the subject's brain when the subject is under multiple types of stimuli. The model generation step uses as input data at least the information on the amount of activity of the plurality of sensory areas, and relates to the psychological state of the subject represented by the degree of comfort and the degree of arousal of the subject as indices. generating the estimation model according to the input data and the output data by machine learning using highly sensitive sensations as output data.

The present disclosure can assist in identifying factors of a subject's psychological state.

FIG. 1 is a block diagram of a control system including a mental state estimation system according to Embodiment 1. FIG. FIG. 2 is an explanatory diagram for explaining the subject's brain. FIG. 3 is an explanatory diagram for explaining attention amounts of multiple sensory areas corresponding to multiple types of sensations. FIG. 4 is a diagram showing an example of a room used when determining reference values in the mental state estimation system according to the first embodiment. FIG. 5 is a block diagram of a control system that includes a psychological state estimation system according to Embodiment 2. FIG. FIG. 6 is a two-dimensional psychological model that uses the degree of comfort and the degree of arousal as indexes. FIG. 7 is a flowchart for explaining the operation of the control system that includes the mental state estimation system according to the second embodiment. FIG. 8 is a flow chart showing a method of generating an estimation model.

(1) Embodiment 1
(1.1) Overview FIG. 1 shows a block diagram of a control system 100 including a psychological state estimation system 1 of this embodiment.

The mental state estimation system 1 of this embodiment includes an acquisition unit 101 and an estimation unit 102, as shown in FIG. The acquiring unit 101 acquires information on the amount of activity of a plurality of sensory areas 91 to 95 in the subject's brain 90 (see FIG. 2). The estimating unit 102 estimates each of the plurality of sensory fields 91 to 95 based on the information on the amount of activity of the plurality of sensory fields 91 to 95 acquired by the acquiring unit 101 when the subject is given a plurality of types of stimuli. Among the corresponding multiple types of sensations, the sensation that is highly relevant to the subject's psychological state is estimated.

According to the psychological state estimation system 1 of the present embodiment, based on the information of the activity amount of the plurality of sensory areas 91 to 95, among the plurality of types of sensations, the sensation highly related to the subject's psychological state is estimated. do. Therefore, the psychological state estimation system 1 can be used to identify factors of the subject's psychological state.

(1.2) Details (1.2.1) Configuration Hereinafter, the control system 100 including the psychological state estimation system 1 of the present embodiment will be described in more detail with reference to the drawings.

The control system 100 includes a measurement unit 2 , a control device 3 , and a stimulus application device 4 in addition to the psychological state estimation system 1 .

The measurement unit 2 measures the biological information of the subject. The measurement unit 2 is, for example, a wearable terminal that can be worn on the subject's body.

The measurement unit 2 includes a brain activity amount measurement unit 21 that measures the activity amount of the subject's brain 90 . The brain activity measuring unit 21 non-invasively measures the amount of activity in a plurality of regions of the subject's brain 90 by being worn on the subject's head, for example. The brain activity measuring unit 21 here is a NIRS brain measuring device using NIRS (Near Infra-Red Spectroscopy).

The NIRS brain measurement device acquires an image of the cerebral cortex portion of the brain 90 by irradiating near-infrared light with a wavelength near 800 nm from the subject's scalp and measuring the transmitted light. It is known that hemoglobin increases in areas of the brain 90 where activity is brisk (high activity level). Near-infrared light with a wavelength of about 800 nm, which is used in the NIRS brain measurement device, passes through human tissues such as the scalp and skull, but is absorbed by hemoglobin in blood. Therefore, in the image of the brain 90 acquired by the NIRS brain measurement device, the amount of activity in each region of the brain 90 is measured by observing the intensity change of the transmitted light of near-infrared light and observing the increase or decrease of hemoglobin. It is possible to

The measurement unit 2 outputs the measurement result to the psychological state estimation system 1 through wired communication or wireless communication.

The mental state estimation system 1 includes a processing unit 10 , an input unit 11 , an operation unit 12 , an output unit 13 and a storage unit 14 .

The input unit 11 is a communication interface that communicates with the measurement unit 2 by wire or wirelessly. The input unit 11 receives measurement results from the measurement unit 2 (brain activity amount measurement unit 21).

The processing unit 10 includes an acquisition unit 101 , an estimation unit 102 and a reference value setting unit 103 . The processing unit 10 comprises, for example, a microcomputer having a processor and memory. The computer system functions as the processing unit 10 by the processor executing appropriate programs. The functions of the processing unit 10 (acquisition unit 101, estimation unit 102, and reference value setting unit 103) are realized by a computer system having a processor and memory. The program may be prerecorded in a memory, or may be provided by being recorded in a non-temporary recording medium such as a memory card or through an electric communication line such as the Internet.

The acquisition unit 101 of the processing unit 10 acquires information on the amount of activity of multiple sensory areas in the brain 90 of the subject based on the measurement results received from the brain activity amount measurement unit 21 . Here, the acquisition unit 101 of the processing unit 10 acquires information on activity amounts of a plurality of primary sensory areas in the cerebral cortex of the brain 90 of the subject.

As shown in FIG. 2, the cerebral cortex of brain 90 includes multiple sensory areas (primary sensory areas) 91-95. A plurality of sensory areas (primary sensory areas) 91 - 95 includes a visual area 91 , an auditory area 92 , an olfactory area 93 , a somatosensory area 94 and a gustatory area 95 . The visual cortex 91 is the area of the cerebral cortex related to vision and is located in the occipital lobe. The auditory cortex 92 is the region of the cerebral cortex related to hearing and is located in the temporal lobe. The olfactory cortex 93 is the region of the cerebral cortex related to smell and is located in the frontal lobe. The somatosensory cortex 94 is the region of the cerebral cortex associated with somatosensory sensation and is located in the parietal lobe. The gustatory area 95 is the region of the cerebral cortex associated with taste and is located between the somatosensory and olfactory areas.

Acquisition unit 101 acquires sensory areas related to the five senses (visual area 91, auditory area 92, olfactory area 93, somatosensory area 94, and Information on the amount of activity of each of the gustatory areas 95) is acquired.

The amount of activity in the sensory areas 91-95 of the subject's brain 90 depends on how much the subject's attention is directed to the stimulus when it is presented to the subject. The control system 100 includes a stimulus applying device 4 as a device capable of applying a stimulus to the subject. The stimulus applying device 4 can, for example, provide the subject with multiple types of stimuli corresponding to multiple types of sensations.

The stimulus applying device 4 includes, for example, a lighting device, a display device, etc. as a device capable of providing visual stimulus to the subject. The lighting device can be adjusted, for example, in brightness, light color, color temperature, brightness, and the like. The display device can, for example, display an appropriate image on the display.

The stimulus applying device 4 includes, for example, a sounding device as a device capable of giving a stimulus to the auditory sense of the subject. The sound generator includes a speaker, and can output appropriate voice, music, environmental sound, etc. from the speaker.

The stimulus application device 4 includes, for example, an odor generator or the like as a device capable of providing a stimulus to the subject's olfactory sense. The odor generating device is, for example, an aroma diffuser, and can generate a chemical substance that is the source of an appropriate odor (aroma) that can be sensed with the sense of smell.

The stimulation device 4 includes, for example, an air blower, an air conditioner, etc., as a device capable of stimulating the subject's somatosensory sensation. Air blowers and air conditioners can adjust, for example, wind direction, wind volume, and the like.

The stimulus applying device 4 includes, for example, a taste stimulator as a device capable of stimulating the subject's sense of taste. The taste stimulator can stimulate the subject's sense of taste, for example, by applying an electrode to the subject's tongue and passing an electric current through the electrode.

The operation of the stimulation device 4 is controlled by the control device 3 . The control system 100 includes, in addition to the stimulation device 4, a device capable of reducing stimulation to the subject (for example, an air purifier capable of reducing olfactory stimulation by removing chemical substances in the space). good too.

Here, the processing unit 10 has a first mode and a second mode as operation modes. The operation mode of the processing unit 10 is switched according to an input operation to the operation unit 12, for example. The operation unit 12 includes, for example, push buttons for switching operation modes of the processing unit 10 .

The first mode is a mode in which the reference value setting section 103 of the processing section 10 operates. The first mode is executed, for example, in a state where the stimulus applying device 4 does not apply any stimulus to the subject. In the first mode, the processing unit 10 uses the information of the activity amounts of the plurality of sensory fields 91 to 95 acquired by the acquisition unit 101 as a reference (baseline) for the activity amounts of the plurality of sensory fields 91 to 95. Define a plurality of reference values to be used respectively. The plurality of reference values correspond one-to-one to the activity amounts of the plurality of sensory fields 91-95.

In the first mode, the processing unit 10 (reference value setting unit 103) sets, for each of the plurality of sensory fields 91 to 95, the representative value of the amount of activity acquired by the acquiring unit 101 while operating in the first mode. Ask for The reference value setting unit 103 obtains an average value of activity amounts of the corresponding sensory areas as a representative value of activity amounts for each of the plurality of sensory areas 91 to 95 . The reference value setting unit 103 determines the representative value (here, the average value) of the calculated activity amounts as the reference value. However, not limited to this, the reference value setting unit 103 may obtain a minimum value, an intermediate value, a maximum value, or the like of the activity amount of the corresponding sensory field as the representative value of the activity amount.

That is, the reference value setting unit 103 sets a plurality of reference values for each activity amount of a plurality of sensory areas 91 to 95 (visual area 91, auditory area 92, olfactory area 93, somatosensory area 94, and gustatory area 95). stipulate. The reference value setting unit 103 sets the visual cortex reference value for the activity amount of the visual cortex 91, the auditory cortex reference value for the activity amount of the auditory cortex 92, the olfactory cortex reference value for the activity amount of the olfactory cortex 93, and the activity of the somatosensory cortex 94. A somatosensory area reference value for quantity and a gustatory area reference value for activity amount of gustatory area 95 are established.

The reference value setting unit 103 causes the storage unit (reference value storage unit) 14 to store the plurality of determined reference values. That is, the storage unit (reference value storage unit) 14 stores a plurality of reference values corresponding to the activity amounts of the plurality of sensory fields 91 to 95 on a one-to-one basis.

The storage unit 14 is a semiconductor memory such as RAM (Random Access Memory) or EEPROM (Electrically Erasable Programmable Read Only Memory). Note that the storage unit 14 is not limited to a semiconductor memory, and may be a hard disk drive or the like.

A second mode is a mode in which the estimation unit 102 of the processing unit 10 operates. The second mode is executed, for example, while the subject is being given at least one type of stimulus from the stimulus applying device 4 . In the second mode, the processing unit 10, based on the information on the activity amounts of the plurality of sensory fields 91 to 95 acquired by the acquisition unit 101, to estimate the senses that are highly relevant to the subject's psychological state.

In the second mode, the processing unit 10 (estimating unit 102) obtains the amount of attention for each of the plurality of sensory fields 91-95. The amount of attention is an amount that indicates how much the subject pays attention to the stimulation of multiple types of sensations corresponding to the multiple sensory fields 91-95. The amount of attention is defined here as the difference between the amount of activity acquired by the acquisition unit 101 when operating in the second mode and the reference value stored in the reference value storage unit 14 .

The estimation unit 102 obtains a plurality of attention amounts for each of the plurality of sensory fields 91 to 95, that is, a plurality of attention amounts for each of a plurality of types of stimuli. The estimation unit 102 calculates the amount of attention to visual stimulation (visual area attention), attention amount to auditory stimulation (auditory area attention amount), attention amount to olfactory stimulation (olfactory area attention amount), attention amount to somatosensory stimulation (body genital area attention) and attention to gustatory stimuli (gustatory area attention).

The amount of attention for each sensory area is the representative value of the amount of activity obtained by the obtaining unit 101 when the processing unit 10 is operating in the first mode, and the representative value of the amount of activity when the processing unit 10 is operating in the second mode. and the amount of activity acquired by the acquisition unit 101 . Therefore, if the amount of attention obtained for a certain sensory area is large, it means that this sensory area is more active than when the reference value was obtained. means directed to a stimulus that

The estimating unit 102 estimates that, among the plurality of types of sensations corresponding to the plurality of sensory fields 91 to 95, the sensation that requires the greatest amount of attention is the sensation that is highly relevant to the psychological state of the subject. For example, as shown in FIG. 3, when the attention amount for smell (olfactory attention amount) is the largest among visual, auditory, olfactory, somatosensory, and gustatory, the estimation unit 102 It is presumed that of the five senses, the sense of smell is highly relevant to the subject's psychological state.

The output unit 13 is a communication interface that communicates with the control device 3 by wire or wirelessly. The mental state estimation system 1 outputs the estimation result obtained by the estimation unit 102 to the control device 3 via the output unit 13 . The psychological state estimation system 1 also outputs the amount of attention for each of the plurality of sensory fields 91 to 95 to the control device 3 via the output unit 13 .

The control device 3 controls the stimulus applying device 4 based on the estimation result output from the output unit 13 of the psychological state estimation system 1 and the amount of attention for each of the plurality of sensory areas 91-95. The control device 3 controls the stimulus applying device 4 so that, for example, the amount of attention to the sense highly related to the subject's psychological state estimated by the psychological state estimation system 1 is changed or maintained. The control device 3 may control the stimulus applying device 4 so as to increase or decrease the amount of attention to the sense highly related to the psychological state of the subject estimated by the psychological state estimation system 1 .

(1.2) Usage Example Next, a usage example of the control system 100 of the present embodiment will be described.

Here, it is assumed that the user of the control system 100, who is the subject, wants to avoid distractions from the surrounding environment by the control system 100 as much as possible for the purpose of devoting himself/herself to his work.

First, the user operates the control system 100 in the first mode in advance to store the reference value in the storage unit 14 (reference value storage unit) 14 . A reference value is determined under the condition that no special stimulus is given to the subject. For example, in a room 200 in which the control system 100 is installed, as shown in FIG. do not operate. Under this circumstance, the amount of activity in the sensory areas 91-95 of the user's brain 90 is measured using the control system 100, and a reference value is established based on the measured amount of activity. That is, the reference value setting unit 103 of the psychological state estimation system 1 sets the reference value based on the activity amounts of the plurality of sensory areas 91 to 95 acquired by the acquisition unit 101 when the subject is not stimulated. stipulate.

Next, the user operates the mental state estimation system 1 of the control system 100 in the second mode in the room 200 where the control system 100 is installed, and performs work. The mental state estimation system 1 obtains the amount of activity and the amount of attention for each of the sensory areas 91 to 95 of the brain 90 obtained from the measurement unit 2 .

Under this circumstance, for example, it is assumed that the attention amount of the sensory area with the largest attention amount among the plurality of sensory areas 91 to 95 exceeds a predetermined threshold (for example, see the "olfactory" graph in FIG. 3). . This means that the user's attention is directed to the stimulus (smell) corresponding to this sensory field (olfactory field 93). That is, the psychological state estimation system 1 estimates that the user's (current) attention is (abnormally) focused on the olfactory stimulus, based on the fact that the amount of attention for the olfactory area 93 exceeds a predetermined threshold. .

Therefore, the control device 3 operates, for example, an air purifier to remove chemical substances in the air that cause odors so that the user's attention is not directed to the odors. Alternatively, the control device 3 may generate an appropriate scent (for example, the scent of roses, lavender, etc. for relaxing the user) by the scent generating device so that the user's attention is not directed to the scent. .

Similarly, for example, when the amount of attention related to the visual cortex 91 exceeds a predetermined threshold (when the user's attention is focused on information from the eyes), the control device 3 controls the light intensity of the lighting device in the stimulus applying device 4 , light color, brightness, color temperature, etc. may be changed.

As described above, according to the psychological state estimation system 1 of the present embodiment, it is possible to estimate the stimulus to which the subject's attention is directed (that is, the sensation highly relevant to the subject's current psychological state). Become. As a result, the psychological state estimation system 1 can be used to identify factors of the subject's psychological state.

Further, according to the control system 100 including the psychological state estimation system 1 of the present embodiment, the subject is stimulated with a sensation that is highly relevant to the subject's psychological state, which is estimated by the psychological state estimation system 1. be able to give.

(2) Embodiment 2
(2.1) Configuration FIG. 5 shows a block diagram of a control system 100 including the mental state estimation system 1 according to this embodiment. Components similar to those of the control system 100 of the first embodiment are denoted by the same reference numerals, and descriptions thereof are omitted as appropriate.

The measurement unit 2 of this embodiment includes a second measurement unit 22 in addition to the first measurement unit 21 corresponding to the brain activity measurement unit 21 . While the first measurement unit 21 measures the physiological response of the subject's central nervous system by measuring the amount of activity in the subject's brain 90, the second measurement unit 22 measures the subject's autonomic nervous system to measure the physiological response of The second measuring unit 22 measures, for example, the subject's heartbeat, blood pressure, respiratory rate, skin temperature, or electrodermal activity (including skin potential level).

In addition to the first estimating unit 102 corresponding to the estimating unit 102 of the first embodiment, the psychological state estimation system 1 of the present embodiment includes a second estimating unit 104 .

The second estimating unit 104 estimates the subject's psychological state based on the measurement result of the first measurement unit 21 and the measurement result of the second measurement unit 22 (hereinafter also referred to as "comfort level"). ) and the degree of arousal (hereinafter also referred to as “awakening degree”) are estimated as indices.

The degree of comfort is an index that indicates the level of comfort. Arousal is an index indicating the level of arousal. For example, as shown in FIG. 6, the psychological state can be represented by a two-dimensional psychological model (for example, Russell's annulus model) using comfort level and arousal level as indices. In this two-dimensional psychological model, the X-axis indicates the degree of comfort, and the Y-axis indicates the degree of arousal.

As for the degree of comfort on the X-axis, the positive area on the X-axis is "comfortable" and the negative area on the X-axis is "unpleasant." Comfort increases as the level (absolute value) in the positive region of the X axis increases, and discomfort increases (comfort decreases) as the level (absolute value) in the negative region of the X axis increases. do.

Regarding the degree of arousal on the Y-axis, the positive area on the Y-axis is "awakened" and the negative area on the Y-axis is "sedated." Arousal increases as the level (absolute value) in the positive region of the Y axis increases, and sedation increases as the level (absolute value) in the negative region of the Y axis increases (arousal decreases). do.

Mental states are classified into types according to which quadrant of the two-dimensional coordinates of the two-dimensional mental model the comfort level and the arousal level are located. The first quadrant Z1 relates to the so-called refresh state. If it is the first quadrant Z1, it indicates that the psychological state is, for example, surprise, excitement, happiness, joy, happiness, or the like. The second quadrant Z2 relates to the so-called irritable state. The second quadrant Z2 indicates that the psychological state is, for example, fear, worry, anger, dissatisfaction, unpleasantness, or the like. The third quadrant Z3 relates to so-called boredom. If it is the third quadrant Z3, it indicates that the psychological state is, for example, sadness, depression, boredom, slackness, or the like. The fourth quadrant Z4 relates to the so-called refresh state. If it is the fourth quadrant Z4, it indicates that the psychological state is satisfied, comfortable, calm, relaxed, or the like.

The second estimation unit 104 estimates the psychological state of the subject by obtaining the comfort level and the arousal level based on the measurement results of the measurement unit 2 (the first measurement unit 21 and the second measurement unit 22). The second estimation unit 104 estimates the psychological state of the subject by obtaining the position of the psychological state of the subject on the two-dimensional coordinates of the two-dimensional psychological model.

The processing unit 10 outputs the estimation result obtained by the first estimation unit 102 and the estimation result obtained by the second estimation unit 104 to the control device 3 via the output unit 13 .

The control device 3 controls the stimulus applying device 4 based on the estimation result output from the output unit 13 of the psychological state estimation system 1 and the amount of attention for each of the plurality of sensory areas 91-95. For example, when the current psychological state of the subject estimated by the second estimating unit 104 is not the state desired by the subject, the control device 3 provides stimulation so that the subject's mental state becomes the desired state. to control the device 4; At that time, the control device 3 determines that the factor that causes the subject's current psychological state is the stimulus to the sensation that is highly relevant to the subject's psychological state, which is estimated by the first estimation unit 102. It estimates and controls the stimulus applying device 4 to change the stimulus for this sensation.

(2.2) Usage Example Next, a usage example of the control system 100 of the present embodiment will be described with reference to the flowchart of FIG.

In particular, it is assumed here that the user, who is the subject, uses the control system 100 for the purpose of relaxing.

As in the case of the first embodiment, first, the user causes the control system 100 to operate in the first mode in advance to store the reference value in the storage section (reference value storage section) 14 .

Next, the user operates the control system 100 and operates the operation unit 12 to input a desired state of mind (a desired state of mind that the user wants to achieve). Here, since the user intends to relax, he or she inputs a relaxed state as a desired psychological state.

In order to relax the user, the control system 100 lights the lighting device included in the stimulus applying device 4 in light bulb color so as to reproduce the atmosphere of a so-called bar or lounge. In addition, the control system 100 causes the aroma diffuser included in the stimulation applying device 4 to generate the scent of roses, which is generally considered to induce a relaxing state.

The second estimation unit 104 of the psychological state estimation system 1 estimates the user's psychological state based on the measurement result of the measurement unit 2 (S1). The mental state estimation system 1 outputs the estimation result to the control device 3 . The control device 3 determines whether or not the estimation result of the second estimation unit 104 is in the quadrant (one of the first quadrant Z1 to the fourth quadrant Z4) corresponding to the desired psychological state in the two-dimensional psychological model. (S2). In this case, the control device 3 determines whether or not the estimation result (second estimation result) of the second estimation unit 104 is in the fourth quadrant Z4 corresponding to the relaxed state.

If the estimation result of the second estimation unit 104 is in the quadrant corresponding to the desired mental state in the two-dimensional psychological model (S2: Yes), the control device 3 determines that the user is in the desired mental state. , the current state of the stimulation device 4 is maintained (S3).

If the estimation result of the second estimation unit 104 is not in the quadrant corresponding to the desired psychological state in the two-dimensional psychological model (S2: No), the control device 3 outputs the estimation result of the first estimation unit 102 (first estimation result ) (S4), and acquires information on a sense highly related to the psychological state of the user among a plurality of types of senses (five senses). Then, based on the estimation result of the first estimation unit 102 and the estimation result of the second estimation unit 104, the control device 3 identifies the factor that the user's state of mind is not in the desired state. Specifically, the control device 3 determines that the feeling highly related to the user's psychological state estimated by the first estimation unit 102 is a factor (a factor feeling) that the user's psychological state is not in a desired state. ) (S5). For example, as shown in FIG. 3, when the estimation result of the second estimation unit 104 indicates that the amount of attention to the sense of smell is the largest, the control device 3 determines that the psychological state of the user is not as desired due to the smell. Assume that it is not in a state.

Then, the control device 3 changes the operation of the device (related device) among the stimulus applying devices 4 that can give the stimulus for the sensation estimated by the first estimating section 102 (S6). For example, the user may dislike the scent of roses. Therefore, the control device 3 stops the operation of the aroma diffuser included in the stimulation applying device 4, or causes the aroma diffuser to emit a scent such as lavender that is different from that of roses.

After changing the operation of the stimulus applying device 4, the control device 3 acquires the estimation result of the second estimation unit 104, estimates the user's psychological state (S1), and determines that the user's psychological state is the desired state ( It is determined whether or not the user is in a relaxed state (S2). The control device 3 repeats the above operations until the psychological state of the user reaches the desired state.

Thus, according to the psychological state estimation system 1 of the present embodiment, the second estimating unit 104 estimates the psychological state of the subject, and the first estimating unit 102 detects a feeling highly related to the subject's psychological state. I'm guessing. As a result, it is possible to combine and estimate the subject's current psychological state and the sensations that are factors that cause the psychological state.

In addition, the control system 100 uses the estimation result of the first estimation unit 102 and the estimation result of the second estimation unit 104 to guide the psychological state of the subject to a desired state.

Note that even if the estimation result of the second estimation unit 104 is in the quadrant corresponding to the desired psychological state in the two-dimensional psychological model (S2: Yes), the control device 3 You may refer to a result (1st estimation result). Then, the control device 3 controls the operation of the device capable of providing a stimulus to a sensation highly related to the psychological state of the user so that the user's psychological state becomes a more desirable state (for example, when the psychological state is in the fourth quadrant). The stimulation device 4 may be operated such that it is more comfortable when in Z4).

(3) Estimation Model Generation Method The estimation model used in the estimation unit (first estimation unit) 102 of the mental state estimation system 1 can be generated by the generation method shown in FIG. That is, an estimation model generation method according to an aspect of the present disclosure includes an acquisition step (S101) and a model generation step (S102). The acquiring step acquires information on the amount of activity of multiple sensory areas (at least two of the sensory areas 91 to 95) in the subject's brain 90 when multiple types of stimuli are applied to the subject. include. In the model generation step, information on the amount of activity of at least a plurality of sensory fields (at least two of the sensory fields 91 to 95) is used as input data, and a sense highly related to the psychological state of the subject is used as output data. It includes using machine learning to generate an estimation model according to input data and output data.

(4) Modifications The above-described embodiments are only some of the various embodiments of the present disclosure. The above-described embodiments can be modified in various ways according to design and the like as long as the object of the present disclosure can be achieved. The modifications described below can be applied in appropriate combination with the above-described embodiments. A function similar to that of the mental state estimation system 1 may be embodied by a mental state estimation method, a (computer) program, a non-temporary recording medium recording the program, or the like.

A mental state estimation method according to one aspect includes an acquisition step and an estimation step. The obtaining step obtains information on the amount of activity of each of the plurality of sensory areas 91 to 95 in the brain 90 of the subject. The estimating step is based on the information of the amount of activity of the plurality of sensory fields 91 to 95 acquired when the subject is given a plurality of types of stimuli, and calculates a plurality of types of information corresponding to the plurality of sensory fields 91 to 95, respectively. Among the senses, the senses that are highly relevant to the subject's psychological state are estimated.

A program according to one aspect is a program for causing one or more processors to execute the mental state estimation method described above.

The mental state estimation system 1 according to the present disclosure includes a computer system in, for example, the processing unit 10 and the like. A computer system is mainly composed of a processor and a memory as hardware. The functions of the mental state estimation system 1 in the present disclosure are realized by the processor executing a program recorded in the memory of the computer system. The program may be recorded in advance in the memory of the computer system, may be provided through an electric communication line, or may be recorded in a non-temporary recording medium such as a computer system-readable memory card, optical disk, or hard disk drive. may be provided. A processor in a computer system consists of one or more electronic circuits, including semiconductor integrated circuits (ICs) or large scale integrated circuits (LSIs). The integrated circuit such as IC or LSI referred to here is called differently depending on the degree of integration, and includes integrated circuits called system LSI, VLSI (Very Large Scale Integration), or ULSI (Ultra Large Scale Integration). In addition, a field-programmable gate array (FPGA) that is programmed after the LSI is manufactured, or a logic device capable of reconfiguring the bonding relationship inside the LSI or reconfiguring the circuit partitions inside the LSI may also be adopted as the processor. can be done. A plurality of electronic circuits may be integrated into one chip, or may be distributed over a plurality of chips. A plurality of chips may be integrated in one device, or may be distributed in a plurality of devices. A computer system, as used herein, includes a microcontroller having one or more processors and one or more memories. Accordingly, the microcontroller also consists of one or more electronic circuits including semiconductor integrated circuits or large scale integrated circuits.

In addition, it is not an essential configuration of the mental state estimation system 1 that a plurality of functions in the mental state estimation system 1 are integrated in one housing. The constituent elements of the mental state estimation system 1 may be distributed over a plurality of housings. Furthermore, at least part of the functions of the mental state estimation system 1 may be realized by, for example, a server device and a cloud (cloud computing).

The mental state estimation system 1 may be used together with the measurement unit 2 alone. In other words, the information about the sensation highly related to the psychological state of the subject estimated by the psychological state estimation system 1 may be used for purposes other than the control of the stimulus applying device 4 by the control device 3 . For example, by collecting information on senses highly related to the subject's psychological state obtained by the psychological state estimation system 1 for a plurality of people, it is possible to determine the influence of these multiple senses on the psychological state of the plurality of people. is possible to estimate strong sensations.

The multiple types of sensation candidates that the psychological state estimation system 1 estimates as sensations highly related to the subject's psychological state need not include all of the five senses. In other words, the plurality of types of senses should include at least two of the five senses (visual, auditory, olfactory, somatosensory, and gustatory). The multiple senses preferably include at least sight, hearing, and smell.

The sensory area from which information on the amount of activity is acquired by the acquisition unit 101 is not limited to the primary sensory area. The sensory area from which information on the amount of activity is acquired by the acquisition unit 101 may include a higher-order sensory area such as a secondary sensory area or a tertiary sensory area instead of or in addition to the primary sensory area. .

The acquiring unit 101 may divide a brain region corresponding to one sensory area out of the five senses into a plurality of regions, and acquire activity amount information for each of the plurality of regions. For example, the acquisition unit 101 separately acquires information on the amount of activity in a region of the visual cortex 91 that responds to movement of an object and information on the amount of activity in a region of the visual cortex 91 that responds to color. may

The reference values stored in the storage unit (reference value storage unit) 14 are determined based on the activity amounts of the plurality of sensory fields 91 to 95 acquired by the acquisition unit 101 when the subject is not stimulated. It doesn't have to be anything. For example, the reference value may be a default value stored in the storage unit 14 in advance.

The mental state estimation system 1 does not have to include the reference value storage unit 14 . For example, the estimation unit (first estimation unit) 102 may use the amount of activity acquired by the acquisition unit 101 when operating in the second mode as the amount of attention. In this case, the estimating unit (first estimating unit) 102 selects the sense with the largest amount of attention (that is, the amount of activity) of the corresponding sensory area among the plurality of types of senses, and determines the degree of relevance to the psychological state of the subject. Assume high sensation.

The estimating unit (first estimating unit) 102 selects the sense with the greatest amount of attention from among the plurality of types of senses corresponding to the plurality of sensory fields 91 to 95 as the sense highly related to the psychological state of the subject. It is not limited to the configuration presumed to exist. The estimating unit (first estimating unit) 102, for example, compares the attention amount of each of the plurality of sensory areas 91 to 95 with a threshold, and compares the sensation corresponding to the sensory area whose attention amount is greater than the threshold with the psychological state of the subject. It may be presumed that it is a highly relevant sensation. In this case, two or more senses can be estimated to be senses highly relevant to the psychological state of the subject. The thresholds may be different from each other for the amount of attention of multiple sensory areas 91-95.

The reference value may be obtained based on information on the amount of activity of the plurality of sensory areas 91 to 95 measured when the subject lives his or her daily life.

The brain activity measurement unit (first measurement unit) 21 is not limited to the NIRS brain measurement device, and may be an MRI (Magnetic Resonance Imaging) device, an electroencephalograph that measures electroencephalograms (γ waves, etc.), or the like.

Existing equipment may be used as the stimulus applying device 4 . For example, the control system 100 may use, as the stimulation device 4, a lighting device, an air conditioner, or the like that is installed in the building in advance.

(5) Summary The following aspects are disclosed from the above-described embodiments, modifications, and the like.

A mental state estimation system (1) according to a first aspect includes an acquisition unit (101) and an estimation unit (102). An acquisition unit (101) acquires information on activity levels of a plurality of sensory areas (91 to 95) in a subject's brain (90). An estimating unit (102) estimates a plurality of sensations based on the information on the amount of activity of the plurality of sensory areas (91 to 95) acquired by the acquisition unit (101) when a plurality of types of stimuli are applied to the subject. Among the multiple types of sensations corresponding to the fields (91-95), the sensations highly relevant to the psychological state of the subject are estimated.

According to this aspect, the psychological state estimation system (1) can be used to identify factors of the psychological state of the subject.

In the psychological state estimation system (1) according to the second aspect, in the first aspect, the estimating unit (102) selects the sensation with the largest activity amount of the corresponding sensory area among the plurality of kinds of sensations as the object. It is estimated that it is a sense highly related to the psychological state of the person.

According to this aspect, the psychological state estimation system (1) can be used to identify factors of the psychological state of the subject.

A mental state estimation system (1) according to a third aspect, in the first aspect, further includes a reference value storage unit (14). A reference value storage unit (14) stores a plurality of reference values corresponding to activity amounts of a plurality of sensory areas (91 to 95) on a one-to-one basis. The estimating unit (102) regards the sensation having the largest difference between the activity amount of the corresponding sensory area and the corresponding reference value among the plurality of kinds of sensations as the sensation highly related to the psychological state of the subject. presume.

According to this aspect, based on the reference value, it is possible to estimate a feeling highly related to the subject's psychological state.

A mental state estimation system (1) according to a fourth aspect, in the third aspect, further includes a reference value setting unit (103). A reference value setting unit (103) sets a plurality of reference values based on the amounts of activity of the plurality of sensory areas (91 to 95) acquired by the acquisition unit (101) when the subject is not stimulated. stipulate.

According to this aspect, based on the reference value determined based on the amount of activity of the sensory areas (91 to 95) when the subject is not stimulated, the subject's psychological state and the sense of high relevance can be estimated.

In the mental state estimation system (1) according to the fifth aspect, in any one of the first to fourth aspects, the estimation unit (102) is the first estimation unit. The mental state estimation system (1) further comprises a second estimation unit (104). A second estimation unit (104) estimates the psychological state of the subject using the degree of comfort and the degree of arousal of the subject as indices.

According to this aspect, it is possible for the first estimation unit (102) to estimate the feeling highly related to the psychological state of the subject estimated by the second estimation unit (104).

In the psychological state estimation system (1) according to the sixth aspect, in any one of the first to fifth aspects, the plurality of types of senses are at least two of sight, hearing, smell, somatosensory, and taste. including one.

According to this aspect, the psychological state estimation system (1) can be used to identify factors of the psychological state of the subject.

A mental state estimation method according to a seventh aspect includes an acquisition step and an estimation step. The acquisition step includes acquiring information on the amount of activity in each of the plurality of sensory areas (91-95) in the subject's brain. The estimation step corresponds to each of the plurality of sensory fields (91 to 95) based on the information on the amount of activity of the plurality of sensory fields (91 to 95) obtained when the subject is given a plurality of types of stimuli. It includes estimating the senses that are highly relevant to the subject's psychological state among the multiple types of senses.

According to this aspect, the psychological state estimation method can be used to identify the factor of the psychological state of the subject.

A program according to an eighth aspect causes one or more processors to execute the mental state estimation method according to the seventh aspect.

According to this aspect, the program for executing the mental state estimation method can be used to identify the factors of the subject's mental state.

The estimation model generation method according to the ninth aspect is the estimation model generation method used in the estimation unit (102) of the mental state estimation system (1) according to any one of the first to sixth aspects. The estimation model generation method includes an acquisition step and a model generation step. The acquiring step includes acquiring information on the amount of activity of multiple sensory areas (91-95) in the subject's brain (90) when multiple types of stimuli are applied to the subject. In the model generation step, input data and output data are obtained by machine learning using as input data information on the amount of activity of at least a plurality of sensory areas (91 to 95) and using as output data senses highly related to the psychological state of the subject. and generating an estimation model according to

1 psychological state estimation system 101 acquisition unit 102 estimation unit (first estimation unit)
103 reference value setting unit 104 second estimation unit 14 reference value storage unit 90 brain 91 to 95 sensory field

Claims (9)

an acquisition unit that acquires information on the amount of activity in a plurality of sensory areas in the subject's brain;
Among the plurality of types of sensations respectively corresponding to the plurality of sensory areas, based on the information on the activity amounts of the plurality of sensory areas acquired by the acquisition unit when the subject is under a plurality of types of stimulation an estimating unit for estimating a feeling highly related to the psychological state of the subject represented by the degree of comfort and the degree of arousal of the subject as indices;
comprising
Psychological state estimation system. The estimating unit estimates that, among the plurality of types of sensations, the sensation with the largest amount of activity in the corresponding sensory field is the sensation that is highly relevant to the psychological state of the subject.
The mental state estimation system according to claim 1. The psychological state estimation system further comprises a reference value storage unit that stores a plurality of reference values corresponding to the activity amounts of the plurality of sensory areas one-to-one,
The estimating unit regards, among the plurality of types of sensations, the sensation having the largest difference between the activity amount of the corresponding sensory field and the corresponding reference value as the sensation highly relevant to the psychological state of the subject. presume that
The mental state estimation system according to claim 1. The mental state estimation system includes:
A reference value setting unit that determines the plurality of reference values based on the activity amounts of the plurality of sensory areas acquired by the acquisition unit when the subject is not provided with the stimulus,
The mental state estimation system according to claim 3. The estimating unit is a first estimating unit,
The mental state estimation system includes:
A second estimating unit that estimates the psychological state of the subject using the degree of comfort and the degree of arousal of the subject as indices,
The mental state estimation system according to any one of claims 1 to 4. wherein said plurality of senses includes at least two of visual, auditory, olfactory, somatosensory and gustatory;
The mental state estimation system according to any one of claims 1 to 5. an acquisition step of acquiring information on the amount of activity in a plurality of sensory areas in the subject's brain;
When the subject is under a plurality of types of stimuli, based on the acquired information of the activity amounts of the plurality of sensory areas, the subject selects among the plurality of types of sensations respectively corresponding to the plurality of sensory areas , an estimation step of estimating a sensation that is highly relevant to the subject's psychological state, which is represented by the degree of comfort and the degree of arousal of the subject as indices;
including,
Psychological state estimation method. to one or more processors;
Execute the mental state estimation method according to claim 7,
program. A method for generating an estimation model used in the estimation unit of the psychological state estimation system according to any one of claims 1 to 6,
an acquiring step of acquiring information on activity levels of multiple sensory areas in the subject's brain when the subject is under multiple types of stimuli;
Information on the amount of activity of at least the plurality of sensory areas is used as input data, and a sensation highly related to the subject's psychological state, which is represented by indices of the degree of comfort and the degree of arousal of the subject, is output. a model generation step of generating the estimation model according to the input data and the output data by machine learning using data;
including,
How to generate the estimation model .

Images