JP7048709B2 - System and method - Google Patents

Description

Embodiments of the present invention relate to a technique for estimating mental stress.

In recent years, a technique of measuring a heartbeat or a pulse using a wearable type biosensor device and analyzing a fluctuation of the heartbeat or the pulse to estimate an autonomic nerve index has been used. Further, for estimating mental stress, there are a method of constantly wearing a sensor and accumulating and analyzing data, and a method of constantly analyzing data.

Japanese Unexamined Patent Publication No. 2011-123579

However, such an estimation has a problem that the degree of mental stress cannot be estimated with high accuracy.

One aspect of the present invention is to provide a system, a method and a program capable of estimating the degree of mental stress with high accuracy.

According to embodiments, the system comprises means of determination. The determination means uses the first short- term stress value of the user before starting the first work and the second short- term stress value of the user during the first work, and after finishing the first work, said. The magnitude of the long- term stress of the user is determined according to the change in the stress value from the second short-term stress value to the first short-term stress value .

The conceptual diagram for demonstrating the system which concerns on embodiment. The block diagram which shows the example of the structure of the system of the same embodiment. The block diagram which shows another example of the configuration of the system of the same embodiment. FIG. 6 is a block diagram showing still another example of the configuration of the system of the same embodiment. The block diagram which shows the system configuration example of the server provided in the system of the same embodiment. The block diagram which shows the system configuration example of the device (sensing device) provided in the system of the same embodiment. The block diagram which shows the functional configuration example of the server of FIG. 5 and the device of FIG. The perspective view which shows the example of the appearance of the vehicle-mounted device (vehicle-mounted device) provided in the system of the same embodiment. The block diagram which shows the functional structure of the device of FIG. The figure for demonstrating the stress value estimated using the device of FIG. The figure which shows the example of the fluctuation of the stress value of FIG. The figure which shows another example of the fluctuation of the stress value of FIG. The figure which shows the result of salivary amylase and STAI corresponding to the stress value of FIG. 10 and FIG. The perspective view which shows the example of the appearance of the portable device provided in the system of the same embodiment. The perspective view which shows the example of the appearance of the wearable device provided in the system of the same embodiment. FIG. 4 is a block diagram showing a functional configuration of the devices of FIGS. 14 and 15. The perspective view which shows the example of the appearance of the notebook PC which is a stationary device provided in the system of the same embodiment. The block diagram which shows the functional structure of the notebook PC of FIG. The perspective view which shows the example of the appearance of the security gate which is a stationary device provided in the system of the same embodiment. The block diagram which shows the functional structure of the security gate of FIG. The perspective view which shows the example of the appearance of the autonomous traveling device provided in the system of the same embodiment. FIG. 21 is a block diagram showing a functional configuration of the autonomous traveling device of FIG. 21. The perspective view which shows the example of the appearance of the toilet type device provided in the system of the same embodiment. FIG. 2 is a block diagram showing a functional configuration of the toilet-type device of FIG. 23. FIG. 6 is a perspective view showing another example of the appearance of the toilet-type device provided in the system of the same embodiment. FIG. 2 is a block diagram showing a functional configuration of the toilet-type device of FIG. 25.

Hereinafter, embodiments will be described with reference to the drawings.

First, the configuration of the system according to the embodiment will be described with reference to FIG. This system is a stress estimation system 1 that estimates a user's chronic mental stress and provides information according to the estimated stress. Information about the user is acquired using various sensing devices 2. The sensing device 2 is a device capable of sensing with a low load on the user, and therefore, it is possible to realize subtle sensing and prevent the user from being aware of the sensing. Each sensing device 2 may be one that senses one user, or may be one that senses a plurality of users. The data (sensor data) obtained by sensing can be collected in parallel by each of the plurality of sensing devices 2. Then, by integrating a plurality of sensor data collected by the plurality of sensing devices 2, for example, which are not time-synchronized, a multimodal and highly accurate mental stress value can be estimated for each user.

The sensing device 2 includes a human body-worn device, a portable device, a stationary device, an autonomous traveling device, and the like. A human body-worn device is a device that can be fixed to a part of the human body (that is, can be worn at all times), and is, for example, a wrist (wristband type) device, an ankle device, a sticking device, a pants type device, or a shirt type. It is a wearable device like a device. The portable device is a device that can be always carried in the form of being held, put in a pocket of clothes, put in a bag, and the like, and includes, for example, a smartphone, a mobile phone, a PDA, a tablet computer, and the like. Stationary devices are devices that are placed and used in specific locations, such as notebook computers (notebook PCs), desktop computers (desktop PCs), security gates, in-vehicle devices, and surveillance cameras. It includes devices, and may also include devices such as chairs, toilets, IoT devices, etc. with built-in sensors. Further, the autonomous traveling type device includes, for example, a robot type device such as a robot cleaner and a communication robot.

The sensing device 2 includes at least one of a sensor that comes into contact with the human body and a non-contact sensor that does not come into contact with the human body. The contact-type sensing device 2A includes, for example, a photoelectric pulse wave sensor that detects a pulse wave (pulse), an electrocardiographic sensor that detects an electrocardiogram (heartbeat), an acceleration sensor that detects movement (exercise, sleep), and the like. .. The non-contact sensing device 2B has a camera that captures images for detecting pulse waves (pulses) and movements (movement, facial expressions, sleep), and records voice for detecting emotions based on conversation volume and utterances. Includes microphones, biomarkers for detecting specific substances from urine and saliva, temperature and humidity sensors for detecting environmental information, and the like. An input device such as a keyboard, a pointing device, or an IC tag reading device can also be used as a sensor for detecting an operation by the user.

In the cloud inspection service, a stress value (stress index) indicating the degree of stress felt by the user is estimated using the data acquired by using the sensing device 2. This cloud inspection service is realized by, for example, a server 3 that can use the user's sensor data. The server 3 also includes various diagnostic / test results such as genomic tests, industrial doctor diagnoses, and depression diagnoses, history, questionnaires, questionnaires, etc. included in the database 4 such as personal health records (PHR). It may be used to estimate the stress value.

In addition, the server 3 can also select a significant factor (factor) that has a correlation with stress by using the data of the case of developing a specific disease (for example, depression). The server 3 estimates the user's stress value based on the data of the selected factor.

Furthermore, the estimation of stress values and the selection of factors that correlate with stress should improve accuracy and reliability by verifying stress values and factors in collaboration with the medical field such as hospitals, doctors, and industrial physicians. Can be done. The server 3 is used, for example, as an index related to a specific substance contained in saliva or urine, which is already used as a stress value in the medical field, and a subjective questionnaire (for example, State-Trait Anxiety Industry (STAI)). It is also possible to estimate the stress value by further using the index based on the index.

The server 3 determines the behavior modification program based on the estimated stress value. Behavior modification programs are programs supervised by professionals such as industrial physicians, counselors, and clinical psychologists. The server 3 notifies the user with a high stress value that the stress value is high, and provides information for reducing the stress. The server 3 may deliver advice such as recommended behavior (mindfulness) and work style to the smartphone used by the user, for example.

The server 3 can also change the behavior change program according to the attributes of the user. The server 3 is set up so that when a user with a high stress value is included in a plurality of users who are employees or drivers engaged in interpersonal work such as a call center, the stress value of that user is reduced. A fixed work shift (schedule) may be automatically generated. Similarly, the server 3 provides information for reducing stress or shifts to work, taking into account the attributes of the user during a specific disease (for example, a cancer patient), postpartum, childcare, long-term care, and the like. May be automatically generated.

The cloud inspection service by the server 3 can be used, for example, by a stress check provider, a company with interpersonal business, an insurance company, and the like.

Next, some examples of the configuration of the stress estimation system 1 will be described with reference to FIGS. 2 to 4.

First, the stress estimation system 1 shown in FIG. 2 includes a plurality of devices 2-1, 2, 2-3, and a mental stress estimation unit 31.

Device 2-1 has a sensor 21-1. The sensor 21-1 collects access log information indicating a log in which the user has accessed the device 2-1 and outputs (transmits) it to the mental stress estimation unit 31. The device 2-1 is, for example, a desktop PC, a notebook PC, an in-vehicle device, a security gate, or the like.

Device 2-2 has a sensor 21-2. The sensor 21-2 collects operation information indicating the operation of the device 2-2 by the user and outputs (transmits) it to the mental stress estimation unit 31. The device 2-2 is, for example, a wearable device such as a wrist device, a portable device, a desktop PC, a notebook PC, or the like.

Further, the device 2-3 has a sensor 21-3. The sensor 21-3 collects the biometric information of the user and outputs (transmits) it to the mental stress estimation unit 31. Devices 2-3 are, for example, wearable devices such as wrist devices, portable devices, desktop PCs, notebook PCs, home IoT devices, home robots, security gates, in-vehicle devices, toilet-type devices and the like. In addition, the biological information includes, for example, analysis results of pulse waves and heartbeats.

The mental stress estimation unit 31 estimates the stress value indicating the user's mental stress by using the access log information, the operation information, and the biological information collected by the devices 2-1, 2, 2 and 2-3. More specifically, the mental stress estimation unit 31 estimates the user's attendance status and activity status using, for example, access log information, and estimates the user's activity status and emotion using operation information. Then, the mental stress estimation unit 31 estimates the stress value using the estimated attendance status, activity status, and emotion, and the biological information.

Although an example in which one sensor is provided in one device is shown here, a plurality of sensors may be provided in one device. Therefore, one device including a plurality of sensors can collect a plurality of types of information such as access log information and operation information. The mental stress estimation unit 31 can estimate the stress value of the user by using the information obtained from at least one device including at least one sensor.

Next, the stress estimation system 1 shown in FIG. 3 includes a device 2-4 and a genome database (DB) 4-1 in addition to the configuration shown in FIG.

Device 2-4 has a sensor 21-4. The sensor 21-4 includes, for example, a biomarker that detects the amount or concentration of a specific substance contained in saliva or urine, and outputs biomarker information indicating the detection result by the biomarker to the mental stress estimation unit 31 ( Send. The biomarker detects, for example, salivary amylase, salivary chromogranin, salivary cortisol, urinary catecholamine, and the like. The device 2-4 is, for example, a toilet type device, a pants type device, or the like.

The genome DB 4-1 stores the genome analysis result obtained when the user undergoes a genome test or the like. The mental stress estimation unit 31 acquires information highly related to stress from the information stored in the genome DB 4-1. The acquired genomic information can be used to determine resistance to stress. The acquired information is, for example, single nucleotide polymorphism (SNP) information that is vulnerable to stress, genetic information related to the onset of manic-depressive illness, and the like. Since the gene associated with the onset of manic-depressive illness has been identified, a user carrying that gene can be determined to be vulnerable to stress.

The mental stress estimation unit 31 includes, in addition to the access log information, operation information, and biometric information collected by the devices 2-1, 2, 2 and 2-3, the biomarker information collected by the device 2-4, and the biomarker information. The stress value is estimated using the genome information obtained from the genome DB4-1.

Further, the stress estimation system 1 shown in FIG. 4 shows an example in which the estimation of the stress value is functionalized. Before estimating the stress value, the mental stress estimation unit 31 extracts sensor data, genomic information, and the like of a case of developing a specific disease (for example, depression) by using the past industrial doctor consultation status and depression diagnosis status. Then, the parameters (factors) 22 (for example, x1 (t), x2 (t), ..., Xn (t)) that are highly related to the onset of the disease are learned. Then, the mental stress estimation unit 31 uses these parameters 22 to generate a function F (x) for calculating the stress value. The mental stress estimation unit 31 uses this function F (x) to calculate the stress value y (t) based on the newly obtained sensor data or the like, thereby calculating the stress value of the user according to the actual case. Can be estimated. The mental stress estimation unit 31 calculates one stress value y (t) for each user, for example, every day, and the stress value is continuously high (for example, exceeding the reference value). Detects that the abnormal value is different from usual.

The stress estimation system 1 may further include a behavior change determination unit 32. The behavior change determination unit 32 outputs information for reducing stress based on the calculated stress value y (t). The behavior change determination unit 32 outputs information for reducing stress, for example, when the stress value is continuously high or when the abnormal value is different from usual. Such information can be presented to the user, for example, via e-mail, a smartphone application, or the like.

More specifically, the behavior change determination unit 32 outputs a warning to the user according to the magnitude of the stress value. The behavior change determination unit 32 displays, for example, a high stress value, a high risk of chronic stress, and the like on the screen of the device 2 used by the user.

Further, the behavior change determination unit 32 outputs information on the behavior recommended by the user in order to reduce the stress according to the magnitude of the stress value. The behavior change determination unit 32, for example, takes mental fullness, takes a deep breath, exercises that change the mood, meditation, exercise, looks at nature, loves Kacho Fugetsu, etc. on the screen of the device 2 used by the user. Display a prompting message. Also, the recommended behavior is determined according to attributes such as whether the user has a particular illness (eg, dementia, cancer, etc.) or is a worker, child-rearing, or caregiver. May be good.

Further, the behavior change determination unit 32 can output information on the recommended work style according to the magnitude of the stress value. The behavior change determination unit 32 displays, for example, a message recommending taking a time break, taking annual leave, short-term leave, long-term leave, etc. on the screen of the device 2 used by the user. Further, the behavior change determination unit 32 is used for employees engaged in work that is extremely stress-prone, such as call center work and driver work, when a plurality of users are using the stress estimation system 1. It is also possible to present a work shift table rearranged according to the magnitude of the stress value. The behavior change determination unit 32 automatically generates a work shift table in which the working hours of the user are reduced, for example, in order to improve the state of the user having a high stress value. Further, the behavior change determination unit 32 automatically generates a work shift table for keeping the stress of all employees low, for example, immediately before the busy season. The recommended work style (work shift) is an attribute such as whether the user has a specific disease (for example, dementia, cancer, etc.), or is a worker, a childcare worker, or a caregiver. It may be determined accordingly.

In the following, a case where the stress estimation system 1 is composed of a server 3 and one or more devices 2 will be illustrated. The server 3 and each device 2 can communicate with each other via a network. One or more devices 2 collect sensor data about the user using the sensors 21 provided in each and transmit the sensor data to the server 3. The server 3 estimates the stress value using the collected sensor data, and generates information for reducing the stress according to the stress value.

FIG. 5 shows an example of a system configuration of the server 3. The server 3 can be realized as, for example, a server computer. The server 3 includes a CPU 301, a main memory 302, a non-volatile memory 303, a BIOS-ROM 304, a communication module 305, and the like.

The CPU 301 is a processor that controls the operation of various components in the server 3. The CPU 301 executes various programs loaded from the non-volatile memory 303, which is a storage device, into the main memory 302. These programs include an operating system (OS) 302A and various application programs. The application program includes the stress estimation program 302B. The stress estimation program 302B includes a command group for acquiring sensor data from the device 2, a command group for estimating the stress value, a command group for behavior change based on the stress value, and the like. That is, the stress estimation program 302B has the functions of the mental stress estimation unit 31 and the behavior change determination unit 32 described above with reference to FIGS. 2 to 4.

The CPU 301 also executes the basic input / output system (BIOS) stored in the BIOS-ROM 304. The BIOS is a program for hardware control.

The communication module 305 is a device configured to perform wired or wireless communication. The communication module 305 includes a transmitting unit that transmits a signal and a receiving unit that receives the signal.

Further, FIG. 6 shows an example of a system configuration of the device 2. The device 2 includes a CPU 201, a main memory 202, a non-volatile memory 203, a communication module 204, a sensor 21, and the like.

The CPU 201 is a processor that controls the operation of various components in the device 2. The CPU 201 executes various programs loaded from the non-volatile memory 203, which is a storage device, into the main memory 202. These programs include an operating system (OS) 202A and various application programs. The application program includes the stress monitoring program 202B. The stress monitoring program 202B includes an instruction group for collecting data from the sensor 21, an instruction group for analyzing the collected data, an instruction group for transmitting the data to the server 3, and the like.

Communication module 204 is a device configured to perform wired or wireless communication. The communication module 204 includes a transmitting unit that transmits a signal and a receiving unit that receives the signal. The device 2 may further include a display 205 for video output and a speaker 206 for audio output.

Next, with reference to FIG. 7, the functional configuration of the stress estimation program 302B executed by the server 3 and the stress monitoring program 202B executed by each device 2-1, 2-2, ..., 2-N will be described. do. Each of the devices 2-1 and 2, ..., 2-N has the system configuration described above with reference to FIG. The server 3 and each device 2-1, 2, 2, ..., 2-N can send and receive data to and from each other via the network 15.

The stress monitoring program 202B executed by each device 2-1, 2-2, ..., 2-N includes, for example, a sensor processing unit 251 and a display control unit 252. Further, the stress estimation program 302B executed by the server 3 includes, for example, a storage processing unit 311, a factor determination unit 312, a stress value determination unit 313, an alert generation unit 314, and a schedule generation unit 315.

The sensor processing unit 251 of the device 2 transmits the sensor data obtained by performing a specific process such as analysis on the data output by the sensor 21-1 to the server 3 via the communication module 204. The transmitted data may include information for identifying the user to be sensed (user ID) and information indicating the date and time of sensing (time stamp). The sensor processing unit 251 may transmit the data output by the sensor 21-1 to the server 3 as it is. In that case, the function of the sensor processing unit 251 for performing a specific process such as analysis on the data is provided in the server 3.

The storage processing unit 311 of the server 3 receives sensor data from the device 2 (devices 2-1 and 2, ..., 2-N) via the communication module 305. The storage processing unit 311 stores the received sensor data in the sensor database 35. For the sensor database 35, for example, the non-volatile memory 303 is used. The sensor database 35 may collectively store sensor data obtained by using a plurality of types of devices 2 (or a plurality of types of sensors 21), or store sensor data for each device 2 (or sensor 21). A plurality of sensor databases 35 may be provided for this purpose. The stored sensor data can identify the sensed target user and the sensed date and time. Further, the sensor database 35 may be provided outside the server 3 as long as the sensor data can be used by the server 3.

The factor determination unit 312 determines a factor (factor) having a high relevance to long-term stress by using the sensor data stored in the sensor database 35 and the data stored in the health record database 4. In the present embodiment, the stress perceived by the user is represented by an index of the magnitude of stress over a certain period, and is defined by short-term stress and long-term stress according to the length of the period. The index (value) of long-term stress indicates the magnitude of stress over a period longer than at least the period of short-term stress. In addition, the index (value) of short-term stress indicates the magnitude of stress over a period shorter than the period of long-term stress at least. As an example, the long term is a period of 1 week or more, 2 weeks, 1 month, 3 months, etc., and the short term is less than 1 week, 2 to 3 days, 1 day, half a day, 2 to 3 hours, 1 hour. , 30 minutes, minutes, seconds, etc. Therefore, it can be said that the long-term stress is the user's chronic stress and the short-term stress is the user's temporary stress (that is, instantaneous stress).

The factor determination unit 312 uses, for example, the degree of change in the magnitude of stress according to the first method and the degree of change in a plurality of factors related to stress, and determines one or more of the plurality of factors. , A factor used to determine long-term stress. This first method determines stress based on at least one of, for example, a doctor's determination of stress, a questionnaire-based stress determination, a urine-based stress determination, and a saliva-based stress determination. It is a judgment method. In addition, multiple factors include one or more factors related to at least one of human heartbeat or pulse, one or more factors related to the operation status of electronic devices, one or more factors related to attendance status, and one or more factors related to emotions. Includes at least two or more of one or more factors related to the state of movement of the human body, one or more factors related to the state of speech, one or more factors related to the state of sleep, and one or more factors related to the environment. It should be noted that the first method described above may further include other methods for determining stress, and the plurality of factors further include other factors based on sensor data sensed by the user. You may. The electronic device is a portable device 2 or a human body-worn device 2.

The stress value determination unit 313 determines the short-term stress value in a short period of time using the sensor data stored in the sensor database 35 of a certain user, and then further determines the short-term stress value in another short period of time. do. The value of short-term stress is determined in a short period of time, for example, using sensor data obtained from at least one sensor. Then, the stress value determination unit 313 determines the magnitude of the long-term stress (for example, the long-term stress value) of the user based on the changes in the two short-term stress values. The stress value determination unit 313 may further use the data stored in the health record database 4 to determine the magnitude of long-term stress.

More specifically, in the stress value determination unit 313, the short-term stress before starting a specific work (for example, driving a car) is the first value, and the short-term stress during the work is larger than the first value. When it is a binary value, the magnitude of long-term stress (long-term stress value) is determined according to the change in the stress value from the second value to the first value after the work is completed. The stress value determination unit 313 may determine the magnitude of long-term stress by further using the information regarding the frequency of operation of the electronic device. In addition, the stress value determination unit 313 may determine the magnitude of long-term stress by further using information on repeated fluctuations of the human body above the threshold value (for example, poverty-sucking).

The alert generation unit 314 generates a warning to the user according to the magnitude of the long-term stress determined by the stress value determination unit 313. In addition, the alert generation unit 314 may generate information on the behavior recommended by the user to reduce the stress according to the determined magnitude of long-term stress. The alert generation unit 314 transmits the generated warnings and information to the device 2 via the communication module 305. The destination device 2 is, for example, a device including the display 205 or a device designated in advance by the user.

In addition, the schedule generation unit 315 generates information on recommended work styles according to the magnitude of long-term stress (for example, long-term stress value) determined by the stress value determination unit 313. For example, when the long-term stress value of each of a plurality of employees engaged in a certain task is determined, the schedule generation unit 315 may reduce the stress applied to the employee having a high long-term stress value. Generate a work schedule for employees. The schedule generation unit 315 transmits information about the work mode such as the work schedule to the device 2 via the communication module 305.

The display control unit 252 of the device 2 receives warnings and information from the server 3 via the communication module 204. Then, the display control unit 252 displays warnings and information on the screen of the display 205.

It should be noted that each part in the server 3 may be provided in the device 2. In that case, for example, sensor data acquired by a plurality of devices 2 may be accumulated in a certain device 2. Further, for example, a certain device 2 may determine a stress value using sensor data, generate a warning or information based on the stress value, and the like.

In the following, some devices 2 to which the above-described configuration is applied will be illustrated.

<On-board unit>
An example in which a stress index of a user driving a car is estimated will be described with reference to FIGS. 8 to 13. This user is sensed by, for example, sensors provided in the on-board unit 2-6 and the wearable device 2-5, which are devices 2 mounted on the automobile. The automobile equipped with the on-board unit 2-6 may be used during non-working hours, or may be used during work of a user engaged in delivery work or the like.

FIG. 8 shows an example of the appearance of the on-board unit 2-6. The on-board unit 2-6 includes, for example, a power button 501, a camera 503, an electrocardiographic sensor 504, a humidity sensor 505, a temperature sensor 506, a vibration sensor / acceleration sensor (not shown), and the like.

The power button 501 is located, for example, on the right side of the steering wheel and is used to start the engine of an automobile. By detecting the engine operation started in response to the detection of the operation of the power button 501, it is possible to acquire information on the operation by the user. The power button 501 is a kind of UI sensor. The camera 503 is provided in a sun visor section or an instrumental panel (instrument panel) section in which meters are installed, and captures a facial image (video) of a user driving a car. The camera 503 is a kind of media sensor.

The electrocardiographic sensor 504 is provided in the seat and detects the heartbeat waveform (electrocardiogram) of the seated user. The electrocardiographic sensor 504 is a kind of biosensor. The humidity sensor 505 and the temperature sensor 506 are provided on the instrument panel, for example, and detect the humidity and temperature inside the vehicle. The humidity sensor 505 and the temperature sensor 506 are a kind of environment sensor. Further, the vibration sensor / acceleration sensor detects the movement of the vehicle by detecting the vibration / acceleration generated in the on-board unit 2-6.

In addition, this user may wear a wearable type (for example, a wristband type) device 2-5. Device 2-5 includes, for example, a vibration sensor / acceleration sensor 502, a microphone, a photoelectric pulse wave sensor, an environment sensor, and the like. The vehicle-mounted device 2-6 can utilize the vibration sensor / acceleration sensor 502 provided in the device 2-5 in addition to or instead of the vibration sensor / acceleration sensor provided in the vehicle-mounted device 2-6.

The on-board unit 2-6 can collect sensor data by using such sensors 501 to 506. The vehicle-mounted device 2-6 may be further provided with another sensor such as a GPS receiver. Further, as described above with reference to FIG. 6, each of the on-board unit 2-6 and the wearable device 2-5 has a built-in CPU 201, main memory 202, non-volatile memory 203, communication module 204, and the like. ..

Next, the functional configuration of the vehicle-mounted device 2-6 will be described with reference to FIG. The on-board unit 2-6 includes the above-mentioned sensors 501 to 506 and the GPS receiver 507, and executes the stress monitoring program 202B. Hereinafter, first, the functional configuration of the sensor processing unit 251 included in the stress monitoring program 202B will be described.

The sensor processing unit 251 is, for example, an engine motion detection unit 511, a face detection unit 512, a video pulse wave calculation unit 513, a pulse wave peak interval calculation unit 514, an autonomic nerve analysis unit 515, a facial expression recognition unit 516, and a heartbeat peak interval. It includes a calculation unit 517, an autonomic nerve analysis unit 518, an environmental discomfort calculation unit 519, a running / stop detection unit 520, and a route analysis unit 510.

The engine operation detection unit 511 detects the engine operation, various odometry, the riding time of the automobile, the speed, and the like in response to the engine being started by the operation of the power button 501. The engine operation detection unit 511 outputs (transmits) such detected information to the mental stress estimation unit 31 as access log information to the on-board unit 2-6. Further, the engine operation detection unit 511 outputs information indicating that the engine has started or stopped to the video pulse wave calculation unit 513 based on the detected information.

The travel / stop detection unit 520 detects whether the vehicle is traveling (moving) or stopped by using at least one of the vibration data and the acceleration data obtained by the vibration sensor / acceleration sensor 502. The travel / stop detection unit 520 can also use at least one of the vibration data and the acceleration data to detect a specific driving operation such as sudden steering, sudden acceleration, and sudden braking. The travel / stop detection unit 520 outputs the detected information indicating the travel or stop of the vehicle and the information indicating the driving operation to the video pulse wave calculation unit 513. The travel / stop detection unit 520 may transmit the vibration data and / or the acceleration data as it is to the mental stress estimation unit 31.

The face detection unit 512 detects a face from an image obtained by using the camera 503. The face detection unit 512 detects the face image on the video frame by detecting, for example, a region having a feature amount similar to the face feature amount prepared in advance from each of the continuous video frames constituting the video. .. The face detection unit 512 outputs information about the detected face image to the image pulse wave calculation unit 513 and the facial expression recognition unit 516.

The image pulse wave calculation unit 513 calculates information about the user's pulse wave by using the information output by the face detection unit 512. More specifically, the video pulse wave calculation unit 513 calculates the pulse wave of the user based on the change in the face image (for example, the change in the color of the skin) detected from the continuous video frames.

The calculated user pulse wave includes information output by the engine operation detection unit 511 indicating that the engine has started or stopped, and that the vehicle output by the drive / stop detection unit 520 is running or stopped. The information indicating the driving operation is associated with the information indicating the driving operation. By using such information, it is possible to specify the timing at which a certain driving operation is detected before starting driving, during driving, after finishing driving, and to extract information. The timing indicating the start, operation, and end of operation may be explicitly input by the user using the input device provided on the on-board unit 2-6 or the wearable device 2-5. ..

The pulse wave peak interval calculation unit 514 calculates the calculated pulse wave peak interval (PPI). Then, the autonomic nerve analysis unit 515 calculates the autonomic nerve index indicating the pulse rate and the variation of the pulse by analyzing the calculated peak interval of the pulse wave. This autonomic nerve index represents the balance of the autonomic nerves. The autonomic nerve analysis unit 515 calculates an autonomic nerve index indicating a pulse rate and a variation in the pulse at the timing when a certain driving operation is detected before, during, or after the driving of the car. May be good. The autonomic nerve analysis unit 515 transmits the biological information including the calculated autonomic nerve index to the mental stress estimation unit 31.

Further, the facial expression recognition unit 516 recognizes the facial expression of the facial image detected from the video. The facial expression recognition unit 516 can recognize not only the types of facial expressions such as emotions, but also wrinkles between the eyebrows, facial movements, and the like. The facial expression recognition unit 516 transmits emotional information including the type of the recognized facial expression to the mental stress estimation unit 31.

The face detection unit 512 and the facial expression recognition unit 516 detect body movements such as hands, feet, and head while driving from images obtained by using the camera 503, and are drowsy (for example, head shaking). Alternatively, you may recognize movements that are likely to occur when you are irritated (for example, repeated movements of your fingers and feet). Information indicating the recognized motion can be transmitted to the mental stress estimation unit 31 as emotional information.

Next, the heart rate peak interval calculation unit 517 calculates the heart rate interval (RR interval), which is the peak interval of the waveform measured by the electrocardiographic sensor 504. The autonomic nerve analysis unit 518 calculates an autonomic nerve index indicating a heart rate and a variation in heart rate by analyzing the calculated heart rate interval. This autonomic nerve index represents the balance of the autonomic nerves. The autonomic nerve analysis unit 518 transmits the biological information including the calculated autonomic nerve index to the mental stress estimation unit 31.

The environmental discomfort calculation unit 519 calculates the environmental discomfort using the humidity measured by the humidity sensor 505 and the temperature measured by the temperature sensor 506. For this environmental discomfort, for example, a Predicted Mean Vote (PMV) or a Wet Bull Globe Temperature (WBGT) is used. The environmental discomfort degree calculation unit 519 transmits the environmental information including the calculated environmental discomfort degree to the mental stress estimation unit 31.

The route analysis unit 510 identifies the road on which the automobile is traveling by using the position of the automobile acquired by using the GPS receiver 507 and the map data used in the car navigation system or the like. Then, the route analysis unit 510 indicates whether the specified road is a road that always travels, an unfamiliar road that rarely travels, and whether it is a general road or an expressway. Route information is generated and transmitted to the mental stress estimation unit 31.

The mental stress estimation unit 31 (stress value determination unit 313) determines the user's short-term stress value using the information output by the on-board unit 2-6, and the user's long-term stress value using the determined short-term stress value. Determine the magnitude of stress. For example, the mental stress estimation unit 31 uses at least one of access log information, biological information, emotional information, environmental information, and route information to the device output from the on-board unit 2-6 for a short period of time. Calculate the value of short-term stress in. Then, when the short-term stress before the user starts driving the automobile is the first value and the short-term stress during driving the automobile is the second value larger than the first value, the mental stress estimation unit 31 determines the automobile. The magnitude of the long-term stress (for example, the long-term stress value) is determined according to the change in the stress value from the second value to the first value after the operation of the above is completed. The faster this change (that is, the rate at which the short-term stress value decreases), the smaller the long-term stress, and the slower the change, the larger the long-term stress. This is because, for users with low long-term stress, even if the short-term stress value temporarily rises due to stressful work such as driving, it is easy to return to the normal short-term stress value when the work is completed, while long-term stress It is based on the fact that once the short-term stress value rises due to the work, it is difficult or not possible for a large user to return to the normal short-term stress value even after the work is completed. Therefore, the magnitude of long-term stress is determined according to the ease of returning from the short-term stress value (second value) during operation to the normal (first value) short-term stress value (first value) after the end of operation. can do.

The change in the stress value from the second value to the first value after the driving of the car is finished is expressed as an absolute value, for example, for a long period of time based on whether or not the stress value drops by more than the threshold value in one hour. The magnitude of stress (long-term stress value) is determined. Further, this change in stress value may be expressed as a relative value. In that case, for example, the magnitude of long-term stress (long-term stress value) is determined based on what percentage of the increase from the first value to the second value decreases in one hour.

Since it is estimated that a driving operation such as a sudden steering wheel is caused by distraction due to stress, the mental stress estimation unit 31 determines that the driving operation such as a sudden steering wheel is detected from the sensor data. It can contribute to higher determined short-term stress values. Similarly, factors that increase the short-term stress value determined by the mental stress estimation unit 31 include a higher pulse rate / heart rate than normal, a large variation in pulse wave and heart rate, and specific factors such as anger. Includes recognition of facial expressions, recognition of drowsiness and irritability, driving on highways and rarely used roads, long ride times, high environmental discomfort, etc. obtain.

Further, the stress value is not only the sensor data acquired by the on-board unit 2-6 and the wearable device 2-5, but also the other device 2 (for example, a portable device, a stationary device, a toilet type device, an autonomous driving type). It may also be determined using the sensor data acquired by the device or the like). In that case, the mental stress estimation unit 31 may calculate the long-term stress value by, for example, comparing the stress value during driving of the automobile with the stress value during the next sleep, or the stress value of the automobile. The long-term stress value may be calculated by comparing the stress value during operation with the stress value after the next wake-up.

FIG. 10 shows an example in which a stress index is determined using pulse wave or heartbeat waveform data. As described above, the pulse wave peak interval calculation unit 514 calculates the pulse wave peak interval (pulse interval) 535 by using the pulse wave 521 calculated by the video pulse wave calculation unit 513. Further, the heart rate peak interval calculation unit 517 calculates the heart rate interval 536, which is the peak interval of the waveform measured by the electrocardiographic sensor 504. The autonomic nerve analysis unit 515, 518 frequency-analyzes the calculated pulse interval 535 or heart rate interval 536 to calculate the power spectral density for each predetermined frequency. The sum of the power spectral densities (HF) contained in the high frequency band (eg, 0.15 to 0.4 hertz) represents the degree of parasympathetic activity, i.e., the degree of relaxation. Further, an index (LF) indicating the total power spectral density (LF) contained in the low frequency band (for example, 0.05 to 0.15 hertz) with respect to the total power spectral density (HF) contained in the high frequency band. / HF) represents the degree of sympathetic function, that is, the degree of stress. The mental stress estimation unit 31 uses this index LF / HF as a stress index for short-term stress (instantaneous stress).

As shown in FIG. 10, the sympathetic nerve function 525 and the parasympathetic nerve function 526 are sequentially changing. The mental stress estimation unit 31 can calculate a stress index according to, for example, the superiority of the sympathetic nerve.

11 and 12 are graphs showing changes in the stress index LF / HF during operation.

First, Graph 527 in FIG. 11 shows changes in the stress index when traveling on an outward route including a highway. In this graph 527, it can be seen that the low stress index 527-1 at the time of departure (immediately after departure) rises during traveling. In particular, the stress index 527-2 during the period of traveling on the expressway is relatively higher than the stress index 527-1 and 527-3 during the period of not traveling on the expressway.

Next, graph 528 in FIG. 12 shows changes in the stress index when traveling on a return route including an unfamiliar road. Also in this graph 528, it can be seen that the low stress index 528-1 at the time of departure rises during traveling. In particular, the stress index 528-2 during the period of traveling on an unfamiliar road is relatively higher than the stress index 528-1 and 528-3 during the period of not traveling on the road.

Further, FIG. 13 shows changes in the saliva amylase value, which is a stress index already used in the medical field, and the subjective questionnaire result (STAI) during operation. In the example shown in FIG. 13, the values before and after the start of operation of the outward route corresponding to the graph 527 shown in FIG. 11 and the values before the start of operation and after the end of the operation of the return route corresponding to the graph 528 shown in FIG. The value and is shown. It can be seen that the values of both saliva amylase and STAI after the end of the operation are higher than the values before the start of the operation.

The mental stress estimation unit 31 can also integrate various sensor data such as LF / HF, saliva amylase, and STAI to calculate stress indexes such as short-term stress value and long-term stress value.

<Portable devices and wearable devices>
An example in which the stress index of the user who wears the wearable device 2-8 is estimated by using the portable device 2-7 will be described with reference to FIGS. 14 to 16. This user is sensed, for example, by sensors provided in the portable device 2-7 and the wearable device 2-8.

FIG. 14 shows an example of the appearance of the portable device 2-7. FIG. 14 illustrates a case where the portable device 2-7 is a smartphone. The portable device 2-7 is an electronic device that is taken outdoors and carried and used, and may be used indoors. The portable device 2-7 has, for example, sensors such as a microphone 601, a button 602, an acceleration sensor 603, and a camera 608.

A display 205 is provided on the upper surface of the main body 60 of the portable device 2-7. The display 205 is, for example, a touch screen display, and is composed of an LCD and a capacitive touch panel arranged on the upper surface thereof. The button 602 is provided on the lower portion of the display 205 and the left side surface of the main body 60, and is used for an operation of shifting the portable device 2-7 to a locked state (sleep state), an operation of returning from the locked state, and the like. The portable device 2-7 can also be operated by using a GUI button or the like displayed on the screen of the display 205.

The microphone 601 is provided on the lower side surface of the main body 60 and collects voice including utterances by the user. The acceleration sensor 603 is provided in the main body 60, for example, at the lower part, and detects the movement of the portable device 2-7. Further, the camera 608 is provided on the upper part of the display 205, and can, for example, take a picture of a user's face or the like when looking at the screen of the display 205. As described above with reference to FIG. 6, the main body 60 contains a CPU 201, a main memory 202, a non-volatile memory 203, a communication module 204, and the like.

Next, FIG. 15 shows an example of the appearance of the wearable device 2-8. Here, a case where the wearable device 2-8 is a pants type device will be illustrated. The wearable device 2-8 has, for example, sensors such as two microphones 601 and four acceleration sensors 603, a photoelectric pulse wave sensor 604, an electrocardiographic sensor 605, and an ultrasonic sensor 609. The photoelectric pulse wave sensor 604 and the electrocardiographic sensor 605 may be either one. Further, the wearable device 2-8 may be further provided with a button for operation.

A contraction portion (for example, a rubber portion) 62 in contact with the user's waist of the wearable device 2-8 is provided with two microphones 601, two acceleration sensors 603, a photoelectric pulse wave sensor 604, and an electrocardiographic sensor 605. There is.

The microphone 601 collects voices including utterances by the user, utterances of the user's dialogue partner, sounds accompanying the movement of the user, internal sounds, and the like. By arranging the two microphones 601 by beamforming, it is possible to distinguish between the utterance by the user and the utterance of the dialogue partner.

The acceleration sensor 603 detects the movement of the wearable device 2-8, that is, the movement of the user who wears the wearable device 2-8, by detecting the acceleration generated in the wearable device 2-8. In the contracted portion 62, one acceleration sensor 603 is arranged on the abdominal side and the back side, respectively.

The photoelectric pulse wave sensor 604 is, for example, a light emitting diode (LED) and a photodetector (Photo).
It is composed of Detector: PD) and detects the pulse wave of the user based on the photoelectric volume pulse wave method. The electrocardiographic sensor 605 detects the waveform (electrocardiogram) of the user's heartbeat. A CPU 201, a main memory 202, a non-volatile memory 203, a communication module 204, and the like may be incorporated in a part 61 of the contracted portion 62 as described above with reference to FIG.

Further, an acceleration sensor 603 and an ultrasonic sensor 609 are provided in the portions of the wearable device 2-8 in contact with the left and right thighs of the user. The ultrasonic sensor 609 detects the muscle mass of the thigh by ultrasonic waves.

The above-mentioned sensors 601 to 609 are examples, and each sensor may be provided in any of the devices 2-7 and 2-8. Further, the devices 2-7 and 2-8 may be further provided with other sensors such as a barometric pressure sensor, an illuminance sensor, a pressure sensitive sensor (electrostatic sensor), a gyro sensor, a temperature sensor, and a humidity sensor. ..

Further, the user may wear another wearable (for example, wristband type) device 2-5 as shown in FIG. Device 2-5 includes, for example, a vibration sensor / acceleration sensor, a microphone, a photoelectric pulse wave sensor, an environment sensor, and the like.

Next, with reference to FIG. 16, the functional configurations of the portable device 2-7 and the wearable device 2-8 will be described. For example, the portable device 2-7 and the wearable device 2-8 include the above-mentioned sensors 601 to 605, a humidity sensor 606, and a temperature sensor 607, and execute the stress monitoring program 202B. The humidity sensor 606 and the temperature sensor 607 may be installed in the devices 2-7 and 2-8, or may be installed in the environment. When installed in an environment, the devices 2-7, 2-8 acquire humidity data and temperature data from these sensors 606,607, for example, by communication via the communication module 204. Hereinafter, first, the functional configuration of the sensor processing unit 251 included in the stress monitoring program 202B will be described.

The sensor processing unit 251 is, for example, a conversation amount estimation unit 611, a voice quality change detection unit 612, a fluctuation detection unit 613, an irritability calculation unit 614, an activity amount calculation unit 615, a pulse wave peak interval calculation unit 616, and a sleep analysis unit 617. , Autonomic nerve analysis unit 618, heart rate peak interval calculation unit 619, autonomic nerve analysis unit 620, and environmental discomfort calculation unit 621.

The conversation amount estimation unit 611 estimates the conversation amount by analyzing the voice data obtained by the microphone 601. The conversation amount estimation unit 611 may distinguish between the utterance by the user and the utterance of another person from the voice, and may estimate the utterance amount of each. The conversation amount estimation unit 611 transmits (outputs) utterance information indicating the conversation amount (or utterance amount) to the mental stress estimation unit 31.

The voice quality change detection unit 612 detects changes in voice quality due to emotions and tension by analyzing the voice data obtained by the microphone 601. The voice quality change detection unit 612 transmits the utterance information indicating the detected change in voice quality to the mental stress estimation unit 31.

The fluctuation detection unit 613 detects the repetition of fluctuations of the human body equal to or higher than the threshold value by using the acceleration data obtained by the acceleration sensor 603. The fluctuation detection unit 613 detects repeated movements (swaying, swinging) of the body, legs, arms, head, etc. that may be caused by stress, such as so-called poor swaying.

When the acceleration sensor 603 is provided in the portable device 2-7, the fluctuation detection unit 613 includes a pressure sensor, an illuminance sensor, and a pressure sensitive sensor (static) further provided in the portable device 2-7 in addition to the acceleration data. Using the data obtained by the electric sensor) or the like, it is determined whether the portable device 2-7 is in the bag, in the pocket of clothes, or is held by hand. Based on this determination result, when the portable device 2-7 is in the pocket of clothes or is held by hand, the fluctuation detection unit 613 detects repeated fluctuations of the human body above the threshold value. May be good.

Further, when the acceleration sensor 603 is provided in the wearable device 2-8, the fluctuation detection unit 613 uses, for example, the photoelectric pulse wave sensor 604, and the user wears the device 2-8. After identifying, the repetition of changes in the human body above the threshold value is detected.

The fluctuation detection unit 613 may further determine whether the user is sitting or standing by using the data obtained by the acceleration sensor 603, the barometric pressure sensor, the gyro sensor, or the like. When it is determined that the user is sitting, the fluctuation detection unit 613 analyzes the image obtained by the camera 608, for example, to shake the head without hesitation, move the hand (finger), or move the hair. You may detect movements that may be caused by stress such as touching.

The irritability calculation unit 614 is a device 2-7, 2-8 based on the detection result by the fluctuation detection unit 613 and the operation of the button (GUI) displayed on the screen of the button 602 or the display (touch screen display) 205. The degree of irritability (irritability) of the user is calculated by using the operation information to.

For example, the irritability calculation unit 614 calculates the frequency and shortness of operating the devices 2-7 and 2-8 based on the operation of the button 602 and the like, and calculates the irritability using the frequency and shortness. do. More specifically, the irritability calculation unit 614 is turned off (locked) again after unlocking the devices 2-7 and 2-8, without any operation of the application or by the operation of the application within the threshold period. ), The degree of irritability is calculated using the number and frequency of unnecessary operations. This threshold period is set to a period shorter than the time required for a significant operation of the devices 2-7 and 2-8, for example, an extremely short time such as several seconds, several tens of seconds, and several minutes. To. In other words, it is presumed that the number of unnecessary operations that are unlocked but turned off again without any useful operation is caused by the user's annoyance and restlessness, which is frustrating. The degree calculation unit 614 sets the degree of irritability higher as the number of times increases. Similarly, the irritability calculation unit 614 may set the irritability higher as the frequency of unnecessary operations increases. The target application may be a specific type of application, and excludes applications in which short-time operations such as alarm ringing, mail reception, message reception, etc. are expected. , It may be an application in the case where it is assumed that no significant operation is performed in a short time, such as when browsing a web page using a browser or sending an e-mail.

The irritability calculation unit 614 calculates the irritability based on the operation status of the devices 2-7 and 2-8 and the presence / absence and degree of repeated fluctuations of the human body above the threshold value. The irritability calculation unit 614 transmits emotional information including the calculated irritability to the mental stress estimation unit 31.

The activity amount calculation unit 615 calculates the activity amount of the user by analyzing the movement of the user using the acceleration data obtained by the acceleration sensor 603. From this amount of activity, it is possible to estimate whether or not the activity by the user is active. The activity amount calculation unit 615 transmits the calculated activity amount to the mental stress estimation unit 31.

The pulse wave peak interval calculation unit 616 calculates the pulse wave peak interval (PPI) detected by the photoelectric pulse wave sensor 604. The sleep analysis unit 617 analyzes the sleep state using the calculated peak interval of the pulse wave. The sleep analysis unit 617 estimates, for example, sleep time, sleep quality (for example, sleep depth, cycle), etc. by this analysis, and transmits biological information including the estimation result to the mental stress estimation unit 31. In addition, the autonomic nerve analysis unit 618 calculates the autonomic nerve index indicating the pulse rate and the variation of the pulse by analyzing the calculated peak interval of the pulse wave. The autonomic nerve analysis unit 618 transmits biometric information including the calculated autonomic nerve index to the mental stress estimation unit 31.

Further, the heart rate peak interval calculation unit 619, the autonomic nerve analysis unit 620, and the environmental discomfort calculation unit 621 have the heart rate peak interval calculation unit 517, the autonomic nerve analysis unit 518, and the environmental discomfort described above with reference to FIG. 9, respectively. It operates in the same manner as the degree calculation unit 519.

The mental stress estimation unit 31 (stress value determination unit 313) uses the information output by the portable device 2-7 and the wearable device 2-8 to determine the magnitude of the user's long-term stress (for example, the long-term stress value). judge. The mental stress estimation unit 31 calculates a long-term stress value using at least one of speech information, emotional information, activity amount, biological information, and environmental information output from devices 2-7 and 2-8. do.

The mental stress estimation unit 31 further uses, for example, the degree of irritability contained in the emotional information to determine the magnitude of long-term stress. For example, the mental stress estimation unit 31 determines a high long-term stress value when the degree of irritability is high, and determines a low long-term stress value when the degree of irritability is low. The mental stress estimation unit 31 may determine the magnitude of long-term stress by further using the information regarding the repetition of changes in the human body above the threshold value. For example, the mental stress estimation unit 31 determines a high long-term stress value when repeated fluctuations of the human body equal to or higher than the threshold value are detected, and determines a low long-term stress value when it is not detected. Further, the mental stress estimation unit 31 may determine the magnitude of long-term stress by further using the information regarding the frequency of operation of the electronic device. For example, the mental stress estimation unit 31 determines a high long-term stress value when the number (or frequency) of unnecessary operations is equal to or greater than the threshold value, and the low long-term stress value when the number of unnecessary operations (frequency) is less than the threshold value. Determine the value.

Further, the mental stress estimation unit 31 determines the magnitude of long-term stress based on the degree of change in the short-term stress value based on the degree of irritability, repeated fluctuations of the human body above the threshold value, the number (or frequency) of unnecessary operations, and the like. (Long-term stress value) may be determined. For example, in the mental stress estimation unit 31, the degree of irritability (short-term stress value) before starting a specific work (for example, driving a car) is the first value, and the degree of irritability during the work is larger than the first value. When it is the second value, the magnitude of long-term stress is determined according to the change in the degree of irritability from the second value to the first value after the work is completed.

Similarly, the factors that increase the stress value determined by the mental stress estimation unit 31 are that the pulse rate / heart rate is higher than normal, that the pulse wave and heart rate vary widely, and that the amount of conversation and activity is large. Includes less, longer or more frequent changes to specific voice qualities such as anger and tension, short sleep times, poor sleep quality, and high environmental discomfort. obtain.

Further, the stress value is not only the sensor data acquired by the portable device 2-7 and the wearable device 2-8, but also the other device 2 (for example, an in-vehicle device, a stationary device, a toilet type device, an autonomous driving type). It may also be determined using the sensor data acquired by the device or the like).

<Stationary computer>
An example in which the stress index of the user who uses the stationary computer 2-9 is estimated will be described with reference to FIGS. 17 and 18. The user is sensed by a sensor provided in the stationary computer 2-9.

FIG. 17 shows an example of the appearance of the stationary computer 2-9. FIG. 17 illustrates a case where the stationary computer 2-9 is a notebook computer. This stationary computer 2-9 is used, for example, during the working hours of the user. The stationary computer 2-9 has, for example, sensors such as a power button 651, a camera 652, a keyboard 653, a touch pad 654, and an input operation panel 655.

As shown in FIG. 17, the stationary computer 2-9 is composed of a computer main body 68 and a display unit 69. A display 205 is incorporated in the display unit 69. The display unit 69 is rotatably attached to the computer main body 68 between an open position where the upper surface of the computer main body 68 is exposed and a closed position which covers the upper surface of the computer main body 68.

The computer body 68 has a thin box-shaped housing, and on the upper surface thereof, a keyboard 653, a power button 651 for powering on / off the stationary computer 2-9, a touch pad 654, and an input operation. A panel 655, a speaker 206, and the like are arranged. Various operation buttons are provided on the input operation panel 655. Further, as described above with reference to FIG. 6, the computer main body 68 contains a CPU 201, a main memory 202, a non-volatile memory 203, a communication module 204, and the like.

Next, the functional configuration of the stationary computer 2-9 will be described with reference to FIG. The stationary computer 2-9 executes the stress monitoring program 202B. Hereinafter, first, the functional configuration of the sensor processing unit 251 included in the stress monitoring program 202B will be described.

The sensor processing unit 251 includes an on / off monitoring unit 661, a face detection unit 662, a video pulse wave calculation unit 663, a pulse wave peak interval calculation unit 664, an autonomic nerve analysis unit 665, a face authentication login unit 666, and a status monitoring unit 667. , Mail monitoring unit 668, and communication tool monitoring unit 669.

The on / off monitoring unit 661 detects whether the stationary computer 2-9 is powered on or off. The on / off monitoring unit 661 stores the access log information to the device including the date and time when the power is turned on and the date and time when the power is turned off in the attendance database 351. In the attendance database 351, in addition to the date and time when the power is turned on and the date and time when the power is turned off, attendance information including secondary information such as attendance time, overtime hours, and absenteeism based on the date and time when the power is turned on / off is stored. To.

The face detection unit 662, the image pulse wave calculation unit 663, the pulse wave peak interval calculation unit 664, and the autonomic nerve analysis unit 665 are the face detection unit 512 and the image pulse wave calculation unit 513, respectively, with reference to FIG. It operates in the same manner as the pulse wave peak interval calculation unit 514 and the autonomic nerve analysis unit 515. Therefore, the autonomic nerve analysis unit 665 transmits the biometric information including the autonomic nerve index indicating the pulse rate and the variation of the pulse to the mental stress estimation unit 31.

Further, the face recognition login unit 666 performs a login process to the stationary computer 2-9 using the face image detected by the face detection unit 662, and performs a login process to the device indicating that the login by face recognition has been performed. The operation information is stored in the operation database 352. Based on this operation information, the operation database 352 stores, for example, data indicating the frequency of login.

The status monitoring unit 667 monitors the user's status based on operations using input devices such as a keyboard 653, a touch pad 654, and an input operation panel 655. The status monitoring unit 667 detects the length of the non-operation time based on the operation using the input device, for example. The status monitoring unit 667 stores the operation information including the length of the non-operation time in the operation database 352.

The mail monitoring unit 668 detects that a mail has been sent or received by the mailer, and calculates the number of times the mail is sent and received and the frequency of sending and receiving the mail. The mail monitoring unit 668 stores the calculated number of times of sending and receiving mail and the frequency of sending and receiving mail in the mail frequency database 353.

The communication tool monitoring unit 669 detects that a communication tool such as a VoIP phone or chat software has been used, and calculates the number of times the tool is used and the frequency of use of the tool. The communication tool monitoring unit 669 stores the calculated number of times and frequency of use of the tool in the communication frequency database 354.

The data stored in the mail frequency database 353 and the communication frequency database 354 may be integrated and stored in the operation database 352. Further, the attendance database 351, the operation database 352, the mail frequency database 353, and the communication frequency database 354 are one of the sensor databases 35 shown in FIG. 7, and are provided in, for example, the server 3. Each database 351 to 354 may be provided in a server different from the server 3 as long as it can be accessed by the server 3.

The mental stress estimation unit 31 (factor determination unit 312) uses various diagnosis / test results such as industrial doctor diagnosis and depression diagnosis, history, questionnaire, questionnaire, etc. to determine a specific disease (for example, depression). Extract sensor data and genomic information of cases of onset, and learn factors that are highly relevant to the onset of the disease. Various diagnoses / test results such as industrial physician diagnosis and depression diagnosis, medical history, questionnaires, questionnaires, etc. are obtained from, for example, the health record database 4. Then, the mental stress estimation unit 31 uses these factors to generate a function for calculating the stress value. The mental stress estimation unit 31 can estimate the user's stress value according to the actual case by calculating the stress value based on the newly obtained sensor data or the like by using this function.

More specifically, the mental stress estimation unit 31 extracts attendance data of cases in which a specific disease has developed from the attendance database 351. The mental stress estimation unit 31 extracts, for example, data before and after the diagnosis of a user diagnosed with depression from the attendance database 351. The extracted data includes a plurality of attendance factors such as attendance hours, overtime hours, and the number of days absent without notice. The mental stress estimation unit 31 learns whether or not each attendance factor has a risk associated with the onset and the degree of the risk by using the data of a large number of onset cases. The mental stress estimation unit 31 determines, among the plurality of attendance factors, a factor having a risk related to the onset as a factor used for determining the value of long-term stress. Further, the mental stress estimation unit 31 may generate a function for calculating a long-term stress value by using the degree of risk corresponding to each factor. The mental stress estimation unit 31 uses this function to calculate a long-term stress value based on the attendance data of a certain user. Based on this long-term stress value, it is possible to determine whether or not the attendance status of the user is likely to develop a specific disease.

Further, the mental stress estimation unit 31 may extract operation data of a case of developing a specific disease from the operation database 352, the mail frequency database 353, and the communication frequency database 354. The mental stress estimation unit 31 extracts, for example, data of a user diagnosed with depression before and after the diagnosis from these databases 352,353,354. The extracted data includes multiple operation factors such as login frequency (number of times), length of non-operation time, email transmission / reception frequency (number of times), and communication tool usage frequency (number of times). .. The mental stress estimation unit 31 learns whether or not each operating factor has a risk associated with the onset and the degree of the risk by using the data of a large number of onset cases. The mental stress estimation unit 31 determines, among the plurality of operational factors, a factor having a risk related to the onset as a factor used for determining the value of long-term stress. Further, the mental stress estimation unit 31 may generate a function for calculating a long-term stress value by using the degree of risk corresponding to each factor. The mental stress estimation unit 31 (stress value determination unit 313) uses this function to calculate a long-term stress value based on the operation data of a certain user. Based on this long-term stress value, it is possible to determine whether or not the operation situation by the user is a situation in which a specific disease is likely to occur.

The mental stress estimation unit 31 also uses various data such as biometric information, attendance data, and operation data, and determines the factor used for determining the long-term stress value in the same manner as the above method, and determines the factor to be used for determining the long-term stress value. You may generate a function to calculate the stress value. Further, the mental stress estimation unit 31 may calculate a long-term stress value based on a factor for each type such as an attendance factor and an operation factor, and then calculate a value in which a plurality of long-term stress values are integrated.

Not only attendance data and operation data but also various sensor data can be used for determining factors having a risk related to such onset and generating functions. For example, even when the sensor data acquired by another device 2 (for example, a portable device, a wearable device, a toilet type device, an autonomous driving type device, etc.) is used, the mental stress estimation unit 31 similarly causes the onset. Factors with risks involved can be determined and a function can be generated to calculate long-term stress values.

In addition, the mental stress estimation unit 31 determines the magnitude of stress based on at least one of stress determination by a doctor, stress determination based on a questionnaire, stress determination based on urine, and stress determination based on saliva. Using the degree and the degree of fluctuation of multiple factors (factors) related to stress, one or more of these multiple factors is determined as the factor to be used for determining long-term stress or calculating the long-term stress value. You may. Multiple factors are one or more factors related to at least one of the human heartbeat or pulse, one or more factors related to the operation status of electronic devices, one or more factors related to attendance status, one or more factors related to emotions, and the human body. Includes at least two or more of one or more factors related to movement status, one or more factors related to speech status, one or more factors related to sleep status, and one or more factors related to environment. The stress judgment by the doctor, the stress judgment based on the questionnaire, the stress judgment based on urine, and the stress judgment based on saliva (data showing the judgment result) are, for example, in the health record database 4 at the time of diagnosis or examination. The stored mental stress estimation unit 31 can acquire data indicating this determination result from the health record database 4.

<Security Gate>
An example in which the stress index of the user who uses the security gate 2-10 is estimated will be described with reference to FIGS. 19 and 20. The security gate 2-10 is a kind of stationary device and is installed at the entrance / exit of a building or a room.

As shown in FIG. 19, the security gate 2-10 is composed of a housing 70 constituting the gate portion and a camera 702. An IC tag receiving portion 701 is provided on the upper surface of the housing 70 so that a user who intends to pass through the security gate 2-10 contacts or brings the IC card 701A into contact with or close to the housing 70. The IC tag receiving unit 701 is a kind of UI sensor. The IC tag receiving unit 701 reads data from the contacted (proximity) IC card 701A. Further, the CPU 201, the main memory 202, the non-volatile memory 203, the communication module 204, and the like are built in the housing 70 as described above with reference to FIG. The camera 702 is attached to a position where a user who is about to pass through the security gate 2-10 can be photographed. The video data acquired by shooting with the camera 702 is processed by, for example, the CPU 201 in the housing 70.

Next, the functional configuration of the security gate 2-10 will be described with reference to FIG. Security Gate 2-10 executes the stress monitoring program 202B. Hereinafter, first, the functional configuration of the sensor processing unit 251 included in the stress monitoring program 202B will be described. The sensor processing unit 251 includes an IC card authentication unit 711, a face detection unit 712, a face authentication unit 713, a video pulse wave calculation unit 714, a pulse wave peak interval calculation unit 715, an autonomous nerve analysis unit 716, and a facial expression recognition unit 717. include.

The IC card authentication unit 711 performs a user authentication process using the data in the IC card 701A read by the IC tag receiving unit 701. The IC card authentication unit 711 stores the access log information indicating that the authenticated user has passed the security gate 2-10 in the attendance database 351 or the activity database 355. As a result, information about the activity such as whether or not the user is active is stored in the attendance database 351 or the activity database 355. In addition, by authenticating at the security gate 2-10 installed at the entrance / exit of a building or room, for example, the user's entry or exit is detected, and the attendance time or leaving time is input to the attendance database 351 like a time clock. You can also.

The face detection unit 712 detects a face from an image obtained by using the camera 702. The face detection unit 712 detects the face image on the video frame by detecting, for example, a region having a feature amount similar to the face feature amount prepared in advance from each of the continuous video frames constituting the video. .. The face detection unit 712 outputs information about the detected face image to the face recognition unit 713, the image pulse wave calculation unit 714, and the facial expression recognition unit 717.

The face recognition unit 713 performs face recognition processing using the face image detected by the face detection unit 712. The face recognition unit 713 identifies a person who intends to pass through the security gate 2-10 by using a plurality of facial features corresponding to a plurality of persons (for example, employees) prepared in advance. Similar to the IC card authentication unit 711, the face authentication unit 713 can store the access log information indicating that the specified user has passed the security gate 2-10 in the attendance database 351 or the activity database 355.

The image pulse wave calculation unit 714, the pulse wave peak interval calculation unit 715, the autonomic nerve analysis unit 716, and the facial expression recognition unit 717 are the image pulse wave calculation unit 513 and the pulse wave peak interval described above with reference to FIG. 9, respectively. It operates in the same manner as the calculation unit 514, the autonomic nerve analysis unit 515, and the facial expression recognition unit 516. Therefore, the autonomic nerve analysis unit 716 transmits biometric information including an autonomic nerve index indicating a pulse rate and a variation in the pulse to the mental stress estimation unit 31. Further, the facial expression recognition unit 717 transmits emotional information including the recognized facial expression type (for example, emotions and sorrows) to the mental stress estimation unit 31.

The attendance database 351 and the activity database 355 are one of the sensor databases 35 shown in FIG. 7, and are provided in, for example, the server 3. Each database 351 and 355 may be provided in a server different from the server 3 as long as it can be accessed by the server 3.

The mental stress estimation unit 31 (factor determination unit 312) has developed a specific disease from the attendance database 351 (or activity database 355) in the same manner as the example of the stationary computer 2-9 shown in FIG. Extract attendance data (or activity data). The mental stress estimation unit 31 extracts, for example, data before and after the diagnosis of a user diagnosed with depression from the attendance database 351. The extracted data includes a plurality of attendance factors such as attendance hours, overtime hours, and the number of days without notice. The data obtained by analyzing the extracted data may be used as a factor. For example, the variation in attendance time calculated based on the data indicating the attendance time can be used as a factor.

The mental stress estimation unit 31 learns whether or not each attendance factor has a risk associated with the onset and the degree of the risk by using the data of a large number of onset cases. The mental stress estimation unit 31 determines, among the plurality of attendance factors, a factor having a risk related to the onset as a factor used for determining the value of long-term stress. Further, the mental stress estimation unit 31 may generate a function for calculating a long-term stress value by using the degree of risk corresponding to each factor.

The mental stress estimation unit 31 (stress value determination unit 313) uses this function to calculate a long-term stress value based on the attendance data of a certain user. Based on this long-term stress value, it is possible to determine whether or not the attendance status of the user is likely to develop a specific disease.

<Autonomous mobile robot>
An example in which the stress index of the user who uses the autonomous mobile robot 2-11 is estimated will be described with reference to FIGS. 21 and 22. The autonomous mobile robot 2-11 is, for example, a home robot used in a home.

FIG. 21 shows an example of the appearance of the autonomous mobile robot 2-11. FIG. 21 illustrates a humanoid autonomous mobile robot 2-11. The autonomous mobile robot 2-11 has, for example, sensors such as a microphone 751, a camera 752, a humidity sensor 753, and a temperature sensor 754.

The autonomous mobile robot 2-11 is provided with a microphone 751, a camera 752, and a speaker 206 on the head 75, and a humidity sensor 753 and a temperature sensor 754 on the body portion 76. A display (touch screen display) 205 is attached to the front surface of the body portion 76. Further, the CPU 201, the main memory 202, the non-volatile memory 203, the communication module 204, and the like are built in the head portion 75 and / or the body portion 76 as described above with reference to FIG.

The microphone 751 collects voice including utterances by users around the autonomous mobile robot 2-11. The camera 752 photographs the surroundings (for example, forward) of the autonomous mobile robot 2-11. The autonomous mobile robot 2-11 can perform a process such that the user turns his face toward the camera 752. For example, talking to the user with the voice output from the speaker 206 by interactive processing, making a cute gesture by the movement of the arms 77, 78 connected to the head 75 and the body 76, and displaying the user on the screen of the display 205. It is possible to display interesting images. The humidity sensor 753 and the temperature sensor 754 detect humidity and temperature.

Next, with reference to FIG. 22, the functional configuration of the autonomous mobile robot 2-11 will be described. The autonomous mobile robot 2-11 executes the stress monitoring program 202B. Hereinafter, first, the functional configuration of the sensor processing unit 251 included in the stress monitoring program 202B will be described. The sensor processing unit 251 includes a conversation volume estimation unit 761, a voice quality change detection unit 762, a face detection unit 763, a video pulse wave calculation unit 764, a pulse wave peak interval calculation unit 765, an autonomous nerve analysis unit 766, and a facial expression recognition unit 767. And the environmental discomfort calculation unit 768 are included.

The conversation amount estimation unit 761 and the voice quality change detection unit 762 operate in the same manner as the conversation amount estimation unit 611 and the voice quality change detection unit 612 described above with reference to FIG. 16, respectively. The face detection unit 763, the image pulse wave calculation unit 764, the pulse wave peak interval calculation unit 765, the autonomic nerve analysis unit 766, the facial expression recognition unit 767, and the environmental discomfort calculation unit 768 are described above with reference to FIG. 9, respectively. It operates in the same manner as the face detection unit 512, the image pulse wave calculation unit 513, the pulse wave peak interval calculation unit 514, the autonomic nerve analysis unit 515, the facial expression recognition unit 516, and the environmental discomfort calculation unit 519.

Therefore, the conversation amount estimation unit 761 transmits the utterance information indicating the conversation amount (or the utterance amount) to the mental stress estimation unit 31. The voice quality change detection unit 762 transmits the utterance information indicating the detected change in voice quality to the mental stress estimation unit 31. The autonomic nerve analysis unit 766 transmits biometric information including an autonomic nerve index indicating a pulse rate and a variation in the pulse rate to the mental stress estimation unit 31. The facial expression recognition unit 767 transmits emotional information including the recognized facial expression type (for example, emotions and sorrows) to the mental stress estimation unit 31. Further, the environmental discomfort calculation unit 768 transmits environmental information including the calculated environmental discomfort (for example, PMV, WBGT, etc.) to the mental stress estimation unit 31. The transmitted utterance information, biological information, emotional information, and environmental information are stored in, for example, the sensor database 35 in the server 3.

The mental stress estimation unit 31 uses the data stored in the sensor database 35 and is based on at least one of a plurality of factors such as conversation volume, voice quality change, autonomic nervous index, facial expression type, and environmental discomfort. Then, the magnitude of stress (for example, long-term stress value) is determined. More specifically, the mental stress estimation unit 31 (factor determination unit 312) extracts sensor data of a case of developing a specific disease from the sensor database 35. The mental stress estimation unit 31 extracts, for example, data before and after the diagnosis of a user diagnosed with depression from the attendance database 351. The extracted data includes a plurality of factors such as conversation volume, voice quality change, autonomic nervous index, facial expression type, and environmental discomfort. The data obtained by analyzing the extracted data may be used as a factor. For example, the number and frequency of smiles calculated based on the data indicating the type of facial expression can be used as a factor.

The mental stress estimation unit 31 learns whether or not there is a risk that each factor is involved in the onset and the degree of the risk by using the data of a large number of onset cases. The mental stress estimation unit 31 determines, among the plurality of factors, a factor having a risk related to the onset as a factor used for determining the value of long-term stress. Further, the mental stress estimation unit 31 may generate a function for calculating a long-term stress value by using the degree of risk corresponding to each factor.

The mental stress estimation unit 31 (stress value determination unit 313) uses this function to calculate a long-term stress value based on the sensor data of a certain user. Based on this long-term stress value, it is possible to determine whether the user is predisposed to develop a particular disease.

<Toilet type device>
An example in which the stress index of the user who uses the toilet type devices 2-12 and 2-13 is estimated will be described with reference to FIGS. 23 to 26. It should be noted that this user may be wearing a wearable device (for example, a wrist device) including a microphone or the like.

FIG. 23 shows an example of the appearance of the first toilet-type device 2-12. FIG. 23 illustrates a case where the toilet type device 2-12 has the shape of a urinal.

The toilet type device 2-12 is composed of a toilet bowl 83 and a camera 803. A water wash button 804 and a motion sensor 801 are provided on the upper part of the toilet bowl 83, and a urine sensor 802 is provided on the lower part of the toilet bowl 83. The motion sensor 801 is, for example, an infrared (IR) sensor, and detects that the user has approached the toilet bowl 83. The urine sensor 802 is a kind of biosensor (biomarker), and detects a component contained in urine excreted in the toilet bowl 83 and an index (for example, concentration) related to the component. The camera 803 is installed so as to capture the face of the user who uses the toilet bowl 83. Further, the toilet bowl 83 has a built-in or attached module including a CPU 201, a main memory 202, a non-volatile memory 203, a communication module 204, and the like, as described above with reference to FIG.

Next, the functional configuration of the toilet-type device 2-12 will be described with reference to FIG. 24. The toilet-type device 2-12 executes the stress monitoring program 202B. Hereinafter, first, the functional configuration of the sensor processing unit 251 included in the stress monitoring program 202B will be described. The sensor processing unit 251 includes a biomarker detection unit 811, a face detection unit 812, an image pulse wave calculation unit 813, a pulse wave peak interval calculation unit 814, and an autonomic nerve analysis unit 815.

The biomarker detection unit 811 acquires urine data from the urine sensor 802 when a manned person is detected by the motion sensor 801 and ends the acquisition of urine data when the water wash button 804 is pressed down. .. Urine data includes, for example, indicators of urinary catecholamines. The biomarker detection unit 811 transmits biomarker information including urine data to the mental stress estimation unit 31.

The face detection unit 812 detects the face from the image obtained by using the camera 803 from the time when the manned person is detected by the motion sensor 801 until the washing button 804 is pressed down. The face detection unit 812 detects the face image on the video frame by detecting, for example, a region having a feature amount similar to the face feature amount prepared in advance from each of the continuous video frames constituting the video. .. The face detection unit 812 outputs information about the detected face image to the image pulse wave calculation unit 813.

The image pulse wave calculation unit 813 calculates information about the user's pulse wave by using the information output by the face detection unit 812. More specifically, the video pulse wave calculation unit 813 calculates the pulse wave of the user based on the change in the face image (for example, the change in the color of the skin) detected from the continuous video frames.

The pulse wave peak interval calculation unit 814 calculates the calculated peak interval of the pulse wave. Then, the autonomic nerve analysis unit 815 calculates the autonomic nerve index indicating the pulse rate and the variation of the pulse by analyzing the calculated peak interval of the pulse wave. The autonomic nerve analysis unit 815 may calculate an autonomic nerve index for a specific period such as at least a part of the period during excretion and a period after excretion. The autonomic nerve analysis unit 815 transmits the biological information including the calculated autonomic nerve index to the mental stress estimation unit 31. The transmitted biomarker information and biomarker information are stored in, for example, the sensor database 35 in the server 3.

Further, FIG. 25 shows an example of the appearance of the second toilet-type device 2-13. FIG. 25 illustrates a case where the toilet type device 2-13 has the shape of a toilet bowl.

The toilet type device 2-13 is composed of a toilet bowl 871, a toilet seat 872, an operation panel 873, and the like. On the operation panel 873, some buttons such as the water wash button 873-1 are arranged. When the user uses the toilet type device 2-13, the toilet seat 872 is arranged on the upper part of the toilet bowl 871. The toilet seat 872 is provided with a contact sensor 851 at a position where the user comes into contact with the seated user, and an electrocardiographic sensor 853 at a position where the user sits in contact with the thigh. A urine sensor 852 is provided at the lower part of the toilet bowl 871. The contact sensor 851 detects that the user has touched (sit) the toilet seat 872. The electrocardiographic sensor 853 detects the heartbeat waveform (electrocardiogram) of the seated user. The urine sensor 852 detects a component contained in the urine excreted in the toilet bowl 871 and an index (for example, concentration, etc.) related to the component.

The functional configuration of the toilet-type device 2-13 will be described with reference to FIG. 26. The toilet-type device 2-13 executes the stress monitoring program 202B. Hereinafter, first, the functional configuration of the sensor processing unit 251 included in the stress monitoring program 202B will be described. The sensor processing unit 251 includes a biomarker detection unit 861, a heart rate peak interval calculation unit 862, and an autonomic nerve analysis unit 863.

The biomarker detection unit 811 acquires urine data from the urine sensor 852 in response to the detection of manned by the contact sensor 851, and ends the acquisition of urine data in response to the pressing of the water wash button 873-1. do. Urine data includes, for example, indicators of urinary catecholamines. The biomarker detection unit 861 transmits biomarker information including urine data to the mental stress estimation unit 31.

The heart rate peak interval calculation unit 862 calculates the heart rate interval, which is the peak interval of the waveform measured by the electrocardiographic sensor 853. The autonomic nerve analysis unit 863 calculates an autonomic nerve index indicating a heart rate and a variation in the heart rate by analyzing the calculated heart rate interval. The autonomic nerve analysis unit 863 may calculate an autonomic nerve index for a specific period such as at least a part of the period during excretion and a period after excretion. The autonomic nerve analysis unit 863 transmits the biological information including the calculated autonomic nerve index to the mental stress estimation unit 31. The transmitted biomarker information and biomarker information are stored in, for example, the sensor database 35 in the server 3.

Note that each toilet-type device 2-12, 2-13 may operate in cooperation with a wearable device (for example, wrist device 2-5, pants-type device 2-8) worn by the user. These toilet-type devices 2-12 and 2-13 and the wearable device transmit and receive data by short-range wireless communication. In that case, the wearable device recognizes the water wash sound from the voice data obtained by the microphone provided in the wearable device, and notifies the toilet type devices 2-12 and 2-13 that the water wash sound is recognized. .. Based on this notification, the biomarker detection unit 811, 861, the face detection unit 812, and the heart rate peak interval calculation unit 862 can also end the acquisition of urine data and the face detection from the video. The toilet-type devices 2-12 and 2-13 may receive voice data or voice features from the wearable device and recognize the washing sound on the devices 2-12 and 2-13.

The mental stress estimation unit 31 (stress value determination unit 313) calculates short-term stress values before, during, and after excretion (urination), for example, using biomarker information and biometric information. Since the user tends to have a normal mind (that is, a psychologically absent state) during at least a part of the period (for example, the latter half) during excretion and the period after excretion, the mental stress estimation unit 31 is in charge of that period. The short-term stress value that serves as a reference (baseline) can be calculated using the biological information. This stress value is, for example, a short-term stress value that serves as a reference for each day. An index of urine catecholamine may be obtained by using a biomarker by a prior urinalysis or the like, and this may be used as a reference value (reference).

The mental stress estimation unit 31 determines the presence or absence of long-term stress and the magnitude of long-term stress (for example, long-term stress value) based on the stress value during at least a part of the excretion period or the post-excretion period. Further, the mental stress estimation unit 31 uses the short-term stress value before excretion, the short-term stress value during excretion, and the short-term stress value after excretion, and the long-term stress is based on the difference and the degree of change between the values. The presence or absence of stress and the magnitude of long-term stress may be determined. For example, if the short-term stress value does not decrease before and after excretion, it is determined that there is long-term stress (chronic stress), and the smaller the difference between these short-term stress values, the greater the long-term stress.

Further, the mental stress estimation unit 31 (factor determination unit 312) may generate a function for calculating a long-term stress value using biomarker information and biometric information. The index of urinary catecholamine included in the biomarker information is used as an index showing chronic stress in the medical field and the like. Therefore, the mental stress estimation unit 31 compares the index of urinary catecholamine with other factors and searches for a promising factor related to chronic stress. The other factors are not limited to biological factors such as the autonomic nerve index obtained by the toilet-type device 2-12, but also the above-mentioned attendance factor, operation factor, environmental factor such as environmental discomfort, and facial expression type. Any factor based on sensor data can be included, such as emotional factors such as, speech volume and speech factors such as changes in voice quality. The mental stress estimation unit 31 identifies, for example, a factor associated with (linked) to the index of urinary catecholamines, and calculates the degree of the association.

The mental stress estimation unit 31 determines, among the plurality of factors, the factor related to the index of urinary catecholamine as the factor used for determining the value of long-term stress. In addition, the mental stress estimation unit 31 may generate a function for calculating a long-term stress value by using the degree of association corresponding to each factor. The mental stress estimation unit 31 (stress value determination unit 313) can use this function to calculate a long-term stress value based on the sensor data of a certain user.

The stress estimation system 1 estimates the stress value of the user by using the sensor data acquired by at least one of the plurality of devices 2 such as the exemplified devices 2-5 to 2-13, and determines the stress value and the stress. Information for reduction can be provided. Further, the stress estimation system 1 can integrate a plurality of sensor data obtained from the plurality of devices 2 to estimate the stress value of the user and provide information for estimating the stress value and reducing the stress.

As described above, according to the present embodiment, the degree of mental stress can be estimated with high accuracy. The mental stress estimation unit 31 ends driving the car when the short-term stress before starting the driving of the car is the first value and the short-term stress while driving the car is the second value larger than the first value. The magnitude of the long-term stress is determined according to the change in the stress value from the second value to the first value. As a result, the magnitude of long-term stress can be determined according to the ease of return after the end of operation from the short-term stress value during operation (second value) to the normal short-term stress value (first value) (before the start of operation). Since it can be determined, the degree of mental stress can be estimated with high accuracy.

Further, each of the various functions described in the present embodiment may be realized by a circuit (processing circuit). Examples of processing circuits include programmed processors such as central processing units (CPUs). This processor executes each of the described functions by executing a computer program (instruction group) stored in the memory. This processor may be a microprocessor including an electric circuit. Examples of processing circuits also include digital signal processors (DSPs), application specific integrated circuits (ASICs), microcontrollers, controllers, and other electrical circuit components. Each of the components other than the CPU described in this embodiment may also be realized by the processing circuit.

Further, since various processes of the present embodiment can be realized by a computer program, the present embodiment and the present embodiment can be obtained by simply installing and executing the computer program on a computer through a computer-readable storage medium in which the computer program is stored. A similar effect can be easily achieved.

Although some embodiments of the present invention have been described, these embodiments are presented as examples and are not intended to limit the scope of the invention. These novel embodiments can be implemented in various other embodiments, and various omissions, replacements, and changes can be made without departing from the gist of the invention. These embodiments and variations thereof are included in the scope and gist of the invention, and are also included in the scope of the invention described in the claims and the equivalent scope thereof.

1 ... stress estimation system, 15 ... network, 2 ... sensing device, 3 ... server (cloud inspection service), 4 ... health record database (PHR), 201 ... CPU, 202 ... main memory, 202A ... OS, 202B ... stress monitoring Program, 203 ... Non-volatile memory, 204 ... Communication module, 205 ... Display, 206 ... Speaker, 251 ... Sensor processing unit, 252 ... Display control unit, 301 ... CPU, 302 ... Main memory, 302A ... OS, 302B ... Stress estimation Program, 303 ... Non-volatile memory, 304 ... BIOS-ROM, 305 ... Communication module, 31 ... Mental stress estimation unit, 32 ... Behavioral transformation determination unit, 311 ... Accumulation processing unit, 312 ... Factor determination unit, 313 ... Stress value determination Department, 314 ... Alert generation unit, 315 ... Schedule generation unit.

Claims (17)

The second short-term stress after the first work is completed by using the first short- term stress value of the user before starting the first work and the second short -term stress value of the user during the first work. A system provided with a determination means for determining the magnitude of long- term stress of the user according to a change in the stress value from the value toward the first short-term stress value . The determination means further uses information about the frequency of operation of the electronic device to determine the magnitude of the user's long- term stress.
The system according to claim 1, wherein the electronic device is a portable device or a human body-worn device. The system according to claim 1 or 2, wherein the determination means further uses information regarding repeated fluctuations of the human body above a threshold value to determine the magnitude of long- term stress of the user. Using the degree of variation in the magnitude of stress according to the first method and the degree of variation in a plurality of factors related to stress, the determination means is a short- term method of the user regarding one or more of the plurality of factors. The stress value is used to determine the magnitude of the user's long- term stress.
The first method is a method of determining stress based on at least one of a doctor's determination of stress, a questionnaire-based stress determination, a urine-based stress determination, and a saliva-based stress determination.
The plurality of factors include one or more factors related to at least one of human heartbeat or pulse, one or more factors related to the operation status of electronic devices, one or more factors related to attendance status, one or more factors related to emotions, and movements of the human body. One of claims 1 to 3, which includes at least two of one or more factors related to the situation, one or more factors related to the speech situation, one or more factors related to the sleeping situation, and one or more factors related to the environment. The system described in. The determination means according to claim 1 to 4, wherein the magnitude of the long- term stress of the user is determined by further using the short- term stress value of the user during at least a part of the period during excretion or the period after excretion. The system described in any one of the items. The system according to any one of claims 1 to 5, further comprising an output means for outputting a warning to the user according to the magnitude of the long- term stress of the user determined by the determination means. Claims 1 to 5 further include an output means for outputting information on the behavior recommended for the user to reduce the stress according to the magnitude of the long- term stress of the user determined by the determination means. The system described in any one of the items. The system according to any one of claims 1 to 5, further comprising an output means for outputting information on a recommended work style according to the magnitude of the long- term stress of the user determined by the determination means. .. The second short-term stress after the first work is completed by using the first short- term stress value of the user before starting the first work and the second short -term stress value of the user during the first work. A method in which a computer determines the magnitude of long- term stress of the user according to a change in the stress value from the value toward the first short-term stress value . The determination comprises determining the magnitude of long- term stress of the user by the computer using further information about the frequency of operation of the electronic device.
The method according to claim 9, wherein the electronic device is a portable device or a human body-worn device. The method according to claim 9 or 10, wherein the determination comprises the computer determining the magnitude of long- term stress of the user by further using the information regarding the repetition of the fluctuation of the human body above the threshold value. .. The determination is made by the computer using one or more of the plurality of factors, using the degree of variation in the magnitude of stress according to the first method and the degree of variation in a plurality of factors related to stress. Including using the short- term stress value of the user with respect to the determination of the magnitude of the long- term stress of the user.
The first method is a method of determining stress based on at least one of a doctor's determination of stress, a questionnaire-based stress determination, a urine-based stress determination, and a saliva-based stress determination.
The plurality of factors include one or more factors related to at least one of human heartbeat or pulse, one or more factors related to the operation status of electronic devices, one or more factors related to attendance status, one or more factors related to emotions, and movements of the human body. One of claims 9 to 11, which includes at least two of one or more factors related to the situation, one or more factors related to the speech situation, one or more factors related to the sleeping situation, and one or more factors related to the environment. The method described in. The determination includes determining the magnitude of the user's long- term stress by the computer using the short- term stress value of the user for at least a part of the period during excretion or the period after excretion. The method according to any one of claims 9 to 12. The method according to any one of claims 9 to 13, wherein the computer outputs a warning to the user according to the magnitude of the long- term stress of the user determined by the determination. Claims 9 to 13 where the computer outputs information about the behavior recommended for the user to reduce the stress according to the magnitude of the long- term stress of the user determined by the determination. The method described in any one of the items. The method according to any one of claims 9 to 13, wherein the computer outputs information regarding a recommended work style according to the magnitude of the long- term stress of the user determined by the determination. .. A program executed by a computer, said program
The second short-term stress after the first work is completed by using the first short- term stress value of the user before starting the first work and the second short -term stress value of the user during the first work. A program that causes the computer to execute a procedure for determining the magnitude of long- term stress of the user according to a change in the stress value from the value toward the first short-term stress value .

Images